Local security and secrets

Docker is the supported 0.1 sandbox backend. It isolates target code from the host-owned world and verifier, but it does not make all evaluation code untrusted.

Trust boundaries

An external Python evaluation package executes as trusted host code. Passing --trust-evaluation-code acknowledges this. A dependency lock improves reproducibility; it is not an operating-system sandbox. Review the package before trusting it: ordinary Python in that package can read the host process environment and files available to your user.

Target isolation

CrashLab's Docker target boundary is designed around:

• an unprivileged user;
• a read-only root filesystem and read-only target source;
• dropped Linux capabilities and no-new-privileges;
• bounded CPU, memory, process count, and wall time;
• temporary writable filesystem locations; and
• no network by default, unless a trusted adapter declares a constrained host gateway.

Evaluation packages should declare only the files, operations, and network paths the target needs. If a requested sandbox requirement cannot be enforced, CrashLab must report infrastructure uncertainty rather than weaken it silently.

Supply a credential locally

load_host_secret("NAME") reads one named value from the environment of the crashlab process and returns an opaque HostSecret. The value is not a CLI argument and a missing credential error names only the environment variable. Supply it through your shell session, an operating-system credential helper, or your CI secret store before starting CrashLab.

CrashLab does not search the repository for .env files or load one automatically. Evaluation code may explicitly pass an env_file path to load_host_secret; that loader resolves only the requested name. Because the evaluation package is trusted host code, review both the selected file and the code that selects it before using --trust-evaluation-code.

Keep .env local

If your team deliberately uses a repository-local .env file:

1. add .env and any secret-bearing variants to .gitignore before adding a value;
2. keep only names and empty placeholders in .env.example;
3. restrict the file to your local user where the platform supports it; and
4. run git check-ignore .env before the first evaluation—the command should print .env.

Working-tree snapshots include tracked files and unignored untracked files. They exclude a correctly ignored .env, but .gitignore does not remove a file that is already tracked. If a key was committed, copied into an image, or printed in evidence, remove it from the affected history where appropriate and rotate it. CrashLab cannot prove that arbitrary customer-authored files are secret-free.

Provider gateway

The real credential terminates at a host-owned provider gateway:

host secret -> allowlisted gateway -> provider
                    |
                    +-> private URL + short-lived trial token -> target SDK
                    +-> sanitized model, usage, latency, and cost evidence

The gateway constrains provider paths, models, request count, request and response sizes, and time. It does not retain prompt or generated-response content as provider-gateway evidence. The target still receives the provider response it requested, so evaluation-owned target output, tool events, and world state may contain sensitive customer data.

CrashLab rejects credential-like names from ordinary target environment bindings before provisioning. The provider gateway is the target-authoring boundary that accepts HostSecret.

If an SDK requires OPENAI_API_KEY, ANTHROPIC_API_KEY, or a similar variable, the gateway adapter may bind that name to a generated trial token and point the SDK at its private gateway URL. The sandbox still does not receive the provider key. The token is constrained to that gateway, model, request budget, and trial, then disappears during cleanup.

The target reaches the gateway through a relay attached only to its internal Docker network. On native Linux the authenticated host listener must be reachable through Docker's bridge gateway rather than host loopback. A request still has to present the random trial token before CrashLab invokes a bridged host process or forwards a provider call; the long-lived provider credential is never placed on that listener's target-facing side.

Provider endpoint authorization and SDK base-URL shape are separate. If the SDK expects a versioned API prefix and appends its own endpoint, the evaluation declares a target-facing path such as /v1; the gateway still authorizes only the resulting full path such as /v1/responses. Both are secret-free execution facts retained for reconciliation. Do not broaden the allowlist or modify the agent merely to compensate for an incorrectly shaped injected URL.

Treat retained results as customer data

CrashLab stores results under .crashlabs/runs/, creates an internal ignore rule for .crashlabs/, and uses private local file modes where the platform supports them. Results intentionally survive sandbox cleanup so you can inspect and compare them.

Results do not contain declared provider-key values, but they can contain agent output, tool arguments, world changes, and bounded failure diagnostics. Review crashlab view <stored-run> --raw before sharing a result, attaching it to an issue, or moving it into another storage system.

Automatic cleanup

A normal simulation command owns one ephemeral Docker resource scope covering all selected cases and repetitions. Resources can be reused inside that command, then CrashLab cleans them in a finally path after success, behavioral failure, infrastructure failure, or interruption.

CrashLab removes only resources it can prove it created:

• the command's dedicated Buildx builder and cache;
• images created or materialized by that scope, only while the tag still points to the recorded immutable image ID;
• per-trial containers and internal networks.

CrashLab never runs a global Docker system, image, volume, or builder prune. Caller-provided pre-existing images are not owned and are not removed.

Failure handling

Cleanup failures are reported explicitly. CrashLab will not delete a resource whose identity has changed since it was recorded. A host SIGKILL can bypass normal process cleanup; automatic cross-command orphan recovery is not yet a public 0.1 command.