Set up an agent

Setup means more than creating a configuration file. It is complete when the real agent has run against a functional world, protected verification has produced an interpretable result, and CrashLab has retained that result.

If you have not selected the behavior yet, use Choose your first workflow before asking an agent to write the package.

Recommended: ask your coding agent

After installing the setup skill, open your agent repository and ask:

Set up CrashLab

The skill instructs the coding agent to:

1. read the repository instructions and architecture;
2. inspect facts without changing the project;
3. confirm the natural entrypoint, tools, state changes, and model settings;
4. connect or author one small, user-visible evaluation;
5. describe that evaluation's workflow, world, capabilities, and checks for pre-run coverage;
6. use the conventional visible crashlab/ project layout;
7. run the real candidate once;
8. inspect the retained result; and
9. commit setup before changing the agent and running a historical regression check.

The coding agent may create repository-specific world, target-adapter, and verifier code. That code belongs in a versioned evaluation package—not in CrashLab core.

The agent under test must already exist in repository source. The target hook imports an existing agent, factory, graph, application entrypoint, or runnable example. It must not invent a new agent, prompt, or tool graph and present that substitute as customer behavior. If the selected repository is only a framework or library and contains no runnable agent, choose the actual application repository or provide its intended entrypoint before continuing.

1. Inspect before writing

You can run the deterministic discovery step yourself:

crashlab inspect .
crashlab inspect . --json

Inspection reports repository facts, framework hints, test commands, an existing conventional evaluation package, and compatible packaged source shapes. It does not claim to understand domain behavior from filenames alone.

In JSON output, authoring.recommended_scaffold is either null or an exact argv array such as:

[
  "crashlab",
  "eval",
  "scaffold",
  "--runtime",
  "python-uv",
  "--json"
]

A coding agent should execute each array element as one process argument. It must not join the array into a shell command and reparse it. The recommendation only identifies a supported build layout. For an unambiguous nested Python project the array also includes --target-project and, when applicable, --requirements-file; it is not evidence that CrashLab has found the agent entrypoint or a useful workflow.

2. Reuse an existing package

If the repository contains crashlab/evaluations.py, CrashLab discovers it automatically; no initialization file is required.

Run initialization from the intended agent project root. In a monorepo, pass that project directory explicitly instead of maintaining a selector file:

crashlab init path/to/agent --json

When the conventional package already exists, initialization reports created: false and returns exact argv arrays under next_commands.smoke_run, smoke_check, and reliability_check. It does not create a CrashLab TOML file. The normal repository convention infers a candidate-owned evaluation relationship; lower-level authoring mechanisms can still bind a baseline-pinned or independently sourced package when the customer chooses.

3. Scaffold an unknown repository

If no evaluation package exists, use the exact recommended scaffold when one is present.

Python with uv or requirements

A root or nested uv project uses its existing pyproject.toml and uv.lock:

crashlab eval scaffold --runtime python-uv --json

The generated target_hook.py runs inside an immutable source snapshot whose production dependencies are installed with frozen uv resolution.

A root or nested requirements project uses the exact command returned by inspection, for example:

crashlab eval scaffold --runtime python-requirements \
  --target-project python-backend \
  --requirements-file requirements.txt \
  --json

The generated suite keeps that existing project directory on the target import path. Requirements resolution is recorded as unlocked. Do not create a new root pyproject.toml or lock file merely to fit CrashLab.

Repository-root Node with pnpm

This source shape requires repository-root package.json and pnpm-lock.yaml files plus a pinned packageManager value such as [email protected]:

crashlab eval scaffold --runtime node-pnpm --json

The evaluation package remains Python because its world and verifier run in the protected host runtime. The generated target_hook.mjs invokes the real JavaScript or TypeScript agent inside an immutable source snapshot installed with pnpm install --frozen-lockfile. CrashLab exposes that snapshot as CRASHLAB_TARGET_ROOT. A nested pnpm workspace should resolve packages from the package.json that owns the selected agent instead of assuming dependencies are hoisted beside the mounted hook.

4. Replace every incomplete boundary

Read the generated crashlab/README.md completely. The scaffold fails on purpose until a coding agent replaces all five boundaries:

1. Coverage: replace the placeholder title, summary, workflow, world resources, capabilities, and verifier descriptions with facts a developer can understand before a provider call.
2. World: seed the minimum realistic state and implement coherent dependency reads and writes.
3. Target hook: import the production agent, preserve its prompts and tool contracts, and replace only external side effects with gateway.call(...). Return its result with the generated explicit TargetOutput(summary=..., data=...) shape; do not guess field names from the framework's result object.
4. Provider gateway: keep real model credentials on the host and expose only short-lived CrashLab gateway bindings to the target.
5. Verifier: check protected final state and at least one collateral-damage invariant using the generated typed evidence example.

Remove a CrashLab scaffold incomplete failure only after the corresponding real implementation exists. A generated or merely validated scaffold is not setup evidence. Placeholder coverage is also incomplete even when the package imports successfully.

5. Lock and validate

uv lock --project crashlab

crashlab eval validate \
  --trust-evaluation-code \
  --json

The directory itself identifies the project and evaluation package. CrashLab infers the entrypoint, lock, current candidate, baseline, and normal regression defaults. Preserve the explicit --trust-evaluation-code acknowledgement when running it.

6. Prove one real workflow

Run one real trial, then inspect the printed stored selector:

crashlab run --repetitions 1 --trust-evaluation-code

crashlab view <stored-run> --json

Executing next_commands.smoke_check before an included target change is useful only as a same-source control/reliability sample. Candidate-owned evaluation code is intentionally shared across both variants. Commit the initial setup, make the intended agent change, and then run the configured comparison before treating the result as regression evidence. Run the configured reliability command when provider cost and time allow. Setup is complete only after the real target has produced a retained protected result under .crashlabs/runs/.

If a required host credential is missing, CrashLab returns a bounded command error before provisioning. Set the named variable on the host and rerun; never put its value in the repository, generated package, target environment, or evidence. See Local security and secrets for .env safety and the provider-gateway boundary.

The setup workflow remains alpha. A generated package, successful validation, or one provider-backed trial is not proof that onboarding is seamless for an unknown repository. The 0.1 release status records the current cross-language evidence and the remaining acceptance work without turning historical exercise details into tutorial steps.