CLI reference

Run crashlab <command> --help for the complete option list installed with your version. This page describes command intent and stable workflow expectations.

Repository workflow

crashlab inspect [project]

Read repository facts without changing files, starting Docker, or calling a model provider.

crashlab inspect .
crashlab inspect . --json

Repository inspection v4 reports compatible packaged source adapters. When one unambiguous repository-root shape matches and no evaluation package exists, authoring.recommended_scaffold is an exact argv array. The recommendation is a build-layout hint, never behavioral coverage.

crashlab init [project]

Validate the conventional crashlab/evaluations.py setup and print the exact next commands. It needs and creates no CrashLab configuration file. If the package is absent, use the setup skill or crashlab eval scaffold first.

crashlab init . --json

The crashlab.onboarding-result.v2 response returns its smoke run, smoke comparison, and reliability comparison as argv arrays. Execute each element as one process argument; do not join and reparse these commands through a shell.

crashlab run [scenario]

Without a positional scenario, run the real configured candidate from the current repository. One trial is the default; each repetition gets a fresh world.

crashlab run
crashlab run --repetitions 3
crashlab run --case support.priority
crashlab run --json

External evaluation packages require --trust-evaluation-code. The standalone scenario form exists for customer-supplied suites and development, but normal repository work should stay configuration-driven.

If a required host provider credential is missing, --json emits a bounded crashlab.command-error.v1 response with the missing environment-variable name and next steps. It exits before provisioning and never includes the credential value or invents an evaluation result.

crashlab check

Compare the configured baseline and candidate using the same selected cases, evaluation source, model configuration, and repetitions.

crashlab check
crashlab check --case support.priority
crashlab check --json

Explicit flags can override source revisions, evaluation source, target, sandbox, repetitions, and policy. Prefer the repository convention in ordinary work so humans, coding agents, and CI execute the same gate.

Retained results

crashlab runs

List recent results under the current project's .crashlabs/runs/ directory.

crashlab runs
crashlab runs --limit 20 --json

crashlab view [result]

Inspect the latest run, a stored ID prefix, or an explicit result path.

crashlab view
crashlab view <stored-run>
crashlab view <stored-run> --json
crashlab view <stored-run> --raw

• default output is a concise human report;
• --json is the bounded coding-agent diagnosis, including case and baseline/candidate context for comparison findings;
• --raw is the complete authoritative result; and
• --tui opens the Ink viewer installed with the Homebrew release.

Customer run and check commands do not accept --output. Retention is automatic so results remain discoverable and comparable.

Setup skill

crashlab skills install

Install the managed setup-crashlab skill for supported coding-agent clients.

crashlab skills install
crashlab skills install --agent all

Evaluation packages

crashlab eval scaffold

Create a new, intentionally incomplete Python or Node/pnpm authoring package without overwriting an existing directory:

crashlab eval scaffold --runtime python-uv --json
crashlab eval scaffold --runtime python-requirements \
  --target-project python-backend \
  --requirements-file requirements.txt \
  --json
crashlab eval scaffold --runtime node-pnpm --json

Replace its explicit coverage, world, target-hook, provider-gateway, and verifier boundaries before use. A scaffold does not count as setup completion.

crashlab eval validate

Validate a selected package revision, lock, entrypoint, and case list. External Python evaluator imports require --trust-evaluation-code.

crashlab eval validate \
  --trust-evaluation-code \
  --json

Advanced alpha namespaces

manifest, request, and batch expose the execution-manifest, content-addressed execution-request, and multi-request batch primitives. They are for evaluation/runtime authors and are not required to run a configured agent. Their current schemas are alpha contracts; use the installed --help and the repository architecture records when integrating them.

Output and exit behavior

When --json is selected, machine-readable result data is written to stdout without terminal color. run and check print the retained selector to stderr so automation can preserve both channels.