0.1 release status
What the local alpha supports and what it deliberately leaves out.
CrashLab 0.1 is an open-source alpha for one complete local development loop: set up an evaluation, run the real agent in Docker, understand the evidence, change the agent, and compare the candidate with a baseline.
Supported path
- Install CrashLab from
crashlabsai/tapwith Homebrew. - Install the bundled coding-agent setup skill.
- Connect one repository-owned evaluation to the real agent.
- Run fresh trials in local Docker.
- Diagnose the retained result under
.crashlabs/runs/. - Run a baseline/candidate regression check.
The Homebrew formula installs the Python CLI, Ink workspace, uv, and their
managed Python and Node runtimes on macOS Apple Silicon and Intel. It retains the
exact release wheel so locked evaluation packages can create fresh runtimes
without a separate PyPI release. The release archive also retains CrashLab's MIT
license and the viewer's generated third-party notices. Every normal simulation
cleans the ephemeral Docker resources it owns.
Capability map
| Capability | 0.1 status | What that means |
|---|---|---|
| Local Docker execution | Supported | Docker Engine/Desktop and Buildx are required. |
run, focused reruns, and check | Supported alpha | They execute the configured real target and retain results automatically. |
| Interactive workspace | Supported | Bare crashlab opens coverage, runs, and past evaluations without a second execution engine. |
| Python/uv, Python/requirements, and Node/pnpm source shapes | Supported alpha | Setup still has to connect a real repository agent, world, and verifier. |
| Deterministic protected verification | Supported alpha | World state, invariants, and causal events stay outside the target. |
| Semantic verification | Supported alpha | A customer-selected protected judge can evaluate bounded evidence; uncertainty never silently passes. |
| Reliability, latency, tokens, and cost | Supported alpha | Missing provider data is shown as unavailable, not zero. |
| Remote sandboxes | Not included | Docker is the only supported 0.1 execution backend. |
| Linux and Windows packages | Not included | The initial Homebrew package is macOS-only. |
What setup proves
A generated folder is not a completed setup. The evaluation must import an existing source-controlled agent, run it against a functional stateful world, and retain at least one protected result. CrashLab refuses to treat a scripted control or an agent invented inside the evaluator as customer evidence.
Clean distribution exercises have completed that loop against unrelated Python and TypeScript/JavaScript applications. They also caught a semantic-only target regression, guided a coding agent to the failed behavior, and passed a repaired baseline/candidate comparison. These exercises support the local workflow; they do not make one trial statistically conclusive for every agent.
Alpha boundaries
CrashLab 0.1 does not claim that:
crashlab initcan infer an arbitrary product workflow by itself;- a source scaffold is a finished evaluation;
- one passing trial proves production reliability;
- unavailable required verification can be skipped;
- plausible final text proves that a tool or mutation executed;
- unreviewed Python evaluation packages are safe to import; or
- current alpha schemas are stable 1.0 contracts.
Evaluation packages should import from crashlab.alpha, keep a current
uv.lock, and run crashlab eval validate during upgrades. Use
crashlab view <stored-run> --json for a bounded diagnosis and --raw only when
you need the complete authoritative evidence.
Report a release-path problem
Include the CrashLab version, host platform, exact command and exit code, retained selector or redacted diagnosis, and Docker/Buildx versions. Remove credentials, customer source, and sensitive evidence before opening an issue in the CrashLab repository.