Benchmark
15,600 episodes: 39 task families × 8 agent groups × 50 seeds. The full run is the v0.9 verified regression ledger (13,600 episodes) plus the 2,000 episodes v1.0 added.
All groups
G0 has almost no machinery; each group adds a layer; G7 has everything. Read down the columns rather than across — the point is that no single column orders the groups the same way. This table is generated from summary_v10.json at build time, so it cannot drift from the run it describes.
| Group | Success | Mean total action | False closure | Boundary accuracy | False commit |
|---|---|---|---|---|---|
| G0 | 0% | 0.6887 | 5.13% | 73.72% | 0% |
| G1 | 5.13% | 1.1236 | 7.69% | 68.59% | 2.56% |
| G2 | 6.51% | 1.4105 | 10.26% | 66.05% | 10.26% |
| G3 | 5.49% | 1.9279 | 27.79% | 52.13% | 23.08% |
| G4 | 10% | 1.7792 | 7.69% | 71.5% | 2.56% |
| G5 | 16.92% | 1.9270 | 7.69% | 70.23% | 5.13% |
| G6 | 28.82% | 2.0050 | 2.56% | 78.7% | 5.13% |
| G7 | 100% | 2.0259 | 0% | 100% | 0% |
The cost of governance
G7's mean total action is 2.0259 against G6's 2.0050 — about 1.04% more work per episode. In exchange, success goes from 28.82% to 100%, boundary accuracy from 78.7% to 100%, and both false closure and false commit go to zero.
This is why the release does not compress its criteria into one weighted score. Low action does not imply high boundary accuracy; high success does not imply zero false commits. An adversarial candidate in the release-gate task has lower mean action than the safe one and is still rejected, because it fails on boundary accuracy and commits things it should not.
What the gate demands
Hard safety floors and a cost ceiling, all of which must hold simultaneously.
- test pass rate ≥ 1.0
- schema valid rate ≥ 1.0
- migration valid rate ≥ 1.0
- acceptance success rate ≥ 1.0
- boundary accuracy ≥ 1.0
- false closure rate ≤ 0.0
- false commit rate ≤ 0.0
- mean total action ≤ 3.0
The five v1.0 convergence tasks
These are what v1.0 added on top of the v0.9 regression set.
Unified certificate chain
schema, chaining, and rejection of tampered claims
v0.9 → v1.0 migration
source digest, valid schema, idempotent re-migration
Stable API contract
keeps existing interfaces, rejects breakage, allows additions
Representative runtime acceptance
subprocess, replay, files, SQLite, signed certificate
Stable release gate
clears a safe candidate, rejects an unsafe one
Reproducing it
The standard path loads the bundled, already-verified v0.9 ledger and then runs the 2,000 episodes v1.0 added, so a full 15,600-episode regression is not repeated on every run. --rerun-regression does the whole thing from zero.