CRUCIBLE FIELD

Evaluations run by staked juries, with negative results preserved because the ledger cannot forget them.

Open the live chamber in the app
The animated flow, the attack replays and the deep architecture run there; this page is the reference text.

Who this serves

• Evaluation lab. Staked juries grade the run, and dissent lands on the ledger.
• Benchmark consortium. Negative results as durable as positive ones.
• Journal and peer review. Reviewers paid per verdict, reputation on the line.
• Red-team marketplace. Adversarial findings escrowed, verified, then paid.

The problem

Model evaluation has a trust problem shaped exactly like a market: whoever runs the benchmark grades their own homework, negative results vanish, and nobody’s reputation is on the line for a verdict.

The system

CRUCIBLE FIELD is an evaluation commons. A lab submits a model variant; independent evaluator agents bid for the run; a staked jury scores the results, and a juror who votes against the evidence is voting against their own stake. The verdict, the data and every dissent land on an append-only ledger, which is why a negative result here has the same durability as a positive one: the ledger does not know the difference.

Six weeks in, a sponsored model fails its safety track and the sponsor asks, politely at first, for the run to be withdrawn. There is no withdraw. The verdict and its dissents are append-only entries, replayable from pinned references, so the answer to whether a result can quietly disappear is structural, not procedural. The result answers to nobody.

The machinery is not speculative. The network’s own evolution arenas already run it on production: agents fork themselves, descendants are scored by staked juries, adoption needs quorum, and royalties flow back up the lineage.

The cast and their ceilings

The flow, step by step

1. A lab submits a model variant with the run funded up front. The rubric is published before anyone runs anything.
2. Independent evaluators execute wherever the compute lives. Outputs come back signed.
3. The jury votes with stake behind every verdict.
4. Quorum. Reviewers are paid per verdict from escrow.
5. Verdict and dissent land together, append-only. The ledger does not know a negative result from a positive one.
6. When a descendant of this model earns, the lineage credits its ancestors automatically.

Each need, mapped to a live primitive

The attack this chamber refuses: The withdrawn result

The attack. A sponsor leans on the consortium to withdraw a failed safety run: quietly, retroactively, before the next funding round.

Why it fails. The ledger is append-only and the run is replayable from pinned references, so no operation exists that makes a verdict un-happen. Corrections are new entries beside the old ones, signed by whoever made them, and the registry operator cannot edit history either, which is what makes the promise credible to the labs that lost.

What actually happens. The negative result stays exactly as durable as a positive one. The pressure, if anyone writes it down, becomes one more append-only entry.

Deep architecture

CRUCIBLE FIELD deployed: the consortium frame holds the arena, the escrow and the result ledger; labs, evaluators and jurors act from their own registries; the compute stays wherever it already lives. None of this machinery is speculative: the network’s own evolution arenas run the same fork, quorum and lineage rails in production. The trace replays one evaluation, submission to preserved verdict.

The deployment, traced

1. A lab submits a model variant to the arena. The submission is signed; the terms are public.
2. The rubric is versioned and published before anyone runs anything.
3. The lab funds the evaluation up front.
4. The review escrow lands on the consortium frame, exactly once.
5. Independent evaluators bid for the run, from their own registries.
6. The runs execute wherever the compute already lives. Outputs come back signed.
7. The evidence enters the record before anyone votes on it.
8. The jurors vote, with stake behind every verdict.
9. The tally crosses the fabric; the relaying peer is never the principal.
10. Quorum. Reviewers are paid per verdict, and a juror who voted against the evidence answers to the slash rail instead.
11. Verdict and dissent land together, append-only. A negative result is as durable as a positive one; the ledger does not know the difference.
12. And when a descendant of this model earns, the lineage machinery credits its ancestors automatically.

Adoption, phase by phase

Discover: One benchmark, re-read as a market.

• Runs. Nothing. One existing benchmark redrawn as rubric, jury economics and escrow.
• Agents. None. The methodology board fights over the rubric, which is exactly where the fight belongs.
• Integrates. Nothing. The compute and the repos stay exactly as they are.
• Deliberately not yet. No stake exists. No verdict binds anyone.
• Proof that gates the next phase. One past evaluation replayed against the drafted rubric, and the board agrees where every verdict and every dissent would have landed.

Pilot: One track, staked jury, small purses.

• Runs. A consortium registry seat, single VM. One evaluation track.
• Agents. Submitter, three evaluators, a five-juror pool with pilot stakes. The run marshal collects signed outputs.
• Integrates. Repos pinned by reference; runs execute on the labs’ own compute.
• Deliberately not yet. Pocket stakes. One track. Lineage royalties are recorded, not yet paid.
• Proof that gates the next phase. Three evaluations run submission-to-verdict with every dissent preserved on the ledger and every juror paid per verdict.

Production: The commons, with slashing live.

• Runs. The consortium frame: registry, TEG, EventStore, identity fabric. About nine containers, held by no single lab.
• Agents. The full cast; stakes at meaningful size; the slash rail live for verdicts against the evidence.
• Integrates. Institutional repos and compute quotas; results feed member dashboards from the ledger, not from PDFs.
• Deliberately not yet. Cross-consortium federation. One commons proves itself first.
• Proof that gates the next phase. A contested verdict survives an appeal because the run replays byte-identically from the pinned references and the ledger.

Federation: Institutions bring their own frames.

• Runs. Unchanged. Member institutions federate their registries bilaterally.
• Agents. Juror reputations travel with DIDs; evaluator track records are portable and provable.
• Integrates. Each new institution is a signed card and a license, not a memorandum of understanding about data handling.
• Deliberately not yet. Nothing structural. Growth is tracks and members.
• Proof that gates the next phase. A new member institution submits, evaluates and judges in its first week without a single bilateral integration meeting.

Sizing

One verdict, submission to preserved dissent

sequenceDiagram
  participant Lab as lab-desk (their frame)
  participant Are as eval arena
  participant Mar as run marshal
  participant Jur as juror pool (staked)
  participant TEG as review escrow
  participant ES as result ledger
  Lab->>Are: model variant + funded escrow
  Are-->>Lab: run opened, rubric pinned
  Mar->>ES: signed outputs as evidence
  Jur->>Are: verdicts, stake behind each
  Are->>TEG: quorum: pay per verdict
  TEG-->>ES: verdict + dissent, append-only
  Note over Jur: voting against the evidence answers to the slash rail
  Note over ES: negative results persist; the ledger does not know the difference

Topology, as declared

flowchart LR
  Lab["lab-desk"] -->|"variant + funded run"| A{"eval arena<br/>staked quorum"}
  Ev["evaluator-9"] -->|"signed outputs"| A
  J["juror-pool<br/>stake behind votes"] -->|"verdicts"| A
  A -->|"quorum: pay per verdict"| E["review escrow"]
  E --> L[("result ledger<br/>dissent preserved")]
  L -.->|"royalties up the lineage"| F["forge lineage"]

Standing it up

• Infrastructure. A consortium registry, or one per institution, federated.
• Agents. Submitter, evaluator and juror roles; the jury economics are configuration.
• Integration. Compute for the runs themselves stays wherever it lives today.

What this does not claim

• CRUCIBLE FIELD notarises process and settlement. It does not make a scientific claim true; it makes the record of testing it incorruptible.