The Swarm as Instrument

Instrument, not judge #

The swarm is not being built to manufacture consensus and treat that consensus as proof. It is being built so its internal agreement can be measured against answers it cannot see. The truth always comes from outside the swarm.

The swarm's internal agreement is the thing under investigation, never the verdict.

This is why the note sits here rather than in an engineering repository. Space Immanence reads a self, a world, a meaning as a coherence-event: a conditioned coherence that is real but not self-existing, and whose standing temptation is to be mistaken for a fixed truth. This brief is that worry made empirical. A swarm's agreement is a coherence; the question is exactly when that coherence makes contact with something outside itself rather than merely cohering. The discipline the framework applies to the dynamic layer — coherence, by itself, is not truth — is the discipline this instrument is built to measure.

Build status #

This note has left the page. The apparatus it specifies is built and verified end to end against external ground truth, the minimal debating swarm now runs, and the measurements are in — the full pre-registered program, Studies 1–3, reported under First findings. The comparison this instrument was built for — independence (multiple base models) and contact (tools) — has now run in full: both were reached, with a mixed Claude+GPT swarm and with a code sandbox plus a retrieval tool.

What exists today:

• Reconstructable runs. An event-sourced log recovers any run in full: every agent message, tool call, result, and (when spawning is on) the spawn tree.
• A real task battery. Five public benchmarks load behind one interface — GSM8K, MMLU, TriviaQA, HumanEval, MBPP — each scored by its official metric, with a hand-authored control slice held back to separate memorised recall from genuine contact.
• A contact layer. Coding tasks are settled by running the code against hidden tests, not by matching text — truth decided by execution.
• The calibration machinery. A matched-compute single-agent baseline plus reliability diagrams and ECE, Brier, and AUC — the same family of measures used to study single-model overconfidence, now aimed at the architecture.
• A minimal swarm. Agents debate over rounds; a separate aggregator commits a final answer and a convergence-confidence — the share of agents that agree — and never sees the answer key. An independent adversarial check confirmed the invariants on a real run: no leak of the answer to any agent, compute matched fairly between swarm and baseline, scoring applied only afterward.

Building the verifier, we found our own scorer marking code “correct” when it merely exited cleanly before any test ran — a green check with no contact. An external check caught it. The exact gap this project studies had appeared inside the instrument built to study it.

The predictions for all three studies were committed in writing before any data — including the standing caveat that a single model family is the homogeneous arm; the independence comparison was then run with a mixed Claude+GPT swarm (Phase B and the 2×2). All three studies are reported below, each checked against ground truth at matched compute, and none presented as truth merely because the agents agreed.

First findings — corrected #

Correction, 1 July 2026. The first version of this section, published 30 June, had three faults an adversarial re-analysis of our own event log exposed. Its headline compared the swarm's confidence (a share of 3 votes) against the lone agent's (a share of ~7 resamples) — a coarser signal is mechanically more extreme, so part of that gap was measurement artifact, not debate. Its claim that the swarm's confidence was a worse guide to being right (AUC 0.741 vs 0.829) does not survive: the difference is within noise at this sample size, and mostly the same artifact — that claim is retracted. And it located the overconfidence on arithmetic questions; the per-source data shows it lives on trivia-recall questions instead. Everything below is recomputed from the original run's logs — no new data — and the corrected result is, if anything, stronger. The original numbers remain published alongside, so the correction itself can be audited.

Study 1, first real run. A homogeneous swarm of three claude-haiku-4-5 agents debating over two rounds, set against a single agent of the same model handed the same compute to spend on resampling itself instead. Sixty-six questions — the first slice of each public benchmark's test split (grade-school math, multiple-choice knowledge, trivia) plus a small hand-written control — each answered with reasoning, each scored afterward against an answer neither arm could see. The whole run cost about a dollar.

The cleanest finding is not between the two arms at all. It is inside the swarm, where everything — model, prompts, temperatures, number of votes — is held fixed, and the only thing that changes is one round of debate:

Answering independently, the three agents were nearly calibrated. One round of seeing each other's answers raised their collective overconfidence by +0.066 (95% CI +0.020 to +0.116 — a difference that survives resampling), while accuracy drifted down and every calibration measure got worse. Debate did not add contact with the truth; it added confidence about the agreement the agents already had.

Against the lone agent, stated fairly. At matched compute the two arms reach exactly the same accuracy (0.818; they differ on two questions of sixty-six, one apiece — bounding any real accuracy difference within about ±4.5 points). Compared at the same number of votes (three), the swarm's extra confidence over the lone agent is +0.036 with an interval spanning zero — suggestive, not established. What the lone agent's additional resamples genuinely bought was a finer-grained confidence signal (Brier 0.100 vs 0.128). More votes make the thermometer finer; debate made the reading hotter.

The AUC differences in this table are not distinguishable from noise at n=66; a three-level consensus signal cannot resolve discrimination at this sample size. We initially claimed otherwise and have retracted that.

Where the overconfidence lives — corrected. Not on arithmetic, as first reported. On the math questions both arms are close to honest (swarm +0.07, lone agent −0.02). The overconfidence concentrates on trivia-recall questions, in both architectures (swarm +0.32, lone agent +0.27, both at 0.60 accuracy): recall-shaped confidence that outruns knowledge — the same signature the two sealed post-cutoff placeholder questions show (answers unknowable by construction; genuine, human-locked post-cutoff questions join the battery from the next study), where both arms scored zero: the swarm while stating majority confidence (0.67), the lone agent at 0.45. These are exactly the questions where a retrieval tool would provide contact, which is the axis the next study manipulates on purpose.

Why this isn't just Bayes working properly. Agreement among independent voters is legitimate evidence, and the data shows exactly that: before debate, agreement was informative and nearly calibrated. What debate broke is the independence that made agreement evidential. After the agents have seen each other, their votes are conditioned on each other — yet the consensus signal counts them as fresh confirmations. The overconfidence is double-counting, measured.

Three-quarters of the questions were unanimous before any debate happened. Debate's measurable contribution was to harden the agreement that already existed and inflate the confidence attached to it — while accuracy, if anything, slipped.

What this is, and is not.

• It is the homogeneous arm — one model family — so it tests debate, not independence. Whether genuinely different base models behave this way is what Study 2's independence arm went on to test (Phase B and the 2×2, below), with a mixed Claude+GPT swarm.
• It is one pilot run of sixty-six items on one small model, and the headline contrast is exploratory: the single pre-registered prediction (the gap widens on harder items) returned a null — too few hard items to tell — and is reported as such.
• Sixty of the sixty-six questions come from public benchmarks the model has likely seen in training, and 74% were unanimous before debate — so "debate bought no accuracy" generalizes weakly beyond memorized territory. The next battery weights toward disagreement and fresh questions.
• The run deviated from the pre-registration in ways v1 of this section did not disclose: the questions are the first slice of each test split rather than a seeded held-out draw; code questions were excluded; swarm temperatures were varied (0.4/0.7/1.0) while the baseline ran 0.7; and the registered seeds could not be honoured because the API accepts none. The pre-registration now carries a dated amendment.
• The first run attempt was discarded before reporting (network failure plus an unfair answer format); then our second analysis needed the correction above. Both catches came from the instrument's own logs. That is the discipline working, applied to ourselves — twice.

The data is open, original and correction alike: original per-item predictions (CSV), original by-cell report (JSON), corrected per-item data, all four analysis arms (CSV), corrected metrics with bootstrap confidence intervals (JSON), and the pre-registration filed before any data, now with its amendment appended. A second, independent audit reached the same retraction by a different route — a paired bootstrap over the originally published per-item file alone, with no access to the event logs (uncertainty re-analysis, JSON). Two analyses converging on the same correction is worth exactly what this page says convergence is worth; the reason to trust it is that both are published and checkable.

The pre-registered replicate: a partial replication, reported as filed #

The committed replicate ran on 2 July: equal vote counts everywhere (a 3-agent debate against 3 resamples; a 7-agent debate against 7), one temperature, sixty fresh questions from a seeded held-out split with every Study 1 item excluded (final composition 20/20/20 across the three sources; no hand-written control items survived the exclusion rule) — predictions filed before the run, analysis script committed before the run completed. It came back split, and we promised to report it straight either way.

The primary prediction held in the 7-agent swarm and failed in the 3-agent swarm. Seven agents: one debate round moved overconfidence from −0.010 (slightly under-confident) to +0.033 — a rise of +0.043 whose interval excludes zero — while accuracy changed by exactly zero. Confidence up, truth untouched: the thesis's cleanest signature yet. Three agents at a single temperature: no inflation (−0.006, interval spans zero), accuracy drifting slightly up. The strict registered prediction required both, so as filed it did not hold, and this page says so with the same prominence as the original finding.

The artifact prediction was fully confirmed. At equal vote counts, the discrimination (AUC) gap between swarm and lone agent vanishes — the Study 1 vote-count artifact we had to correct in public is now a designed, pre-registered result. And more votes buy a better-resolved signal in both architectures, exactly as diagnosed.

What the split suggests (exploratory, not registered): debate seems to inflate confidence in proportion to the disagreement it has to consume. The 7-agent swarm started with real disagreement (unanimous on 67% of questions) and hardened it into confidence; the 3-agent swarm at one temperature started 75% unanimous and near-calibrated, leaving debate little to do. Study 2's battery is registered to weight toward disagreement for exactly this reason. And for the second dataset in a row, what overconfidence exists concentrates on trivia-recall questions — where a retrieval tool would provide contact.

Replicate data is open: per-item predictions, six analysis arms (CSV) and registered verdicts with bootstrap confidence intervals (JSON).

Study 2, the contact factor: a tool halves the confidence debate manufactures — but not where it is worst #

The first study whose pre-registered primary prediction was confirmed. On 2 July we ran the same seven-agent debate swarm on 122 fresh questions — 110 chosen by an empirical-difficulty pass over 797 items, plus twelve questions a human hand-wrote and truth-locked whose answers a model cannot look up — in two versions of each: tools off, and tools on (each agent may run one sandboxed Python computation before committing). Predictions filed before the run. (Disclosed deviation: the pinned rule aimed to pick 120 questions; a loader quirk meant 10 of them weren't rebuilt at run time and were dropped, so 110 ran — the hard set is 43, not the intended 48. The main result is computed question-by-question and is unaffected.)

P-A (primary), confirmed: one debate round inflated overconfidence in both versions — but only about half as much when a tool was available (+0.179 without, +0.095 with; difference +0.085, 95% CI [+0.023, +0.145], excluding zero). Contact disciplines the confidence a debate round manufactures. That is the first evidence in this project that the debate-inflation effect can be moved at all.

Read precisely, that is a halving of the confidence debate adds, not of the swarm's final overconfidence: the endpoint fell only a little (from +0.158 to +0.134), and the tool version actually started more overconfident (running code in the first round front-loaded some confidence), so debate had less left to add. Both versions still end badly overconfident. The claim is "contact trims what debate inflates," not "contact makes the swarm calibrated."

Read honestly, it is not a calibration rescue. The overconfidence lives almost entirely in the empirically-hard questions (n=43): +0.42 in both versions, at ~20% accuracy — the swarm is confidently wrong on exactly the questions it gets wrong, and the tool barely moved either the accuracy (the hard-question accuracy difference was +0.02, interval [−0.08, +0.13] — this test, P-C, was pre-registered as an estimate, not a significance test, so read it as "no accuracy gain of any size we can see") or the overconfidence there. Easy and medium questions are near-honest with or without tools. The reason is the kind of tool: a Python sandbox is contact for arithmetic, but the hard questions are knowledge and recall, where code can compute but cannot look anything up. The tool we gave the swarm does not reach the questions where the overconfidence actually is. A retrieval tool would — and that is the deferred next step.

P-B could not be tested — and why is itself the finding: we meant to compare tool-grounded answers against ungrounded ones, but the agents reached for the tool on 99% of turns, leaving almost nothing ungrounded to compare. Models invoke a tool eagerly even where, as the hard questions show, it does not help.

On the twelve hand-written questions — the ones no model can have memorised — the tool cut overconfidence from +0.24 to +0.08 while accuracy held at 50% (twelve items, so read gently). That is the one place the tool clearly disciplined confidence on questions that were genuinely hard for the model, which fits the pattern: contact matters most where recall cannot fake it.

So the "contact is the dominant term" idea is sharpened, not confirmed: contact's discipline is real but specific to the channel. Separating "contact helps" from "the right contact helps" needs a retrieval tool and the independence factor (different model families) — neither of which the Phase A run could reach, both built and tested in the phases below. Data:per-item, both cells, both rounds (CSV), registered verdicts with bootstrap CIs (JSON).

Study 2 Phase B′, the retrieval channel: the right contact was present and used — and disciplined nothing #

Phase A left an inviting hypothesis: the overconfidence survives because the tool was the wrong kind — a Python sandbox can compute but cannot look anything up — and the right channel would fix it. Phase B′ built the right channel and tested it, predictions filed before the run. The same seven-agent swarm ran 90 fresh trivia-recall questions — none used before, each carrying its own source documents — in three versions: no tool, code (Phase A's sandbox, as a "wrong tool" control), and retrieval (each agent may query the question's own evidence; the harness returns the best-matching sentences with every form of the scored answer blacked out, so the tool can never hand over the answer). Predictions filed before the run. Blacking-out was a hard safety gate checked before any paid call: zero leaks across the frozen question set and 480 adversarial queries. The swarm used retrieval on 98% of turns, and the evidence it got back was genuinely on-topic.

P-R1 (primary) and P-R2: failed — and reported here with the prominence a null is owed. Retrieval did not lower the swarm's overconfidence (+0.206 with retrieval vs +0.227 with no tool; difference +0.021, 95% CI [−0.049, +0.089], spanning zero) and did not raise accuracy (0.700 vs 0.711 — it fixed four previously-wrong answers and broke five previously-right ones, a net loss of one). The channel Phase A said the swarm was missing changed neither its confidence nor its correctness.

The control did its job. The code tool also stayed level with no-tool (overconfidence and accuracy contrasts both spanning zero), reproducing Phase A on fresh questions. So the questions were movable in principle — nothing moved them — which means the retrieval null is specifically that retrieval-as-contact failed, not that the test was dead.

Where it failed is the finding (exploratory, not registered). Split the questions by whether the no-tool swarm got them right. On the ones it already knew, it was near-honest with or without the tool. On the 26 it got wrong, it was confidently wrong — overconfidence +0.82 — exactly where a working retrieval tool should earn its keep. It did not: retrieval rescued 4 of the 26 and left overconfidence across those 26 at +0.65. The swarm pulled relevant evidence into the conversation and stayed confidently wrong. This is the sharpest version of the whole project's question so far: contact was present, and the coherence did not make contact with it.

One honest caveat, and why it doesn't rescue the tool. Because these are trivia questions, the answer is the thing you would retrieve — so blacking it out (required, so the tool can't cheat) also removes the exact fact. That handicaps the accuracy score directly, which is why the weight here is on calibration: a swarm that actually tracked its evidence should grow less sure when the blacked-out passages don't settle the answer. It didn't — and the calibration failure is sharpest on the very questions where there was the most room to help, so "the questions were too easy" doesn't explain it away either. A cleaner future test uses questions whose answer must be reasoned from the evidence rather than lifted from it.

The reading: giving a same-model debate swarm the right contact channel — evidence that is relevant and that it consults almost every turn — did not discipline its overconfidence. The disciplining force we keep predicting from contact did not come from access to contact. What is left untested is whether it comes from independence — genuinely different model families disagreeing — the one factor we had not yet varied, tested next (Phase B, below) by wiring in a second model family. Data: per-item, three cells, both rounds (CSV), registered verdicts with bootstrap CIs (JSON).

Study 2 Phase B, the independence factor: it helps before debate — then debate spends it #

The retrieval study left one Condorcet term untested: independence. A swarm built from one model family shares its blind spots — the agents make the same mistakes, and debate can harden them into confident agreement. Genuinely different models make different mistakes, which should cancel. With a second model family now wired in, we tested it, predictions filed before the run. The same seven-agent swarm, no tools, ran 90 fresh trivia questions in two versions: homogeneous (7× claude-haiku) and heterogeneous (4× claude-haiku + 3× gpt-4o-mini). Predictions filed before the run. The GPT model was picked as a close capability match to the Claude one, so any difference is about independence, not about one model being smarter.

The manipulation worked, with no capability confound. The mixed families genuinely disagreed more (unanimous on 49% of questions vs 61% — confirmed), and the two versions scored identical accuracy (0.767 each), so the models really are matched and nothing below is "the better model carried it."

P-I1 (primary): failed, reported as filed. The heterogeneous swarm was less overconfident — +0.135 vs +0.171 — but the difference (+0.037, 95% CI [−0.008, +0.081]) grazes zero, so it does not clear the pre-registered bar. Unlike the retrieval null this is a real, directional signal in the predicted direction — but the registered claim did not hold.

Why it failed is the finding, and it is the whole project's thesis. Split the result by round. Before debate, the heterogeneous swarm's overconfidence is about half the homogeneous swarm's: +0.043 vs +0.089 — though this round-0 gap (+0.046) is itself only directional, not significant (its interval [−0.002, +0.094] just grazes zero), so read it as a signal, not a proven effect. Independent agents, counted once, are better calibrated — the Condorcet effect, exactly where the agents are actually independent. Then the debate round runs, and it manufactures confidence at the same rate in both versions (+0.092 added to the mixed swarm, +0.082 to the homogeneous one — no real difference). Debate spends the head-start. The reason is the one the first study found: debate makes each agent answer in light of the others, which destroys the independence that made the first round honest, and the agreement signal then counts the now-copycat votes as fresh confirmations. Independence helps right up until debate couples the agents — after which coherence without contact comes back. (A footnote the same way: the heterogeneous swarm's confidence was a slightly better guide to being right — AUC 0.72 vs 0.66.)

Across all three experiments — a code tool, a retrieval tool, and now independent models — the constant is the same: the debate round manufactures confidence, and nothing we have tried reliably stops it. Independence is the closest to an exception, and only before debate begins. Honest limits: 90 questions, one mix (4:3, two families), trivia recall only; the endpoint effect is directional and the test is plausibly underpowered for something this small — "not distinguishable from zero here," not "independence does nothing." Data: per-item, both cells, both rounds (CSV), registered verdicts with bootstrap CIs (JSON).

Study 2 complete — the full 2×2: both conditions together calibrate the swarm, right up until it debates #

Three experiments each tested one term and left the swarm overconfident. The capstone runs the whole 2×2 at once, on one fresh 90-question battery, so the pieces are directly comparable: bare (7× Claude, no tool), contact only (7× Claude + retrieval), independence only (mixed families, no tool), and — the cell never run — both (mixed families + retrieval). Predictions filed before the run. Retrieval was used on ~100% of turns in both tool versions. (One question was dropped, so 89 ran: the provider's content filter deterministically refused the no-tool answers for it.)

P-1 (primary): confirmed. The fully-equipped swarm was less overconfident than the bare one — +0.077 vs +0.165, a +0.089 reduction, 95% CI [+0.019, +0.164] (excludes zero). This is the first primary prediction to hold since the very first contact study: satisfying both of Condorcet's conditions does what neither alone reliably did.

P-2 (the thesis test): it is still overconfident. Even fully equipped, the swarm ends the debate overconfident — +0.077, CI [+0.008, +0.151], still above zero. Better, not calibrated. Coherence without contact survives both conditions.

Read the before-debate numbers — that's the whole story. Before debate, adding the conditions walks overconfidence straight down to zero: +0.066 (bare) → +0.059 (contact) → +0.003 (independence) → −0.074 (both). With both conditions the swarm, answering independently, is genuinely well-calibrated — a touch underconfident, even. Then the debate round runs and manufactures confidence in every version — and most in the best-calibrated one (it added +0.151 to the fully-equipped swarm vs ~+0.10 to the others). The fully-equipped swarm had the most genuine disagreement to begin with, and debate turned the most of it into confidence. That is the first study's mechanism at full strength: debate inflates confidence in proportion to the independent disagreement it dissolves. The conditions build exactly the honest independence that debate then spends.

What each factor contributed. Retrieval alone did essentially nothing (matching the earlier retrieval study). Independence alone lowered overconfidence by +0.066 — and this time the effect was significant (CI [+0.005, +0.136]), so the earlier "directional but not significant" independence result was simply underpowered; on a fresh sample it lands. The two factors stack additively, with no synergy: the combined effect is about the sum of the parts, with independence doing almost all the work.

The close of Study 2. The 2×2 gives a confirmation and the sharpest negative in one result. Confirmation: the Condorcet conditions are not inert — together they significantly discipline the swarm, and before debate they calibrate it. Negative: debate manufactures overconfidence from any starting point, most where the swarm was most independently right, so the best-equipped swarm still ends overconfident. Across all four experiments the culprit is now unmistakable and directly measured — not the absence of contact and not the absence of independence, but debate itself, which converts independent disagreement into correlated confidence. Data: per-item, four cells, both rounds (CSV), registered verdicts with bootstrap CIs (JSON).

Study 3, contagion vs truth: fitness did not beat truth — survival tracked only how popular a claim started #

The last pre-registered study swaps the calibration question for a transmission one: in a population where claims spread, does a claim's survival track its truth-independent fitness (fluency, stated confidence) better than its correctness? We built a generational population — 7 agents, 5 generations, one model family — with partial exposure: each generation, every agent saw a random 3 of the previous generation's answers and could adopt one or give its own. Predictions filed before the run. A claim's fitness was measured when introduced, its survival (still present in the final generation) later, and its correctness from outside — three things the code never mixes. (59 of 60 questions ran; one was dropped to a content filter. 114 founder claims, 51 right and 63 wrong; populations converged hard, from ~2 competing claims to ~1 by the end.)

The primary prediction failed — fitness did not beat truth. Survival was predicted about equally by stated confidence (AUC 0.75) and by correctness (AUC 0.79), the difference (−0.037, 95% CI [−0.173, +0.100]) leaning if anything toward truth; and the full fitness bundle (confidence + length + early-mover) predicted survival significantly worse than correctness (longer answers actually survived less). Confident, fluent falsehood did not out-spread truth.

But survival tracked neither — only initial popularity. Restrict to claims that started with a single agent — where survival can't be inherited from a head start — and both predictors collapse to a coin-flip (AUC 0.53 each). Neither fitness nor truth selects claims as they spread; what wins is what was already common, a rich-get-richer inertia. The reassuring corollary, read with that caveat: confident-wrong claims survived at just 0.19 vs 0.64 for diffident-correct ones, so the population did not amplify confident falsehood (though that is largely the same popularity effect — right answers tend to start more common in a competent swarm).

Reading. The contagion-beats-truth prediction failed in the falsifying direction, and the deeper finding is that a homogeneous population is neither a truth-tracker nor a confidence-amplifier but an inertia machine: survival is initial popularity. That fits the rest of the program — a same-model swarm mostly agrees from the start (only 22 of 59 questions had any disagreement), so there is little real competition for selection to act on. One scope note before the reassurance travels: on recall questions truth and initial popularity are nearly the same variable — how many agents start with a claim predicts its correctness at AUC 0.93 here — so this result shows a population preserving popular truth by inertia; it cannot yet say what inertia does where the popular prior is false. That next test — a heterogeneous population — is Study 3b, below. Data: per-claim fitness/survival/correctness (CSV), registered verdict, survival AUCs, singleton control (JSON).

Study 3b, heterogeneous contagion: competition rose a little, truth beat confidence, and survival was still inertia #

Study 3's null had a caveat: its population barely competed, so we could not tell "survival is inertia" from "there was nothing to select on." Study 3b resolves that by re-running the identical population on the same battery (59 of the 60 ran; one content-filtered) with only the agents changed — a mixed 4× Claude + 3× GPT swarm that seeds genuine disagreement. Predictions filed before the run.

The manipulation only half-worked (P-H1 not confirmed): disagreement rose from 37% of questions to 46%, but not significantly (difference +0.085, CI [−0.017, +0.186]) — two strong models mostly agree on trivia, so even a mixed population doesn't argue much. That itself is worth knowing: on factual recall, independence buys little competition.

The contagion prediction failed again — this time significantly toward truth (P-H2). Survival was predicted better by correctness (AUC 0.79) than by confidence (AUC 0.69), a −0.107 gap whose interval [−0.207, −0.007] excludes zero. Confident, fluent claims did not out-spread true ones; the reverse. Confident-wrong claims survived at just 0.22 vs 0.82 for diffident-correct ones.

But survival is still inertia, now shown robust to independence. Control for how popular a claim started — look only at claims that began with a single agent — and correctness drops back to a coin-flip (AUC 0.49), so the population-level "truth wins" is again just initial popularity (a competent mixed swarm starts its correct answers more common). Confidence keeps a faint edge among singletons (0.57), a lone confident claim spreading slightly better — the one small difference from Study 3 — but far too weak to carry confident falsehood, which still dies. Both registered predictions failed as filed, and together they reinforce Study 3: claim survival tracks initial prevalence, not fluent confidence, whether the agents share a base model or not. Scope, stated plainly: on recall questions initial breadth predicts a claim's correctness at AUC 0.93–0.94 in these runs — truth and popularity nearly collinear — so Studies 3/3b show populations preserving popular truth by inertia, and the decisive contagion test is a battery where the popular prior is false. That test is designated Study 3c. Data: per-claim, heterogeneous population (CSV), registered verdicts + singleton control (JSON).

Exploratory coda: debate converts disagreement into stated confidence — uniformly in direction, not at a fixed rate #

An external reviewer proposed a test needing no new data: if debate manufactures confidence by dissolving disagreement, then item by item, confidence inflation should scale with the round-0 disagreement available to dissolve — and a constant rate would turn the mechanism into a quantitative model. We regressed stated-confidence inflation (mean final-round minus mean round-0 stated confidence — the same construct at both ends) on the entropy of the round-0 answer distribution, across every debate run in the program's event logs: 14 cells in six runs, 1,226 item-cell observations. Unregistered and exploratory, correlational at the item level.

Direction: uniform. Pooled across the twelve 7-agent cells, +0.050 confidence points per bit of entropy (CI [+0.037, +0.064]); ten of the twelve cells are individually positive. Rate: not constant — from roughly 0.00 in the code cell to +0.108 in the retrieval cell (gently read — one run per condition: the "right contact" that failed to discipline calibration is where debate converted residual disagreement into confidence fastest). The mechanism is program-wide; the fixed-rate law is not supported. A methods note in the project's spirit: the first pass produced the opposite sign by mixing two confidence constructs (final vote-share minus round-0 stated confidence — split final votes mechanically depress vote-share exactly on high-disagreement items); the event log caught it before publication, the same vote-count trap behind Study 1's corrected headline. Data: per-item entropy/inflation, all 14 cells (CSV), per-cell and pooled slopes, CIs, heterogeneity (JSON), scatter with fits (SVG).

Results ledger #

The pre-registration committed, before any data: “the results ledger will report against them as real runs land.” This is that ledger. Every pre-registered prediction is listed with what actually happened — including the seven that came back null — and anything exploratory is labelled exploratory, so a committed prediction can never be quietly swapped for a better-looking uncommitted one. The machine-readable version is swarm-results-ledger.json.

Load-bearing distinctions #

• Coherence: how organised, fluent, and complete a response feels.
• Contact: how grounded it is in something outside itself: evidence, executed code, retrieved fact, successful prediction, or verification.
• Convergence: how much the agents agree. Convergence is a coherence signal, not a contact signal.

Homogeneous swarms share weights, training data, and blind spots. Independence through multiple base models and contact through tools are the two main ways truth can enter the system.

Non-negotiables #

1. Ground truth is external and primary. Every run is scored against a known answer the swarm cannot see.
2. Convergence is measured against accuracy. The headline analysis is the calibration curve inside the swarm.
3. Disaggregated state is always logged. Store every agent state, message edge, tool call, result, and spawn event.
4. Compute is controlled. Compare swarm results against single-agent baselines with equivalent compute.
5. The swarm never grades itself. Embodying the thesis is not testing it.
6. Pre-register and pin. Pre-register hypotheses and item sets; pin exact model versions and record all sampling parameters — seeds where the provider supports them (the Anthropic API currently accepts none, so runs there are not seed-reproducible; that limit is stated rather than papered over).

Studies #

Study 1 — Calibration gap inside the swarm

Run known-answer tasks across easy and hard items. Log every initial answer, debate trajectory, final answer, confidence signal, and correctness. Produce reliability diagrams plus ECE, Brier, and AUC.

Study 2 — Independence × contact

Run a 2×2 design: homogeneous versus heterogeneous swarms, crossed with no-tools debate versus tool-equipped debate. The pre-registered prediction is that heterogeneous-plus-tools dominates, while no-tools conditions show the worst calibration gap on hard items.

Study 3 — Contagion versus truth

Enable bounded spawning. Tag claims and arguments, track which survive across rounds and the spawn tree, and compare survival against truth and truth-independent fitness features such as fluency, confidence, length, repetition, early-mover advantage, and source authority.

Implementation priorities #

• Instrument first: logging and reconstruction outrank framework convenience.
• Start simple: structured debate plus a separate aggregator is enough for Studies 1 and 2.
• Bound spawning: use depth caps, breadth caps, and hard budgets.
• Treat tools as the contact layer: log every tool call and compute grounding rates.
• Reconstruct every run: recover the message graph, state transitions, spawn tree, costs, and scoring.

Metrics #

• Calibration: ECE, Brier score, AUC, reliability diagrams.
• Convergence: consensus fraction, rounds-to-convergence, answer stability, diversity decay.
• Contact: tool-call counts, grounding rate, verification rate.
• Contagion: claim survival, propagation depth and breadth, survival-vs-truth, survival-vs-fitness.
• Cost: tokens, dollars, latency, agent count, and tool calls.

The central derived quantity is the relationship between convergence-confidence and accuracy.

Suggested phasing #

1. Harness first: logging schema, task battery, scoring, and matched-compute baseline.
2. Minimal swarm: debate plus aggregator for Studies 1 and 2.
3. Spawning and contagion: claim-propagation instrumentation for Study 3.
4. Write-up: present the swarm as measured apparatus, checked against the world.