The rounds and the measurements

Every experiment the project has run, in one table, with its verdict and the page that holds it. The measure changed on 14 Sep: rounds 1 to 6 timed agents; from round 7 the reading is reliability, speed and memory, the loop, and dependencies. See state-of-the-project for what it adds up to.

rounddatetasklanguageswhat it readverdictpage
112 Seplogstat from its specMo, Go, Pythonagent timeMo 25.5 min, Go 12.9, Python 8control-run
213 Septhe same, on the day’s toolchainMo, Go, Pythonagent timeMo 11.8 min, zero failed runscontrol-run-2
313 Septhe same, after steps 14 to 16Mo, Go, Pythonloops by causeMo 9 loops (one real bug caught by a test and a never), Go 2, Python 1control-run-3
413 Septhe same, after steps 17 to 20Mo, Go, Pythonagent time, loopsMo 16.1 min, Go 9.3, Python 9.0; the laws keptcontrol-run-4
513 Septhe same, pre-registeredMo, Go, Pythonthree predictionstwo held, one missed by a loop; Mo 1.28 times Go’s timecontrol-run-5
614 Seplogstat and jobq, with every check bolted onMo, Go, Pythonfour predictionsfailed on all four; no check caught a bug in any language; three laws removed by step 27control-run-6
714 Septhe job queue, on Robert’s measureMo, Go, Pythonreliability, speed, loop, dependenciesheld on all four: 0 defects each, Mo 981 pairs a second to 478 and 467, loop 0.38 s to 18.7 and 8.7control-run-7
815 Sepchange 1 to round 7’s queues by fresh maintainersMo, Go, Pythonfive predictions, two hidden suitesheld on all five; the fourth oracle found Mo’s outage; no check caught a change-induced bugcontrol-run-8
1015 Sepround 7’s queue and change 1 in ElixirElixirthe BEAM null hypothesis2 defect causes, twice Mo’s speed, 0 run-time dependencies; P6 the BEAM’scontrol-run-10
916 Sepchange 1 by smaller models in Pikimi-k3, deepseek-v4-flash, gpt-5.5, qwen 27B, Haiku 4.5reliability and loops per modelreliability moves with the model on the Mo side only; the diagnostics carry small models to green; the local 27B made no editcontrol-run-9
measurement 115 to 16 Sepevery program regenerated from its stripped spec, twiceMocompletenesstwelve of twelve at 1.0; with tests deleted too, 0.74 under the original testsbodies-as-cache
measurement 215 Septhe queue’s board regenerated five times, comparedMoagreement over 132,000 operationsidentical except one impossible recordsampling-as-verification
erosion, generation 216 Sepchange 2 to all four queues by fresh maintainersMo, Go, Python, Elixirregressions, a third suite, a fourth oracle, P6nothing eroded in Mo, Go, Python; Elixir refuses a torn line and cannot restart after a full disk; the deferred reply usederosion-round
erosion, generation 316 Sepchange 3, the store restarts itselfMo, Go, Python, Elixira fourth suite, P6Mo 55, Go 55, Python 55, Elixir 53 of 55; the Mo queue killed under load back in 106 mserosion-round
erosion, generation 416 Sepchange 4, idempotent creates and the archiveMo, Go, Python, Elixira fifth suite, the speed rowMo 77, Python 77, Go 76, Elixir 76 of 77; the Mo queue nine times slower on the lease patherosion-round
erosion, generation 516 Sepchange 5, a lease handed off and a rename in flightMo, Go, Python, Elixira sixth suite, speed per generationPython 94, Elixir 94, Go 92, Mo 93 of 94: the first Mo-only defect; no never trippederosion-round
the scaling run15 Sepstep 30 at 1, 4, 10, 14 cores on the M3 MaxMothroughput per core countnothing scales; every row fastest at 1 coremac-scaling-run
the model bake-off12 Septhe corpus brief by Opus, Grok, and CodexMothe corpus each producedon branches corpus-grok and corpus-codexmodel-bakeoff

The instruments

The suites and probes live beside their plans and are never shown to a worker: control-run-7-suite/ (the 121-check defect suite and the measure), control-run-8-suite/ (regressions, change 1’s defects, the fourth oracle), control-run-10-suite/ (p6.py), control-run-9-suite/ (the briefs, the paced Python runner), sampling-as-verification-suite/, bodies-as-cache-suite/ (the stripper and the verifier), erosion-round-suite/ (the third suite with its RAM disk, the fourth oracle), interpreter-step-30-suite/ (the Mac run).

The programs the rounds run on

program-1 (the job queue) is the round 7 to 10 program; program-2 (logstat) the round 1 to 6 program; the change specs are 01b-job-queue-change and 01c-job-queue-change-2. See the-programs.