Step 42: runtime memory safety
Orientation
On 19 Sep 2026 both runtimes handed an asker freed memory: an answer to a
kept ask held a bare region value across a frame return, the frame compacted,
and the value was overwritten (toolchain/STEP-RAW-MEMORY-REPORT.md; read it
first). Robert asked how the whole class is prevented, not the one case. This
step makes the class fail loudly in the suite, then audits for its other
members. Two terms, used below. A region value is a Value pointing into a
process’s region, valid only until the next compaction. A parcel is a
packed, owned copy that outlives compaction (what send makes). Worker: a
fresh clean OMP session using GPT Sol at high reasoning, own worktree and
Herdr tab. The lead owns the wiki, audit and acceptance.
No new syntax; no change to Mo’s semantics.
Resumption authorized, 19 Sep, evening: continue from 0a4dffcd on
toolchain/step-42-memory (original base 2f669902). Read WIP.md and branch
history, preserve the Opus worker’s part E RED/GREEN evidence, then finish A–D
and verify E’s remaining limits. Do not treat the WIP’s implementation ideas
as accepted designs. Benchmarks and full-suite timing need lead scheduling;
focused builds/tests remain in this worktree, never the server worker’s.
Write scope
toolchain/src/**, toolchain/runtime/**, toolchain/build.zig,
toolchain/bench/step36/guard.py, toolchain/bench/step42/**,
toolchain/STEP-42-REPORT.md, toolchain/WIP.md, toolchain/README.md (the runtime
paragraphs only), toolchain/PRELUDE.md if the prelude changes, new corpus files
under examples/ with sidecars and verified: lines (real tools only).
No examples/programs/workspace-server/**, no harness code, no wiki, no audit/, no
HANDOFF.md. No machine, Docker, /opt. No push. Never use tr; ls is
aliased, use /bin/ls. The Linux VM is the lead’s; you run on this Mac only.
You are not alone on this host: preserve others’ edits, no nested delegation.
Parts
A. Stress mode for the whole corpus, both runtimes (RED first)
Today -DMO_STRESS (mo_rt.h:240, budgets 0) is used by one test, and the
interpreter’s frame_budget, loop_budget, walk_budget (vm.zig:167) are
settable only from Zig unit tests (vm.zig:2337). Give both a supported
switch (an environment variable such as MO_STRESS=1 read by mo run, mo test and by built binaries, or a build flag for C if a runtime read costs the
hot path: measure, then choose, and say which). Then a standing test runs every
corpus program that has expected output under stress in both runtimes and
demands the same output as the normal run. Commit its first output before any
fix: whatever it finds is this step’s RED. Keep its wall time in the report; if
it more than doubles the suite, propose a sampled default with the full sweep
behind a flag, do not silently sample.
B. Poison what compaction frees
In stress mode, fill every byte a compaction releases with a fixed pattern
(say 0xA5) in both runtimes, so a stale read yields a recognisable wrong
value or a trap instead of plausible old bytes. For the C runtime add an
ASan build of the stress sweep with manual region poisoning
(ASAN_POISON_MEMORY_REGION on release, unpoison on reuse); it may be a
separate, slower test step. Prove each with a mutant: reintroduce the raw
PendingAnswer value behind a test-only switch or a patch in the report, and
show stress plus poison catches it at the 45-byte size.
C. The audit
List everything either runtime holds outside the stack across a frame return
or a safe point, with file and line, and for each say region value or parcel
and why it is safe: pending answers (fixed), timers and send_later, mailboxes,
kept replies, the runtime surface’s snapshots, bricks’ buffers, the blocking
pool’s requests and results (step 41’s Exec included), supervisors’ restart
state, the simulator’s queues, and the no-packs modes the raw-memory report
left unchanged; and blocking.run (step 30’s pool), which steps 40 and 41 both
reported still returns if block errors while its job is on the pool, leaving
the job with a dead stack (blocking.alone has the guard). Fix what is unsafe, each with a failing test first. The list
goes in the report as a table.
D. The type split
In Zig, make a long-lived struct unable to hold a region value by accident: a
distinct Parcel (owned) type already exists in effect; give the fields found
in C a type that only pack produces, so storing a bare Value there is a
compile error. In C, the same by a distinct struct type and a comment at each
holder. State plainly in the report what the types do not prevent.
E. The guard kills the group
toolchain/bench/step36/guard.py kills only its direct child (p.kill()), and
an orphaned test binary later deleted the shared zig-out. Start the child in
its own session or process group and kill the group on timeout, on the memory
limit, and on a forwarded TERM or INT; the exit status and messages stay as
they are. Test it with a child that spawns a grandchild that ignores TERM.
Numbers
Best of five, both runtimes, load average beside them: the suite’s wall time before and after; the stress sweep’s wall time; three corpus benchmarks of your choice in normal mode before and after, to show the switch costs nothing when off.
Worker stress sweeps and benchmark measurements require an explicit exclusive lead-issued slot after sibling builds/runtime checks drain. The unfiltered suite and its best-of-five timing remain lead-owned; the worker’s slot does not authorize them or replace independent integrated acceptance. Record exact before/after commits and commands, all five samples (not only the best), load averages, guards, real exits and any failed or invalid samples. Do not change implementation during a measured series; corrections require a new checkpoint and a clearly separated series. No Linux or machine runs in the Darwin slot. Siblings resume checks only after explicit lead release.
Done when
Every process under guard.py; after any kill, check for orphaned test
binaries (ps for .zig-cache/o/*/test). zig build exit 0; focused tests
by -Dtest-filter with real summary lines and exit codes; the unfiltered
full suite is the lead’s (Linux is deferred). Tee every run your report quotes
into a filed log with an exit file beside it, under toolchain/bench/step42/. RED output committed before GREEN.
Small commits as yourself with a Co-Authored-By line naming your model. Not
in scope: compaction points as a dimension the simulator varies by seed (say
in the report what it would take). Write your final report to
toolchain/STEP-42-REPORT.md and commit it. While anything runs, wait in the
foreground so your tab does not look finished.
Lead review of be64e8a5 (19 Sep, evening)
Not accepted. Preserve all worker RED/GREEN evidence and the final report; these findings require a fresh corrective session, not a resumed conversation.
- Completion is not last access. In
blocking.zig:118-121,246-250,done.store(true)precedes readingjob.write_fdand signaling it. The caller may observe done, close the fd and unwind its stack first. Fix the ownership/notification handoff in both pool and standalone paths, including scheduler-wait errors; copying the descriptor alone does not prevent close and fd reuse. The helper-only 20ms test does not exerciserunor this race: replace it with a deterministic production-path control and mutation that fails when early completion/early error return is restored. - ASan must know the fiber stacks.
mo_rt.c:7378,7508switches custom stacks with no sanitizer start/finish notifications. LLVM’s public fiber interface requires them. Implement correct ASan-only first-entry, switch, resume, reuse/destruction and scheduler-stack bookkeeping. Remove the warning stripping ine537abfeand its normalization-only test. Preserve raw diagnostics; expected-crash corpus cases must compare without suppression, and the 45-byte stale-answer mutant must still produce a real ASan error. Do not globally disable ASan checks or fake-stack protection. - Finish Part D, not just pending answers. Zig
Entry,Outgoing,Later,Answeredand corresponding native/scheduler/log holders still pair raw values with independent parcel pointers. Carry the distinct owned representation across every retaining boundary identified by the audit; migrate all callers. Explicitly distinguish no-packs/test-only values and traced process-state roots instead of wrapping roots as owned parcels. Compile-failing bare-value mutations must cover more than pending answers; runtime controls must cover transfer, drop, timeout, crash and cleanup. - Do not erase an effect to avoid retaining its value. The new
if (sim.packs) returninSim.emitdrops interpreter server events; native behavior alone is not permission to remove existing interpreter behavior. Preserve emitted values safely through the existing event lifecycle, without inventing a new external sink. Replace the test that merely asserts the list is empty with retention/compaction and simulator controls. - Keep ASan opt-in. The existing large-answer regression now unconditionally builds through Clang/ASan. Restore ordinary-runtime coverage there and select sanitizer builds explicitly; document the host Clang requirement for that mode. Retain both ordinary and sanitizer mutant proofs. Do not silently sample or omit the exhaustive stress sweep.
- Finish honest measurements. Echo minima rose 12.90%/8.60%, with substantial
variance and changing load; this establishes neither a zero-cost switch nor
a reliable regression magnitude. Inspect the new normal-mode hot-path
loads/checks, prepare controlled interleaved before/after measurements and
request a serialized slot. Fix a demonstrated avoidable cost, not noise.
Three corpus runtime benchmarks means three executable workloads in both
runtimes;
jobqbuild timing alone is not the third runtime benchmark. Preserve all samples and failures. Full-suite timings stay lead-owned.
Corrective write scope remains the original scope, plus
toolchain/bench/step38/measure.py strictly for complete per-sample reporting
and the measurement controls above. The prior worker changed that helper
outside its named scope; the lead reviewed the diff and now explicitly owns
this bounded scope extension. Keep benchmark corpus inputs unchanged between
trees, and do not break existing output-path callers. Report which prior
results no longer apply after corrective code changes.
Current safety evidence continuation
20 Sep 2026. Fresh OMP/GPT Sol/high worker step42-safety-sol; lead Astra
remains in w4:p1. Worktree toolchain-step42-safety-evidence, branch
toolchain/step42-safety-evidence, exact base
65b3dd37c4b2acc74bbdc425a723a163b69e50f8. The previous corrective worker is
no longer registered. Preserve its worktree and untracked review evidence
unchanged; no conversation resume, fork or import.
Static preparation only while Step44 owns runtime verification. Read the
lead inventory at
audit/evidence/2026-09-19/omp-resumption/step42-review-obligations-inventory.txt
in the main checkout. Its seven remaining obligations are not waived:
- Exhaustive normal stress sweep at the current runtime checkpoint.
- Exhaustive ASan corpus sweep with raw stderr; no warning suppression.
- Actual ASan use-after-poison from a current-representation stale-answer mutant at n=10, the 45-byte boundary; print the actual input.
- The same stale boundary caught by ordinary interpreter and native
-DMO_STRESS, separately from ASan. - Real packs=true retaining-boundary transfer, drop, absent/timed-out waiter, update crash, process restart/end and committed-event cleanup outcomes.
- Normal-mode hot-path analysis grounded in current source and the existing correction-only measurements; no zero-cost claim.
- Before/after unfiltered suite timing remains lead-owned, not waived; Linux remains deferred.
For this static stage, write only
toolchain/bench/step42/safety-evidence/**: executable probe/test drafts,
unapplied mutation or focused-test patches, and a command/coverage manifest.
Existing runtime, source tests, build files, guards, corpus, measurement
driver and final report stay frozen. Use existing harness conventions; do not
invent public hooks, instrumentation or a second runner. If a production or
test hook is necessary, name the exact prerequisite rather than applying it.
The manifest must map every outcome to its existing or drafted control and
exact command, expected diagnostic/count, guard, restoration and cleanup.
Account for current ParcelValue holders and custom-fiber sanitizer hooks.
Never read an auditor’s hidden suite or operate the lead’s audit inbox.
No build, test, runtime/probe execution, py_compile, formatter, linter, benchmark, debugger, machine/Linux operation, push or merge in this stage. No nested delegation. Preserve historical evidence and the measured 65b3dd37 runtime/driver checkpoint. The completed 60-sample comparison is against prior unaccepted be64e8a5, not whole-step overhead; do not rerun or rewrite it, including its documented load-heading error.
Done for this stage: bounded static drafts and concrete remaining command/ coverage manifest, source-based hot-path notes, exact prerequisites and changed paths reported to the lead, explicitly unexecuted; then HOLD for review and a serialized runtime grant. Do not write the final report yet.
Static review refinements, 20 Sep
The first bundle is not ready for execution. The n10 draft must fail if its required fixture is missing, print actual arguments and all labeled raw stdout/stderr (including successful runs), and reject unexpected runtime stderr. It currently hides warnings when exit/stdout match. Build failures must expose their diagnostics. Ordinary poison may produce a recognizably wrong value or trap, as PartB specifies; ASan still requires use-after-poison.
The delayed-send control must exercise a successful update’s outbox-to-timer commit, not only an external sendLater call or a crashing update. Pending answers must reach the real waiter consumption path, not only a test’s manual fetch/free. Crash-held values should originate in the actual crashing update. Container emptiness and pointer copies alone are not reclamation proof.
The proposed private observable is approved for unapplied draft only: actual Parcel construction/destruction in Zig and C, including Zig’s errdefer failure cleanup; thread-safe and case-isolated; no public Mo hook, new CLI, normal/stress hot-path cost or default activation in ASan sweeps. Specify native coverage and one-free-at-a-time mutation controls. No manual free may manufacture the expected production outcome. This resolves the design prerequisite in favor of a bounded test-only observable rather than assuming Apple LeakSanitizer support or a clean global-allocation baseline.
All writes remain under the evidence-draft directory. No source application, execution, final report or acceptance yet. The preserved60-sample comparison is unchanged. Original best-of-five suite and stress timing requirements remain unwaived; the worker does not own the unfiltered suite.
Revised static review, after Step44 acceptance
Step44 is accepted at main db515f9b; this tree remains frozen65b3dd37 with no source application/runtime grant. Revised n10 diagnostics and real delayed commit/ask/crashing-holder paths address the first review. The test-only audit is source-plausible, not compiled or exercised. Remaining static work:
- Complete the errdefer test using the installed Zig0.16 allocator API and demonstrate real allocator cleanup as well as observer balance.
- Remove the incidental formatted crash-state assertion after Relay gained a held field; retain semantic rollback checks. Require Error(Timeout), not any outer Error, for the late-answer path.
- Finish native lifecycle fixtures/argv and isolate each omission’s expected count. The first C drop mutant targets route_packed’s no-turns path, while native main delegates to turns_answer: target the actual main-mode free.
- Arena-based packed tests do not establish producer-region invalidation. State that limit and exercise dynamic payloads across real safe points in main-mode controls; counters are not whole-heap LeakSanitizer.
- Keep ten unfiltered suite samples (five before/after), but the original Numbers asks five current exhaustive stress times, not a before/after stress comparison against a baseline lacking that test. Both runtimes remain included. Locate exact whole-step three-workload before/after evidence; preserved be64-to65 correction samples are not that comparison.
Metadata/source inspection and git apply —check are allowed, never application
or execution. Lead readings: audit/evidence/2026-09-20/step42-static-review-02.json
and step42-native-drop-path.json; the latter preserves all25 reviewed drafts.
Further source review: observable values and counts
Review03 confirms the corrected Timeout, real native drop site, allocator failure draft and explicit proof limits. All remain unapplied/unexecuted. Before a private-bundle execution grant:
- Print case-labeled actual allocated/destroyed/live counts on passing and mutant Zig runs, before any balance assertion can stop the test. The errdefer case must expose allocator counts and its audit snapshot before the expected mutant failure.
- Compare complete dynamic nested payloads, not list lengths; read the Logger’s retained values with its single final observation, preserving the one-eviction case. Observe the supposedly untouched Sink instead of claiming a check that the fixture does not perform.
- Distinguish ten ordinary native paths from the two end_process paths reached through the private quiescent production-sweep hook.
- Frozen65 is pre-composition. The lead must freeze accepted-main-plus-Step42 before/after checkpoints before timing; the ten suite, five current stress and60 workload sample obligations remain, with historical evidence intact.
Source tags/witnesses and the delivered static-only directive are in
audit/evidence/2026-09-20/step42-static-review-03.json. Server correctness
owns the runtime slot; no memory application/execution is authorized.
Composed source preparation after revised oracle review
The revised six artifacts address review03 at source level: paired Zig counts precede balance assertions; native waiter/end compare all nested parts; Logger returns all16 ordered labels; Sink is actually observed empty; and the native map distinguishes ten ordinary paths from two private-sweep end paths. No combined patch or control has run. Review04 preserves the complete drafts.
Lead prepared lead-step42-composed-4e0bfc0e at accepted-main4e0bfc0e and
started an uncommitted merge of65b3dd37. Merge base is2f669902. Only corpus.zig
and sim.zig have textual conflicts; automatic merging elsewhere is not
semantic proof. The same worker owns static resolution and all necessary
Step44 retention-caller migration in this separate tree. Original frozen
source/drafts, historical evidence, guards and measurement drivers stay
unchanged. Named source staging is permitted, not a commit or runtime.
The worker must preserve all accepted Step44 controls and Step42 exhaustive
gates, and check draft applicability without applying instrumentation or
mutants. BEFORE is anchored at4e0bfc0e; AFTER is not frozen. No timing grant.
Server formatter verification owns runtime; memory remains static-only.
Exact scope, receipt and reviewed source:
audit/evidence/2026-09-20/step42-static-review-04.json.
Composition review: ownership on failed enqueue
The worker resolved corpus/sim conflicts while preserving both full corpus gates, both chunks simulator controls and the packed-event test. Individual applicability is30/31: the packs-true patch’s last context assumes the event test immediately follows an older fault test; Step44 now intervenes. Only a private copy may receive a context-only rebase; frozen drafts stay unchanged. Inherited historical-log whitespace is preserved, not “cleaned.”
Source review found a blocker, not a runtime result: sources.send packs
inline before a fallible enqueue. Enqueue itself inserts the mailbox entry
before fallible runnable marking and trace insertion; existing caller cleanup
can therefore free a published value. A local errdefer alone is unsafe.
Required contract: enqueue transfers ownership only on success; failure
leaves it with the caller and publishes no mailbox/trace/runnable reference.
The worker must propose the smallest failure-atomic change, audit fresh,
popped and batch-held callers, preserve FIFO/synchronization, and design
allocation-failure proof before production edits. AFTER remains unfrozen.
audit/evidence/2026-09-20/step42-composition-review-05.json preserves the
source witnesses and exact static-only scope.
Composition review06 independently checked the rebased private patch: all195 changed rows match the frozen original; applicability0, unapplied. The four-file source repair is now authorized without runtime. Keep existing markRunnable: reserve queues/trace first, then its fallible resize/publication, then an infallible commit under the runtime lock. No split scheduler API is needed. Use prefix cursors, not per-slot sentinels, for error-only batch cleanup; cover already-owned emits/answers if an earlier commit phase fails. Adjacent answer-map and mailbox-to-log ownership seams are explicitly in scope. Three permanent allocation-failure behavior controls plus private parcel controls are authorized as source, not execution. Review06 records the exact limits; normal, ASan, mutant, reclamation and timing proof are still withheld.
Independent frozen-source review07 read the native retained-value migration, fiber switching/retirement, blocking completion and region/stress changes. Two additional bounded corrections are authorized: compile ASan-only fiber context fields/initialization out of normal builds, and remove the test that only asserts three internal budget assignments. Real stress/mutant proof remains mandatory; the observable blocking lifetime/error control stays. No new native ownership redesign or runtime grant. The normal path still has runtime budget loads and a changed retained-log layout; timing proof cannot be replaced with a zero-cost assertion.