Cassette Build Report 015 — The Tests Passed, but the Machine Was Elsewhere
Opus 5 and Opus 4.8 reported platform limits, while GPT-5.6 Sol Ultra had to check Cassette's S03 implementation against its clauses instead of trusting its status records.
Scope note: This covers the Opus 5, Opus 4.8 High, and GPT-5.6 Sol Ultra handoff around S03. It distinguishes what each session could read, run, repair, and prove; it does not treat one environment as universal.
“The tests pass” is a claim with a machine attached to it. I learned that by watching three agents describe the same repository from three different execution surfaces.
Claude Opus 5 stated the boundary plainly. Its file tools could read and write my repository, but its shell was an isolated Linux container. Cassette pinned Python 3.13 and required Metal, MLX, F_FULLFSYNC, and a real removable volume. Opus 5 could not run those rows on my MacBook Air, so it closed no queue step.
That was a success in evidence discipline. Opus 5 did not turn a Linux result into proof of Apple behavior. It also misread the testimony around it, first treating the stop order for Fable 5 as its own and then assigning the S01 account to itself by elimination. The source had to correct its attribution before its platform statement could be trusted.
Opus 4.8 High reported the same kind of boundary in a different session. It initially said the pinned test could not run because pytest was missing. That was too strong. The pinned package could be installed in the available environment and the suite passed. The real limit was narrower: the session could author Apple-silicon code but could not start the MLX, Metal, or filesystem rows on the Mac through its available shell.
The distinction mattered when both sessions treated S03’s existing commit and green ledger as evidence that the step was done. The status was not enough. GPT-5.6 Sol Ultra took the step back to the code and the actual clauses.
Sol found that the generator emitted only a partial Q50 record. Q31’s RunRequest and CapabilityProfile did not exist. The fixtures reproduced the reduced contract set produced by the generator, so they could pass while the required records were absent. The files called themselves schemas without being the Draft 2020-12 documents that Q31 required. The ledger compared generated bytes with a digest stored beside them, so someone could change both and preserve the appearance of integrity.
Drew stopped Sol before it could treat the status record as authority. That interruption mattered because the earlier sessions had already made a green commit look like a completed step. Sol then ran the pinned command on the Mac, replaced the circular proof with an independent contract, and generated twelve schemas covering the required records. The validator stayed within the Python standard library. The tests used complete golden records, rejected missing and malformed nested values, and caught a coordinated edit to a generated file and its digest.
The first repair exposed another defect in the commit record itself. Literal \n characters had been passed inside one shell string, so the ledger found one required field and missed the other two. Sol amended the message into real paragraphs, reran the clean-checkout proof, and closed S03 only after the repaired code, tests, ledger, and commit record agreed.
The agents did different things well. Opus 5 stated the platform boundary. Opus 4.8 corrected its package claim and identified under-covered clauses, but initially overread its session limit. Sol performed the implementation repair, but only after Drew forced it to inspect the code instead of the records. The common lesson is not that one model is the winner.
It is that evidence has units. A suite that passes in Linux does not prove an MLX row on Apple silicon. A status record does not prove its clauses. A model’s statement about its own reach does not replace a description of the session that supplied that reach.
Cassette applies that rule to the agents building it. The machine, the contract, and the result must stay attached.