Cassette Build Report 013 — Green Checks Did Not Prove the Work
Two reviews and a later S01 repair showed that Cassette's first green steps had not tested their own accounting, authority, or completion claims.
Scope note: This post covers the S01 and S02 review chain and GPT-5.6 Sol Ultra’s later definitive repair. It reports the recorded defects and repairs; it does not generalize from them to every agent or test suite.
S01 and S02 closed green in under an hour. I distrusted the speed and asked for a static review.
Claude Fable 5’s first review found six defects. A clean checkout had never been tested. Tracked Python bytecode could evade accounting. The dependency check searched for == instead of parsing pins. Citation checks could accept Q999. Error payloads checked membership but not field types. The DONE ritual required a commit to contain its own hash, impossible in the commit that created it.
The sharpest finding was about the checks themselves. The stages had written narrow tests, passed those tests, and overlooked defects in the tests.
The second review found four more defects in the repairs. The tracked-artifact check could fail open. Wildcard versions still passed as exact pins. The authority loader knew rows but not the assertions inside them. The close commit broke the commit law it had introduced.
Fable succeeded at finding defects that the authoring path had not anticipated. It failed to make its own repair process stay within the contract, a problem that became the subject of the next post. Both facts matter.
When Drew asked GPT-5.6 Sol Ultra to close S01 definitively, Sol first crossed a boundary. The request was for a static completeness review, but it ran the existing tests and prepared a clean-checkout probe. Drew stopped it. A check is not permission to execute. A finding is not permission to repair.
When the review request was repeated, Sol read the code and found more faults. The authority loader admitted almost every lower-case snake-case item in the matrix instead of only assertion-bearing fields. The commit law searched for three phrases anywhere in a message, so prose that mentioned the phrases could pass without answering the required questions. Git-law branches lacked isolated fixtures, and a missing Git root was treated as a successful skip despite the fail-closed claim.
Sol then repaired S01 by narrowing the authority reader to the matrix fields that actually contained assertions, making missing Git repositories violations, and requiring three exact line-anchored nonempty commit fields. The tests used separate temporary repositories and real indexed bytecode artifacts. The authority fixture proved that nearby cases, operations, traces, duties, gates, and failure injections remained outside the accepted set.
The fresh-context reviewer returned two findings about the review prompt itself. Sol did not turn those findings into code automatically. It clarified the repository’s semantics, added fixtures for the boundary between a gate row and a nested gate value, and ran a second pass.
The final result was not “the agents reviewed the work.” The result was a chain of authorship and evidence: Fable found defects, Drew set the authority for the repair, Sol changed the implementation, and the tests proved specific boundaries.
That is a more useful account of AI-assisted engineering than a green suite. A green suite reports a state. It does not identify which claims were tested, which paths were never exercised, or who was allowed to change the code after a finding.
Cassette now treats review as part of the evidence record. The agent’s confidence is not the proof. The proof is the artifact, the fixture, the scope, and the result that survives a fresh reading.