Cassette Build Report 033 — The Reviewer's Fix Was Worse Than the Implementer's
A failed R2 probe looked like an unfixed defect until the reviewer discovered that the probe imitated an environment without being one.
Scope note — This report covers the R2 environment-accounting review after the S01/S12 remediation. It is about the validity of the probe and the quality of the proposed repair, not a general claim about environment isolation.
The remediation had already been difficult to close when Opus 5 Extra returned to one of its own findings. R2 said the ledger counted installed dependencies inside the checkout as Cassette source. The original defect was real. Opus planted a file and produced the exact violation.
The repaired test looked broken when Opus planted the same shape again and got the same two violations, so for a moment the implementation appeared to have changed nothing even though the probe had not recreated the condition the repair recognized.
The probe had changed the condition — it created a site-packages tree without the pyvenv.cfg file that Python writes when it creates an environment, so the test was not exercising an environment at all; it was exercising a directory that resembled one.
The repair does not exclude a directory named .venv. It starts with Git-owned files and intentionally introduced, nonignored files, then excludes an untracked file when a real pyvenv.cfg exists in an ancestor. The boundary is ownership plus the interpreter’s marker, not a folder name.
The source makes that choice directly.
for rel in untracked:
if not eligible(rel):
continue
parents = [rel.parent, *rel.parent.parents]
if any((root / parent / "pyvenv.cfg").is_file() for parent in parents):
continue
owned.add(rel)
The ledger’s environment boundary ignores an untracked Python file only when the file belongs to a marked environment. It does not prove that every environment is safe. It proves the narrower rule that a foreign interpreter is not Cassette source while an untracked Cassette file remains governed.
The review could call the repair broken only after it had created the same environment that the repair was designed to recognize, because a directory with a familiar name is not evidence of interpreter ownership and the ledger’s contract depends on that distinction.
Once the marker was added, both halves held. Two differently named environments containing MLX produced zero violations and left the product line count unchanged. An uncommitted compiler.py importing MLX was caught immediately by the header and confinement checks. The repair preserved the second clause instead of solving the first by making new source invisible.
My first repair instinct would have been to add .venv/ to .gitignore. That would have been a worse fix. It would have silenced a directory regardless of its ownership and could have hidden governed source placed there. The reviewer found the defect and would have shipped a broader blind spot than the implementation had.
That is not a small footnote about test hygiene. A review has two jobs that models often merge — find a condition that violates the contract, and understand the structural correction that preserves every other clause. The first can be right while the second is wrong. The second can be wrong in a direction that makes the next test look cleaner.
Opus also recognized the pattern in its own probe errors. It had constructed the appearance of a condition instead of the condition, then read a correct refusal as a defect. The same thing happened in the S13 numeric review when an incompletely resealed certificate stopped at an earlier digest guard. A probe that fails early has told the reviewer where the implementation refused the input. It has not yet told the reviewer about the later guard that was meant to be inspected.
I want this retained as an agent comparison without turning it into a model ranking. Opus was good at finding pressure in a review and honest about its own malformed fixtures. Its proposed .venv exception would have weakened the ownership law. The implementation agent was better at preserving the contract in this narrow repair. The difference came from the condition each one actually tested.
R2 closed after the marked environment reproduced the intended boundary, the governed untracked file remained visible, and the complete proof stayed green. The lesson is plain enough to use elsewhere — before calling a repair broken, make sure the probe has built the thing the repair is meant to recognize.
