Cassette Build Report 059 — 315 Green Tests Missed the Import
Cassette's L01 gate was green until an independent review found reflective Python imports outside the two permitted runtime owners.
Scope note: This report covers the L01 import-confinement defect and its repair. The final proof remained a controlled macOS machine result; no physical-drive or real-model claim follows from it.
L01 had 315 passing tests and a clean ledger. I asked for another review anyway.
The first closeout looked complete. Cassette had replaced the old capacity machinery, updated its hardware rules, and passed the full machine gate. The evidence was not invented. The tests ran. The ledger was green. The stage had a recorded completion.
The independent review found three ways to retrieve Python’s module loader outside the two files permitted to own the MLX runtime.
That sounds like a narrow security trick. It was more consequential than that. Cassette’s architecture depends on a small number of declared owners. Numerical execution belongs in the pager and trainer. Other product and tool files may ask those owners to perform work, but they may not quietly acquire the runtime themselves. If reflective Python can route around that boundary, the codebase can grow a second execution path while its ledger still reports one.
Codex reproduced all three imports against the prior checker. The existing test went red as soon as the first form entered the fixture. The repair remained inside the existing static checker. Imported loader namespaces could expose only a directly inspected loader call. A dynamic import could not return another import namespace. Ordinary literal standard-library imports still worked.
That first repair was not the end. Tracing the rule through the ledger showed that Q30 checked product files but skipped tools, even though its own fixture named a forbidden tool path. Extending the same rule to tools immediately found an existing teacher-corpus generator obtaining MLX through the pager’s internal runtime. Codex moved the conversion into the pager and regenerated the protected corpus byte for byte. The ablations and identity stayed the same.
The three supplied examples were real, but the useful review had to continue into the governing question: where does this rule actually run, and what else becomes visible once its declared scope becomes true?
Codex then made a familiar mistake. It started the broad gate before finishing its direct review of the checker. Two more import forms appeared while the dependent runs were already in motion. Python supplies an implicit __builtins__ namespace. Its import function can also resolve a literal module name relative to a package when given the proper context. Both forms passed the earlier repair.
The agent stopped the runs, preserved them as failed candidates, and tightened the rule again. That decision matters. It would have been easy to let a nearly complete suite finish, patch the checker afterward, and describe the earlier result as close enough. Cassette’s evidence model does not permit a run to change meaning after its source changes.
The current checker makes the confinement boundary visible:
loaders = {"__import__"}
namespaces = {"__builtins__": "__import__"}
machinery = {"importlib": "import_module", "builtins": "__import__"}
violation = [f"{rel}: unresolved or MLX dynamic import outside {sorted(MLX_ALLOWED_FILES)} (Q30 confinement)"]
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name.split(".")[0] in machinery:
namespaces[alias.asname or alias.name.split(".")[0]] = machinery.get(alias.name)
The excerpt comes from public tools/ledger.py. The complete rule also rejects reflective access, computed loader aliases, and relative context. It does not claim to confine arbitrary Python behavior. It is a strict source rule for this repository, and it knowingly refuses some benign namespace inspection outside the two owners.
The next full run passed 314 tests and failed one. The failure was not product behavior. The off-tree proof collector forced a shared pytest temporary directory into a nested run. Removing that override made the isolated control pass in less than a second. Codex retained the failed attempt, changed the queue record rather than the executable source, and reran the complete gate.
The corrected result was 315 passing tests, no failures or skips, twelve component controls, twelve deletion controls, and twelve executable bypass failures. The final repair added fifteen authored executable lines. The same Codex agent performed the repair review, so the record does not call that review independent.
The number 315 means less to me than it did before this exchange. A suite can prove exactly what it contains and still omit the path that matters. Green is not a general property of a system. It is a statement about a defined set of observations.
My contribution was not to discover a clever Python escape. The reviewer did that. Mine was to ask for another review after the closeout already reported 315 passing tests, which gave the reviewer the exact frozen claim against which to supply the missed loader paths.
The agent’s contribution was to reproduce those paths, follow the rule through its real scope, stop contaminated runs, and leave a narrower claim than “Python is confined.” The resulting contract is more useful: inside this governed source tree, these known acquisition paths cannot place MLX outside its two declared owners without making the ledger fail.
The accepted L01 result now means that the enumerated acquisition paths fail outside the pager and trainer across every governed product and tool file in the tested revision.
