What 8,523 passing checks leave to test

A test suite can pass thousands of checks while leaving the intended application untested. The number is still valuable when it identifies precisely what was exercised. Trouble begins when a report lets a narrow computational result stand in for a broader claim about the system’s usefulness.

WITNESS is a proposed instrument for checking disputed drawing evidence. An independent run of its recovered mathematical core passed 8,523 self-test checks on October 18, 2026. The receipt also records 1,458 interval cases and 8,192 admitted intermediate masks examined. These counts describe related parts of the same test run, in which interval cases require examining admitted masks, so adding them together would obscure what was exercised and suggest independent demonstrations of application accuracy that the receipt does not contain.

The kernel operates on mathematical objects and decoded image samples. Its tests cover small and synthetic domains, including image connectivity and critical-threshold behavior. The complete PDF interpreter, evaluation on real drawings, and measured effect on a model’s decisions remain further work. The next test plan should preserve the kernel result while addressing those missing claims directly.

Begin with the claim you want to add

If the proposed claim is reliable extraction from PDF sources, test extraction. Supply files with known rendering features and compare the recovered observations against a trusted reference process. Include revisions, covered marks, reused objects, and unsupported features that influence the inspected region, checking that the parser identifies content it cannot reconstruct rather than returning a blank area that a downstream connectivity test might incorrectly accept as evidence of absent linework.

If the claim is better engineering interpretation, a different evaluation is needed. Define the questions, expected answers, acceptable unresolved outcomes, and reviewer process. The unit of measurement should be a supported interpretation of a specific drawing question, with the evidence needed to inspect disagreements.

Do not combine these stages into one overall success percentage too early. A system can extract samples correctly and interpret their meaning incorrectly. It can also reach the right answer from a defective extraction by relying on unrelated context. Retaining stage-level results makes both conditions visible.

An illustrative real-drawing evaluation plan

Build a permission-cleared set of drawing questions covering several relevant conditions. Include crisp vector linework, scans with faint regions, repeated symbols, occluded marks, and cases where the necessary legend appears elsewhere. Hold some documents aside before developing the acquisition procedure so the final evaluation does not reuse only familiar examples.

For each question, have qualified reviewers identify the answer and supporting locations, then preserve disagreements that the available source cannot settle. An unresolved reference answer is useful when the task genuinely lacks decisive evidence. The system should not be rewarded for selecting a confident answer where reviewers determined that the source is insufficient.

Compare a defined baseline workflow with a workflow using the evidence tool under comparable conditions. Keep the model version, task instructions, available source material, and resource allowance recorded for each comparison, repeating runs where model variability could change the outcome and reporting the distribution of results so the study does not depend on selecting its most favorable attempt.

This is an illustrative evaluation design, not a completed WITNESS experiment. Its purpose is to make the additional evidence requirement concrete. A successful kernel run alone supplies none of the proposed real-drawing accuracy measurements.

Measure useful uncertainty and acquisition cost

Count correct supported answers and incorrect answers separately from unresolved results. Examine whether the system asks for evidence that actually distinguishes the competing interpretations. An additional crop is not automatically useful merely because it was requested and returned.

Record the cost of decoding, rendering, inspection, and model calls when those operations are part of the workflow. A small returned image may require substantial prerequisite work. Report failure cases where the tool reaches its budget before acquiring decisive context, including whether another permitted operation could have answered the question.

Inspect source association errors specifically. A model may read a dimension correctly but attach it to the wrong feature, or confuse repeated depictions with repeated physical parts. These errors can survive perfect arithmetic and should appear as distinct categories in the evaluation report.

Keep the release claim versioned

Tie the evaluation to the exact kernel, parser, acquisition policy, and model configuration tested. When a component changes, determine which results remain applicable and rerun affected cases. Retain failed examples as regression material while protecting any restricted source content.

The next defensible WITNESS report would state both the reproduced kernel result and the completed results of whichever application stage was actually tested. Until those tests are run, the work can accurately report its 8,523 passing checks and identify the remaining parser and real-drawing experiments, with their acceptance criteria ready for execution.

Source notes

Based on the recovered WITNESS kernel and independent October 18, 2026 test receipt. All proposed real-drawing evaluation procedures are illustrative and unexecuted; no application-accuracy result is claimed.