# Agentic Engineering Atlas Work through seven stages: foundations, evidence pipelines, coordinated engineering, controlled action, verification, organizational systems, and integration. Each lesson develops a project-derived method or a clearly labeled illustrative scenario, with failure modes and evidence limits. Try each exercise before opening the model answer. Use the scored rubric to identify what needs repair; a score is a learning aid, not a certification of production readiness. Source-grounded examples remain distinct from constructed practice data. Public edition: Drew Wiberg’s personal project methods, mistakes and innovations retain their stated evidence limits. Nonpublic organizational status, security and configuration details are excluded. Those teaching passages use explicitly illustrative scenarios. Public essays are linked; nonpublic project references have minimal bibliographic descriptions without account or document links. No private source document is included. # Measure the information problem before automating Use query evidence to decide which information-system intervention is warranted before selecting an agent architecture. ## Objectives Separate retrieval failure from missing knowledge and unclear terminology. Connect a proposed intervention to an observable user outcome. Include maintenance cost when choosing between search, navigation and curation. ## Evidence boundary and originating problem The query-data work describes a feedback loop connecting intranet searches with retrieval tuning, navigation, knowledge-gap filling and course planning. Related retained essays describe semantic-network work, folder naming rules and a partial expertise graph. These are concrete knowledge-engineering methods. They are not evidence that a language-model agent performed the work. A proposed author–resource–user extension must remain separate from the partial graph actually described. A particularly useful constraint comes from the findability work: automating every link map was reconsidered because upkeep mattered. The lesson is not to minimize automation universally. It is to compare the durable user benefit of an intervention with the work required to keep its representations correct. ## Classify the information failure before choosing a mechanism A query is a trace of an attempted task, not a complete statement of the task. Preserve the query, available context, candidate results, selected result and whether the user could finish. Cluster related queries cautiously: identical words may express different intents, and different words may refer to the same concept. Use three initial diagnoses. A retrieval failure means a suitable source exists but is not found. A knowledge gap means no suitable source is available. A terminology mismatch means the user and the collection describe the same subject differently. The interventions differ: ranking or navigation for the first, new or maintained content for the second, and governed aliases or concept mappings for the third. A chatbot cannot compensate reliably for absent authoritative material merely by producing a fluent answer. ## Choose an intervention and a maintenance owner Record the evidence for the diagnosis and at least one competing explanation. Then choose the smallest change that could discriminate between them. A query rewrite experiment may test terminology without committing to a graph rebuild. A manually curated landing page may test demand before a costly automated map. For each derived representation, name its source, update trigger, responsible owner and stale-state behavior. A link map whose sources change faster than it is maintained can increase confidence while reducing accuracy. Treat the cost of checking and correcting it as part of the system, rather than unpaid work outside the evaluation. ## Measure task completion rather than interface activity Compare the same task set before and after the change. Record successful completion, wrong-result selection, unanswered tasks and human review effort. Search count, page views and graph size are diagnostic measures, not substitutes for completion. A reduction in searches could mean improved findability or user abandonment; the outcome evidence decides which interpretation is supported. Preserve causal documentation: what changed, why it was expected to help, what was observed, and what remains confounded. The retained conceptual work proposes this outcome-linked revision discipline; it does not establish a particular measured improvement for every intervention. ## Worked example Consider a synthetic set of twelve failed information tasks modeled on the query-feedback method. In five, an approved source exists under an unfamiliar term. In four, no approved source exists. In three, the source is found but its owner and applicability are unclear. A single “better semantic search” project addresses only part of the problem. The five terminology failures justify an alias/navigation trial; the four gaps need curation; the three authority failures need provenance and applicability metadata. These counts are invented exercise data, not results from the original query logs. ## Exercise Design a bounded intervention plan for the twelve-task fixture. Assign each task group a diagnosis and a competing explanation. Choose one low-cost test per group and name its maintenance owner. Define an outcome measure that can reject the intervention. Explain what a larger graph or more searches would fail to prove. ## Model answer A satisfactory plan assigns alias trials to the five terminology cases, content creation or explicit unresolved responses to the four gaps, and authority metadata plus review to the three applicability cases. It must not count a generated answer to a missing-source task as successful knowledge retrieval. Suitable tests use the same task questions with blinded outcome assessment where feasible, and record review effort. Alternative interventions are acceptable if their observation can distinguish the stated diagnosis from its competitor. ## Rubric 3 points: All three diagnoses lead to different, justified tests. 2 points: Every proposed representation has an owner and stale-state behavior. 3 points: Success measures task completion and wrong conclusions, not activity alone. 2 points: Synthetic counts and the source’s evidentiary limits remain explicit. ## References S038, S029, S039, S040, S030, S041, S042, S043 # Turn RFP questions into quotation-backed analysis Build an RFP-analysis chain in which every material conclusion can be inspected against its supporting quotation. ## Objectives Separate document preparation from model interpretation. Design source-span records that survive summarization. Distinguish a reported custom pipeline from independently demonstrated end-to-end reliability. ## Evidence boundary and originating problem The RFP work proposes conventional NLP or semantic processing before LLM assistance and human review. A revised artifact makes the distinction between API access, browser interaction and context limitations more explicit. A bylined presentation reports a custom Python/API analysis pipeline and dashboards containing direct supporting quotations. These records support the method and the reported implementation. They do not constitute a current runtime audit or prove that every extracted conclusion was accurate. The distinctive teaching problem is how to preserve a reviewable connection between a document, a question and the answer produced from it. A dashboard is useful when it exposes that connection; it is not itself evidence that the conclusion is supported. ## Make preparation a reversible transformation Keep the original document immutable in the analysis workspace. Produce an extracted representation with document identity, extraction version, page or section locator and text spans. Record omissions such as unreadable tables or missing appendices. A later model should not silently treat the clean text as a complete substitute for the source. Separate deterministic preparation from probabilistic interpretation. Basic normalization, section detection and span bookkeeping should be observable and repeatable. Model calls can then classify or explain prepared evidence. A failure at extraction must be visible before interpreting the absence of a requirement as evidence that no requirement exists. ## Ask bounded questions and retain quotation-level support Represent the task as a question or criterion, not only as “summarize the RFP.” For each response retain the conclusion, supporting source spans, any contradictory spans and an uncertainty state. A quotation must be a real span from the retained source, not a model’s reconstruction in quotation marks. Context limits change the design. If the source is partitioned, record which partitions were inspected and how results are reconciled. A requirement split between a main clause and an exception in an appendix cannot be resolved by taking whichever chunk produced the strongest answer. The aggregation stage must search for qualifications and preserve unresolved conflicts. ## Review the evidence path as well as the wording A reviewer should be able to move from a conclusion to its exact supporting passage and then to the original document. Test the path in both directions: a correct quotation may still be irrelevant to the question, while a plausible conclusion may cite the wrong span. Include unsupported, contradictory and missing-source cases in development. API success, a completed browser action and a populated dashboard are different observations. Record each at its proper boundary. An HTTP response proves that a request returned; it does not prove correct extraction, appropriate interpretation or a decision fit for use. ## Worked example A synthetic RFP says “The submission must include a delivery plan.” An appendix adds “A delivery plan is waived for the preliminary expression of interest.” The task concerns the preliminary stage. A chunk-level answer that quotes only the first sentence produces a wrong mandatory requirement. The supported output is that the requirement is waived for this stage, with both passages and the stage assumption visible. If the stage is unknown, the result must remain conditional rather than choosing a convenient interpretation. ## Exercise Create an evidence record and review procedure for the conflicting-clause fixture. Write one record for the main clause and one for the exception. Specify the aggregation rule and the unresolved state. Define tests for an invented quotation, a wrong locator and an omitted appendix. Show what the dashboard must expose to a reviewer. ## Model answer The answer must retain both clauses, make the preliminary-stage assumption explicit and link to both exact spans. With a missing appendix, the system may report only a provisional requirement with incomplete coverage; it cannot claim the exception was absent from the source. A fabricated quote fails literal span validation. A correct quote with a wrong page fails locator validation. The dashboard needs enough context to distinguish these failures, not just a green completion indicator. ## Rubric 3 points: The conclusion includes its applicability condition and both source spans. 2 points: Missing coverage produces a qualified or unresolved result. 3 points: Tests separately detect quotation, locator and interpretation errors. 2 points: Reported implementation is not presented as independently audited reliability. ## References S044, S045, S028 # Expand a taxonomy without confusing volume with correctness Use the taxonomy-expansion work to separate extraction, candidate generation, classification and expert acceptance. ## Objectives Audit duplicate concepts and classification rows independently. Prevent generated labels from becoming governed concepts automatically. Design semantic review that cannot be replaced by structural counts. ## Evidence boundary and originating problem The surviving taxonomy artifacts include a ten-term seed and corresponding broader-term, narrower-term and definition output. An authored report describes Python extraction, model-assisted expansion and classification of an 822-concept seed, retry recovery and expert refinement to 36 top concepts. The recovered workbook was separately counted: 1,184 classification rows, 1,182 distinct concept strings and seven families; summary and mapping sheets each contain 36 rows. These observations have different scope. The workbook counts are inspectable structure. The reported process is evidence of the author’s implementation account. Neither alone establishes semantic correctness. The six-versus-seven scheme inconsistency in accompanying descriptions is a reason to reconcile the artifacts, not to silently select the tidier number. ## Give each transformation its own output type Separate extracted term, generated candidate, proposed relationship, reviewed concept and accepted classification. A candidate should carry the seed, prompt or procedure version, source context and generation status. A concept should have a stable identity independent of its display label. Otherwise a spelling change can look like a new concept, while identical strings with different meanings can be merged incorrectly. Keep generation retries distinct from semantic revision. Retrying a failed API call may restore transport success; it does not improve the truth of the generated relationship. When retrying, retain the candidate identity or a reproducible request key so that duplicate outputs can be detected. ## Audit the workbook before interpreting its meaning Count rows, distinct identifiers, distinct normalized labels, relation types and family assignments separately. A classification table can legitimately have more rows than concepts when a concept has multiple assignments. Conversely, repeated rows may be accidental duplication. The count discrepancy identifies a question; it does not answer it. Check whether the 36-row mapping and summary sheets refer to the same accepted concepts and whether their joins are complete. Reconcile the documented number of schemes with the observed family values. Report an unresolved inconsistency if the artifact does not establish which statement is authoritative. ## Review meaning using explicit error categories Select a review sample that includes rare families, ambiguous terms, duplicates and boundary cases. Define errors before inspecting outputs: incorrect broader relation, circular hierarchy, conflated meanings, unsupported definition and wrong family assignment. Expert review should record the decision and rationale, not merely overwrite a cell. Measure semantic agreement or error rates only against a labelled review set with a stated sampling method. The number of generated terms, successful retries or populated cells is not a semantic-accuracy metric. Preserve rejected candidates so that later expansions can be tested against known failure modes. ## Worked example A synthetic table contains six rows: A→Family 1, A→Family 2, B→Family 1, C→Family 2, C→Family 2 and D→Family 3. It has six rows, four concepts and one exact duplicate assignment. The multiple assignment for A may be valid; the duplicate C row requires investigation. If D is “river bank” while another source uses “bank” for finance, label normalization alone cannot establish identity. A structural cleanup and a semantic decision are separate operations. ## Exercise Audit the six-row fixture and design a small review protocol. Report row, concept and unique-assignment counts. Identify which discrepancies can be resolved mechanically. Define three semantic error categories and a deliberately difficult review sample. State what the original workbook counts do and do not demonstrate. ## Model answer The fixture has six rows, four concepts and five unique assignments. The exact duplicate C assignment can be flagged mechanically, although removal should still preserve an audit record. A’s two families need a policy decision about multi-membership. The two senses of bank require contextual semantic review. A strong answer distinguishes observed structure from correctness and does not use the retained workbook’s scale as an accuracy claim. ## Rubric 3 points: All three fixture counts are correct and independently defined. 3 points: Multi-membership, duplication and homonymy are treated differently. 2 points: Semantic review includes adversarial or boundary examples and recorded reasons. 2 points: The source’s reported process and counted structure retain separate labels. ## References S046, S047, S048, S049, S027, S028 # Define what each kind of evidence can establish Use a claim ledger to prevent requirements, reports, artifacts and tests from being promoted into stronger evidence. ## Objectives Match an assertion to the observation that can support it. Record untested conditions alongside a successful result. Distinguish a mathematical certificate from an engineering conclusion. ## Evidence boundary and originating problem The corpus contains requests for a fivefold improvement, reports of completed work, recovered specifications and a reproduced WITNESS kernel result. They cannot be combined into a single “completed” category. The WITNESS self-tests reproduced 8,523 checks, including interval cases and intermediate masks. That is stronger evidence for the recovered kernel’s tested behavior than a statement that the tests passed. It remains narrower than a full PDF-understanding or real-drawing performance claim. The source-derived practice is to bind a claim to the specific artifact and observation that can establish it. This lesson provides the ledger used throughout the remaining course. ## State the claim before choosing its test Write the assertion in a form that can be contradicted. “The agent is reliable” is too broad. “After a timeout, the workflow does not issue a second external effect until it reconciles target state” identifies a behavior and a failure condition. Record the environment, input class and version boundary. A passing result outside those conditions may be irrelevant. Different questions require different evidence. A user instruction establishes authority or desired behavior. A source artifact establishes its retained contents. A runtime observation establishes what happened in a particular run. A test can establish a bounded property but cannot grant permission or prove that a source is authoritative. ## Separate observation, report and interpretation A ledger row should identify the claim, source identity, locator, source version, evidence class, observed result, limitations and the interpretation being made. If the source is a report, retain the reporting verb. If a recovered artifact is independently inspected, say which property was inspected. Do not let a summary erase that distinction. Two documents repeating the same report are not two independent tests. Conversely, a discrepancy between a report and an artifact should remain visible even when a plausible explanation exists. Record the competing explanations and the observation needed to resolve them. ## Make success invalidate at the correct boundary A receipt belongs to the artifact and conditions tested. Changing an input adapter may leave a pure mathematical result intact while invalidating an integration claim. Changing the kernel may invalidate both. Preserve enough dependency information to know which claims need rerunning. Include an explicit “not established” field. For a pixel-evidence kernel, unestablished conditions may include PDF decoding, drawing scale, domain semantics or physical connectivity. This is not a disclaimer added after success; it is part of the claim’s definition. ## Worked example A synthetic receipt records that a connected-component kernel passed all fixture masks. A new PDF decoder is then added. The kernel receipt can remain valid for the same kernel bytes and fixture inputs, but it does not certify the new decoder. If the decoder drops a layer, the end-to-end result can be wrong while every kernel assertion still passes. The appropriate response is an adapter test and representative document cases, not more repetitions of the unchanged kernel tests. ## Exercise Write three claim-ledger rows for a requirement, a reported outcome and an independently reproduced result. Use the proposed fivefold improvement, a reported Ralph run and the reproduced WITNESS self-tests as distinct evidence classes. Name the missing observation for a broader reliability claim. Draw the dependency between decoder, kernel and outcome interpretation. Define which receipts become stale when each component changes. ## Model answer The fivefold value is a target, not an achieved improvement. The Ralph result remains a user report unless the relevant artifacts or run are inspected. The WITNESS result is a reproduced bounded test outcome. A decoder change requires adapter and end-to-end checks; a kernel change requires kernel-dependent checks as well. No number of kernel assertions establishes authorization or the semantic correctness of an engineering decision. ## Rubric 3 points: No requirement or report is upgraded to reproduced evidence. 3 points: Every row specifies an artifact/condition boundary and a limitation. 2 points: Receipt invalidation follows component dependencies. 2 points: The proposed improvement target remains explicitly unachieved in this evidence. ## References S002, S003, S004, S005, S050, S051, S052, S013, S015, S018, S053, S022, S014, S016, S008, S054, S055, S056, S057 # Write a bounded task contract from a product goal Translate a broad application goal into a contract that agents can execute without silently expanding scope. ## Objectives Separate user-visible outcomes, shared foundations and independent work. Write acceptance criteria with negative and preservation cases. Make file ownership and completion evidence part of the task definition. ## Evidence boundary and originating problem Kelpie begins as a specification for converting Markdown and Mermaid into editable previews and PDFs. Its coordination request assigns shared foundations to one agent, followed by parallel agents that must not modify the same files. A later revision replaces coarse phases with smaller canonical Ralph PRDs, feature groupings and stronger pass/fail criteria. The assistant reports producing task packages, but those packages were not reopened and repository-tested in this research. The teaching value is the explicit change in unit of control: from a large phase to a small, verifiable task with dependencies and ownership. It is evidence of design intent, not proof that the requested three-agent execution ran. ## Define the user-visible result and the invariants A contract starts with an outcome that a reviewer can observe. “Implement export” is incomplete. Specify the accepted input, expected editable or rendered output, error behavior, and what must remain unchanged. Distinguish a usable artifact from an intermediate build result. A successful compiler invocation does not establish that a presentation is editable or that a PDF preserves layout. List invariants such as existing navigation, document identity, source provenance and unsupported-input handling. Preservation requirements should be testable at the same level as the new feature. Otherwise the agent can satisfy the new output while damaging unrelated behavior. ## Put dependency barriers before parallelism Shared data structures, interface contracts and foundational files must be stable enough for independent work. Assign an owner to each mutable file or interface. An agent may propose a change outside its boundary, but the contract should say how that proposal is reviewed and integrated. Disjoint filenames alone are insufficient if two files encode incompatible versions of the same interface. Use a small integration check after the foundation stage. Parallel work begins only when the shared contract is available and its examples pass. This prevents every worker from independently guessing the shape of the system. ## Make completion rejectable and evidence-bearing A story should specify positive cases, negative cases, preservation checks and the artifact to retain as evidence. Keep tasks small enough that failure can be localized. “Build the entire application with excellent UX” cannot support a precise retry. “Render a supported diagram while preserving existing document selection and rejecting malformed input without losing source text” can. Record what a worker changed, what it checked and which conditions remain untested. Avoid a checklist that merely repeats the task wording. Each line must identify an observation or comparison that could fail. ## Worked example For a synthetic Kelpie-style export task, one worker owns parsing and a second owns preview rendering. Both depend on a shared document schema containing source text, blocks and stable block identifiers. If the renderer changes identifiers locally to fix selection, it has crossed an interface boundary even without editing the parser’s files. The correct handoff is a schema-change proposal and updated fixtures, followed by rechecking both consumers. ## Exercise Write two parallel task contracts and one foundation contract for the synthetic export task. Assign file/interface ownership and define the barrier before parallel work. Specify a supported input, malformed input and preservation case for each task. Define how an interface change is proposed and accepted. List completion evidence without claiming a deployed product. ## Model answer The foundation contract establishes the shared schema and stable identifiers with fixtures. Parsing and rendering contracts consume that version and own disjoint implementation surfaces. Any schema change returns to the shared owner and invalidates affected consumer checks. Valid evidence includes retained test inputs and outputs, interface examples and visual comparisons; deployment remains a separate task and claim. ## Rubric 3 points: Shared interfaces have an owner, version and pre-parallel check. 3 points: Each task includes positive, negative and preservation observations. 2 points: Interface changes trigger explicit reconciliation rather than silent local divergence. 2 points: Completion evidence is distinguished from deployment evidence. ## References S058, S001, S006, S007 # Build section-wise document review with a human boundary Use a section-wise review pipeline to make document evaluation inspectable without giving the evaluator approval authority. ## Objectives Separate heading validation from content judgment. Preserve section coverage and source locators through a review loop. Audit the difference between an architecture drawing and its documented operational boundary. ## Public teaching scope and evidence boundary This public teaching scenario models section-wise document review. It is an illustrative design, not a claim about a named organization’s implementation. Separate heading checks, section extraction, prompt construction, captured responses and human review. Each boundary can fail independently. A precise process diagram does not prove that operational controls exist or that a live execution succeeded. The reviewer remains accountable for acceptance. The model prepares evidence and recommendations; it does not grant formal approval. ## Make section coverage a first-class output Start with a validation template that names required headings and content criteria separately. A missing heading is usually a structural observation. An inadequate explanation within a present section is a semantic judgment. Mixing them into one score makes it difficult to tell whether the model reviewed the intended material. For each expected section, record whether it was found, how boundaries were determined, the extracted span and the criteria applied. Preserve empty and ambiguous cases rather than skipping them. The completion condition is that every expected section has an explicit state, not that some model calls returned useful-looking text. ## Control the review loop and the human decision Construct each prompt from the relevant criteria and section text, and capture the response against that exact input version. Present recommendations with their supporting passages. The author then decides whether to revise. If the document changes, rerun affected criteria and preserve the previous result as superseded evidence. A model recommendation is not formal approval. The reference’s human review boundary matters: triage prepares material for accountable review. Do not make the interface imply that a green result grants architectural acceptance, especially when the criteria are incomplete or the source extraction is uncertain. ## Verify the operational claims independently Build an inventory with one row per component or control: shown in diagram, described as current, described as planned, observed in a run, or unresolved. If prose says a control is absent while a diagram shows it, retain the contradiction. Do not infer deployment from visual completeness. Temporary document retention, output handling and failed-run cleanup need their own tests. A review tool can produce useful feedback while mishandling sensitive inputs or leaving partial outputs behind. These are distinct acceptance boundaries and require distinct evidence. ## Worked example A synthetic design document has headings for identity, storage and recovery. The recovery heading exists but contains only “handled by the platform.” The heading check passes; the recovery-content criterion fails because no recovery behavior or responsibility is specified. A missing storage appendix produces an incomplete-coverage state. Neither case should be summarized as “the document passed because all headings were present.” ## Exercise Design a section-coverage manifest and a human review screen for the fixture. Define structural and semantic checks independently. Record the missing appendix without silently treating it as empty evidence. Specify which results become stale after the recovery section changes. Identify two controls that require runtime evidence beyond the HLD. ## Model answer The manifest should mark recovery as structurally present but semantically insufficient, storage coverage as incomplete, and identity according to its actual content. Revising recovery invalidates its content result and any overall readiness summary, but need not invalidate an unchanged identity section if dependencies are explicit. Retention cleanup and access isolation are examples of controls requiring operational tests. Formal approval stays with the designated human process. ## Rubric 3 points: Structural presence and semantic adequacy have separate outcomes. 2 points: All expected sections receive an explicit coverage state. 3 points: Revision invalidation and human approval boundaries are defined. 2 points: The retained reference is not misattributed as a personally verified deployment. ## References S020, S021, S022, S023, S026 # Carry evidence through work-winning research and drafting Design a work-winning assistant around corpus readiness, bounded retrieval and reviewable generation rather than a single drafting prompt. ## Objectives Explain why findability and metadata precede reliable drafting assistance. Compare heterogeneous portfolio components without assigning them one readiness state. Evaluate vendors or approaches with a common task-and-evidence rubric. ## Public teaching scope and evidence boundary This instructional portfolio separates retrieval, taxonomy, categorization, metadata and indices from drafting. A folder-scoped assistant, a static dashboard and a proposed bid assistant solve different parts of the task. Compare them through explicit evidence and human review rather than replacing every component with an autonomous agent. The public exercise does not disclose an organization’s internal tools, funding decisions or pilot status. Its proposed evaluation supplies a study design, not measured bid-win improvement or savings. ## Scope the corpus before expanding the answer space The folder-scoped bot pattern uses collections of relatively uniform items, such as brochures or capability sheets. This reduces some ambiguity about expected content, but does not establish that every item is approved, current or suitable for a particular bid. Add authority, applicability and ownership checks rather than treating location as sufficient approval. When retrieval also grounds generation, retain the item identities and passages used in the draft. A draft should make it possible to inspect whether the cited capability actually supports the requested claim and whether conditions or exclusions were lost during rewriting. ## Keep portfolio readiness states separate A static dashboard can be ready for a bounded use while automated refresh remains planned. A proof of concept can demonstrate a path without establishing production readiness. A vendor interview can inform comparison without constituting a pilot result. Record these distinctions per component and per claim. Version omissions are also ambiguous. If a platform appears in one portfolio slide and not another, the omission alone does not prove cancellation. Ask for a change record or responsible decision. This prevents document comparison from turning into invented organizational history. ## Use a common evaluation case and explicit review cost Compare candidate approaches on the same representative tasks and source corpus. Score evidence retrieval, correct applicability, draft support, failure behavior, access requirements and human correction effort. Keep vendor demonstrations separate from independently run cases. A common rubric is useful only when the evidence behind each score is comparable. Make operational dependencies visible: account access, sharing constraints, source maintenance and refresh frequency. A model that drafts well for one evaluator may still be unsuitable for the intended audience or unable to access the approved corpus. Do not hide these failures inside a general quality average. ## Worked example In a synthetic comparison, Tool A writes a polished capability paragraph but cites an unapproved draft; Tool B retrieves an approved sheet and produces awkward prose; Tool C cannot access the target folder. The correct evaluation does not rank A first merely for fluency. Authority is a gate, access is a prerequisite, and editing effort can be measured for B after the evidence condition is met. These are invented comparison results used to demonstrate the rubric. ## Exercise Create a common rubric and a readiness register for the three tools. Separate non-negotiable gates from graded quality dimensions. Record the evidence needed to score each dimension. Include a refresh or stale-source case. Explain why a testimonial or interview cannot fill a missing pilot result. ## Model answer Authority and permitted access should be gates. A fails the source-authority case, C fails the access prerequisite, and B is eligible for a bounded drafting trial with editing effort recorded. A stale-source test should require either an updated approved item or an explicit limitation. Interview statements can be retained as vendor claims, but cannot be scored as independently observed performance. ## Rubric 2 points: Readiness is recorded per component rather than as a single portfolio label. 3 points: Authority and access are gates distinct from prose quality. 3 points: All comparative scores use stated observations on common tasks. 2 points: Testimonials, plans and pilot results remain separate. ## References S020, S021, S022, S023, S026 # Preserve negative evidence in chained research Prevent a research chain from turning prospective techniques into claims of existing use. ## Objectives Carry factual constraints across extraction, categorization, visualization and synthesis. Preserve negative and missing evidence through summaries. Design an evaluation proposal without presenting its targets as achieved benefits. ## Evidence boundary and originating problem The client-segmentation research describes Python extraction, categorical prompting, a network or dashboard representation, CSV-informed synthesis and source checks. The retained prompt chain explicitly instructs the writer not to claim that proposed techniques are already used by the organization. That is an important factual-control instruction; its presence does not prove successful enforcement in every output. The implementation proposal includes comparisons with earlier campaigns and named outcome measures, but no measured lift or causal attribution was recovered. The instruction here is therefore about controlling a research-to-proposal chain, not reproducing a demonstrated business outcome. ## Carry a factual-control record beside the content Create a compact record of what is established, proposed, contradicted and unknown. Each stage must consume this record together with the relevant source spans. Do not rely on a warning in an early prompt surviving several summaries. A downstream writer that sees only a network diagram may mistake a proposed relationship for an observed one. For each extracted statement, preserve its modality: “could,” “planned,” “reported,” “observed” and “not established” change the claim. If a categorizer removes these terms, the later synthesis must recover them from structured status fields rather than guess from fluent prose. ## Treat visual and tabular representations as lossy projections A category table or graph makes patterns easier to compare but usually omits context. Retain links from each node, row or aggregate to the evidence and its limitations. A connection on a diagram is not automatically a causal relation, and a count of mentions is not a measure of adoption. When a stage groups or compresses material, test whether known negative evidence survives. Include a source that explicitly says a technique is not currently used. The output should preserve that boundary even if other sources describe its possible benefits. ## Keep evaluation plans in the future tense logically, even without dates An evaluation design should identify a baseline, comparison condition, outcome measure, unit of analysis and confounders. None of these establishes an achieved improvement. A grammatically past-tense bullet inside a future-goals section is not sufficient evidence that a result occurred. Separate the proposed test from its eventual observations. The acceptance gate is a retained measurement record, not a polished implementation roadmap. Where causal attribution is not possible, report association or observed change with the relevant limitations instead of crediting the intervention automatically. ## Worked example A synthetic source set contains “Technique X is proposed,” “Team Y does not currently use X,” and “X improved engagement in an unrelated published case.” A careless summary becomes “Team Y improved engagement using X.” The correct synthesis says X is a candidate for evaluation, that current use by Y is not established and is explicitly denied in the fixture, and that the external case is only contextual evidence. The proposed pilot still needs its own baseline and measurements. ## Exercise Write an extraction table, intermediate graph description and final proposal paragraph for the fixture. Assign each statement a status and source locator. Specify how the negative statement survives the intermediate representation. Write a bounded evaluation question without claiming a result. Add a test that detects the false adoption claim. ## Model answer The proposal should retain “not currently used” as a constraint and describe any benefit as a hypothesis to test. The external case cannot be transferred to Team Y. A useful automated check flags a final statement asserting current use or achieved benefit when the status record contains only proposed or negative evidence; human review still checks paraphrases that evade literal matching. ## Rubric 3 points: The negative adoption statement survives every stage. 3 points: External contextual evidence is not transferred into a local outcome claim. 2 points: The evaluation question includes a baseline and observable outcome. 2 points: The failure test checks semantics, not only prohibited words. ## References S032, S031 # Model recorded media as temporal, revisable assertions Preserve the identity, temporal meaning and correction path of knowledge derived from recorded media. ## Objectives Distinguish source-event order, capture/analysis order and recording position. Separate transcript segments, semantic statements and governed concepts. Recover from enrichment failures without destroying source evidence. ## Evidence boundary and originating problem The audio design is anchored by a user correction: after-the-fact analysis of deliberately selected recorded media is the default; live capture is an additional mode. The architecture then distinguishes audio chunks, transcript segments, statements, concept candidates and assertions. It specifies that failed transcription preserve audio, failed extraction preserve the transcript, and failed graph writing preserve pending assertions. These are source-supported requirements and architecture decisions, not a demonstrated complete application. The lesson applies their boundaries to a small reproducible ingestion exercise rather than implying that every proposed capability exists. ## Preserve more than one temporal coordinate A statement’s event order is not its analysis order. Retain source identity and recording-relative position so that a claim can return to the passage that supports it. For the undated exercise, use source event T1, analysis event T3 and position P120 rather than calendar values. Reanalyzing the same recording at T4 must not make its content a new event at T4. Playback, direct-file ingestion and live capture need distinct adapters but compatible domain records. Seeking backward or replaying a segment should not duplicate the underlying statement simply because the capture path produced another chunk. ## Keep extraction layers typed and rebuildable A transcript segment records what the transcription layer produced and where it belongs. A semantic statement records an interpretation of that text. A concept candidate proposes a reusable entity or category. A governed concept requires an acceptance process. Do not promote each extracted noun into the accepted ontology. Bind derived records to source and transformation versions. If transcription changes, identify which statements and assertions depend on the altered span. Preserve the earlier derivative as superseded where needed for audit, while rebuilding the current projection. Never overwrite the only source copy with the latest interpretation. ## Separate model interpretation from deterministic validation Adapters should translate vendor-specific output into canonical objects before semantic processing. Validate required identities, locators, relation types and schema constraints deterministically. Semantic review then asks whether the proposed assertion is actually supported. A structurally valid graph edge can still be false. The proposed architecture treats RDF, SKOS/OWL exports and human-readable notes as projections of canonical objects. A projection should preserve evidence and human edits according to explicit rules. Round-trip checks should test the fields that matter, not merely whether an export file can be parsed. ## Worked example A synthetic recording contains “The preferred material is copper” at P120 and “For the wet zone, use stainless steel instead” at P180. An extracted universal assertion “preferred material = copper” loses the condition. The corrected representation keeps both statements, their positions and the zone condition. Replaying P120 creates another processing attempt, not a second independent source supporting copper. A transcript correction at P180 marks dependent assertions for review. ## Exercise Design canonical records and failure recovery for the two-statement fixture. Represent source event, analysis attempt and recording position separately. Define a stable key that prevents replay duplication. Show candidate-to-governed-concept promotion as a separate step. Inject a graph-write failure and specify the retained retryable artifact. ## Model answer A sound design uses source identity plus span/position and transformation version for traceability, with attempt IDs separate from semantic identity. Both conditional statements remain accessible. Graph-write failure leaves validated pending assertions and their evidence intact; retry does not retranscribe or invent a new source. Promotion of “wet zone” or a material concept requires review rather than automatic acceptance. ## Rubric 3 points: Event, processing attempt and source position are distinct. 3 points: Replay and correction behavior preserve identity and dependency tracking. 2 points: Graph-write failure retains source-linked pending work. 2 points: Schema validity and semantic support remain separate checks. ## References S019, S017 # Govern concepts, labels and graph relationships Specify concept identity and relationship meaning before allowing a visualizer or language model to edit a knowledge graph. ## Objectives Distinguish labels, concepts, assertions and evidence. Define what each relationship means for the intended task. Make human challenge and revision part of the concept lifecycle. ## Evidence boundary and originating problem The organic taxonomy-visualizer request explicitly connects a detailed tool specification with Codex implementation, but completion was not verified. Related semantic-extraction requirements name SKOS/OWL edges and a heavy-model/light-harness division. Published knowledge-engineering arguments emphasize purpose, evidence, identity, revision and human challenge. The source-grounded lesson is not that selecting an ontology vocabulary solves meaning. The engineering task is to define which distinctions the application must preserve and how a contested assertion can be corrected without destroying its provenance. ## Define identity independently of presentation A concept can have preferred and alternative labels without becoming several concepts. Conversely, identical labels may refer to different concepts. Assign stable identifiers and record scope notes. The visualizer should let a reviewer inspect this distinction rather than merge nodes solely because their labels match. State what a relation asserts. “Broader than” is not interchangeable with “related to,” “part of” or “used by.” Use only relation types whose intended semantics are defined for the task. If a model proposes an edge outside that vocabulary, retain it as a candidate needing review rather than silently coercing it into the nearest allowed type. ## Validate structure without pretending to validate meaning Deterministic checks can reject missing identifiers, broken references, disallowed relation types and certain cycles. They cannot establish that a broader-term claim is semantically right. Make structural results visible alongside the evidence a reviewer needs for the meaning decision. Test the visualizer with misleading but structurally valid examples. A neat tree can hide a wrong parent; a dense graph can make an unsupported relation look established. The interface should expose edge status and source, not encode confidence only through color or position. ## Treat governance as a reversible decision process A concept proposal should include purpose, evidence, expected consumers and the effect of acceptance. Review decisions need reasons and a route for challenge. A rejected candidate can still be retained for analysis or reconsideration, but must not appear as an accepted concept in downstream exports. Version the accepted vocabulary and dependent mappings. When a concept splits or a relationship is withdrawn, identify the classifications and generated views that need reassessment. Do not rewrite every old statement as if it had originally used the new meaning. ## Worked example A synthetic vocabulary contains BANK-FINANCE and BANK-RIVER, both with the label “bank.” A model proposes merging them because the strings match, then adds “river bank broader than loan.” Structural validation might accept the identifiers and edge type. Semantic review rejects the merge and relationship because the scope notes and source contexts differ. The visualizer must make those contexts accessible at the point of review. ## Exercise Specify the review interaction for the ambiguous-label fixture. Define stable concept IDs, labels and scope notes. List structural checks and separate semantic review questions. Show accepted, proposed, rejected and withdrawn states. Describe downstream invalidation after a concept split. ## Model answer The answer retains two concepts with shared lexical form. It flags the proposed merge as a semantic decision, exposes the source contexts and rejects the unsupported hierarchy. A later split should produce a mapping review queue rather than retroactively rewriting all source assertions. A graph that parses successfully is not sufficient evidence that the ontology is useful or correct. ## Rubric 3 points: Concept identity is independent of label equality. 3 points: Structural and semantic checks are separately specified. 2 points: Proposal, rejection and withdrawal remain distinguishable in exports. 2 points: Revision effects on dependent mappings are explicit. ## References S059, S060, S061, S062, S063, S023, S024, S025, S026 # Maintain human and machine views of the same knowledge Design human-readable and machine-readable views as controlled projections with different reasoning tasks. ## Objectives Explain the distinct roles of source records, derived metadata and review interfaces. Identify where writeback changes authoritative state. Prevent a polished diagram or projection from overstating operational capability. ## Public teaching scope and evidence boundary The published knowledge-engineering essays distinguish the reasoning tasks served by machine and human representations. Apply that distinction to an illustrative local suite: one component reads files, another proposes enrichment, a third exports analysis, and a fourth diagnoses workspace conditions. Several named components do not establish autonomous multi-agent behavior. In this teaching design, a local analysis export and a metadata writeback have deliberately different effects. Local output does not automatically upload itself; a queued mutation still needs authority, conflict handling and an observed result. ## Map each representation to its task and authority A machine projection may support joins, validation and automated selection. A human view may support comparison, explanation and challenge. Neither should silently become the original source. Define the canonical record and the provenance needed to reconstruct each projection. For every output, state whether it is a diagnostic, a candidate interpretation, an accepted fact or a write to a connected system. A local spreadsheet for review and a metadata update on a shared item have different consequences even if both contain the same generated keywords. ## Make writeback a separate contract Before writeback, identify the target item, fields, source versions and acceptance state. Preserve the ability to inspect the proposed change. A generated summary should not overwrite user-authored text merely because both occupy a field called description. Test stale-source and partial-write cases. If the target changes after analysis, the old proposal may need reconciliation. If only some fields are written, the receipt should describe the actual state rather than mark the entire operation complete. A queue can persist work without proving that an external effect happened exactly once. ## Audit the picture against the runnable boundary Compare a design diagram with the implementation inventory in the exercise. Mark each control current, planned, reported or observed. Do not copy an aspirational drawing into a readiness claim. For any proposed deployment, verify sign-in, role controls, secret handling, monitoring and service responsibilities as separate requirements. Naming a control does not establish implementation; each needs inspected evidence. ## Worked example A synthetic enrichment job proposes keywords for a shared document while an analysis job exports excerpts to a local workbook. The first has an external write boundary; the second has an output-handling and review boundary. If the document changes before the keyword update, the proposal is stale. If the workbook exists, that does not prove its contents were accepted or uploaded. The two jobs need different receipts even when they share extraction code. ## Exercise Create a representation-and-authority map for the two jobs. Mark source, candidate, human view and external write surfaces. Define a stale-version gate for metadata writeback. Specify receipts for partial write and local export. Reconcile one aspirational control with an explicitly absent baseline control. ## Model answer The map should keep original content authoritative, generated metadata proposed until accepted, and local exports separate from connected-system updates. A stale source blocks or queues reconciliation before writeback. Receipts name actual fields updated or files produced and retain failures. An absent control remains absent until implementation evidence is supplied, regardless of its appearance in a diagram. ## Rubric 3 points: Every representation has a purpose and explicit authority status. 3 points: Writeback and local export use different acceptance boundaries. 2 points: Stale and partial outcomes have inspectable receipts. 2 points: Reported modules and aspirational controls are not promoted into verified deployment. ## References S020, S021, S022, S023, S026 # Decompose Ralph work into independently rejectable stories Make a Ralph story small enough to execute, reject and retry without losing the product-level objective. ## Objectives Separate a story’s behavioral contract from the loop runner. Use persistent completion records without treating them as proof. Bound open-ended improvement by novelty, dependencies and complete user-facing outcomes. ## Evidence boundary and originating problem The Ralph work insists on compatibility with canonical loop structure, then asks for more granular PRDs and binary acceptance gates. A separate proposal lets an agent choose useful improvements while consulting progress.txt, avoiding assumed future dependencies and delivering complete user-facing features. The proposed large iteration count is not evidence that those iterations ran. Reported poor results from tightly specified attempts are important counterevidence: a detailed PRD does not guarantee execution quality. Story design, tool compatibility, verification and product judgment remain separate parts of the system. ## Choose the smallest complete behavior, not the smallest edit A useful story crosses enough of the product to create an observable outcome. Splitting a feature into “add button,” “add handler” and “add storage” can produce three passing fragments and no usable feature. Prefer a thin vertical slice with an explicit user action, state change, output and failure behavior. Keep the slice small enough that a failed observation identifies a bounded problem. If a story simultaneously replaces storage, navigation and rendering, a failure is difficult to localize and rollback is costly. State which dependencies already exist. A future component cannot silently become a prerequisite invented by the worker. ## Use completion memory as an index to evidence A progress record helps the loop avoid repeating work, but a sentence saying “done” is not the acceptance artifact. Link each completed story to the relevant version, tests and output. If a later edit invalidates a dependency, the completion state must be revisited. Novelty is also semantic. Renaming the same feature or implementing another superficial variant should not satisfy a requirement to add new value. Compare candidate work with completed outcomes and known unresolved needs, not only with task titles. ## Separate hard gates from comparative improvement claims A binary gate should identify an input, check and rejection condition. Compilation, a defined security check and a bounded performance test can be separate gates. Comparative claims need a real baseline and the same measurement procedure. “Faster” without a baseline is not a pass/fail property. Open-ended selection still needs stop conditions. The agent may propose means, but cannot redefine the requested outcome or relax correctness to win a performance score. If the measurement is invalid or the next feature depends on unapproved scope, record the blocker instead of producing a nominally completed iteration. ## Worked example A synthetic app needs an exportable task list. Story A adds an export menu item with no file output. Story B implements a complete minimal CSV export, including a cancelled dialog and an empty-list case. Story C rewrites storage in anticipation of a future sync system. B is the strongest first vertical slice under the stated goal. A is incomplete; C assumes a dependency outside the present contract. A progress entry for B should point to the exported fixture and checks, not merely mark the menu work finished. ## Exercise Rewrite the three candidate stories into a bounded execution plan. Choose the first complete slice and state its existing dependencies. Define positive, empty and cancelled-action outcomes. Specify the evidence linked from progress.txt. Add a rule for revisiting completion after a shared export component changes. ## Model answer The plan should select a minimal complete export, defer speculative sync work, and reject a menu-only completion. A cancelled dialog must create no unintended output; an empty list must have a defined valid result. Completion points to the tested version and retained CSV fixture. Changing a shared serializer invalidates dependent export checks and requires rerunning them before the record is considered current. ## Rubric 3 points: The first story is a complete user-visible slice. 2 points: No unbuilt future dependency is assumed. 3 points: Completion memory links to version-bound observations. 2 points: Novelty and regression invalidation are explicit. ## References S064, S065, S002, S003, S004, S005, S066, S067, S068, S069, S008, S070, S071, S072, S073, S074, S075, S076, S077, S078, S079, S080, S058, S001, S006, S007 # Coordinate parallel agents through interfaces and file ownership Coordinate specialized agents through typed handoffs, ownership and end-to-end acceptance rather than role names alone. ## Objectives Define the artifact passed between each specialist. Prevent parallel work from creating incompatible representations. Verify a finished document at content, branding and layout boundaries. ## Evidence boundary and originating problem The document-production concept specifies five roles: read source documents; transform summaries into prose, lists or data; prepare a branded document; fill it; and visually verify branding and layout. It requests sophisticated Word and PowerPoint manipulation support. Kelpie separately establishes shared foundations before parallel work with disjoint file ownership. These are explicit workflow designs, not verified five-agent executions. The important innovation is the combination of specialization with an independent visual check. The lesson makes the interfaces between those roles testable so that specialization does not merely multiply plausible but incompatible outputs. ## Define handoff artifacts before assigning workers The reader should produce source-linked findings, not an untraceable narrative. The synthesis stage should preserve claim-to-source mappings while selecting the structure and wording. The template stage should expose a layout contract: placeholders, style names, allowed element types and constraints. The filling stage should produce an editable artifact plus a placement report. The verifier should inspect the actual rendered result against the intended content and design. Every handoff needs a schema or example, a completeness rule and a rejection path. A downstream worker must be able to say that required evidence or a placeholder is missing without inventing it. ## Parallelize only independent surfaces Source reading and template preparation may proceed concurrently if their shared assumptions are explicit. Final filling depends on both. Two agents editing the same presentation file are not independent merely because one is “content” and the other “design.” Assign a single owner to the mutable artifact or use separately reviewable patches with a controlled integration step. The clean-room visual-resource request adds another boundary: reference acquisition must honor the allowed source set and exclusions. A visually useful asset does not become permissible merely because it helps the output. Keep provenance and usage constraints with the asset handoff. ## Make verification independent of the producer’s claim Check content fidelity, source support, editability, branding and rendered layout separately. The filling agent’s report that all placeholders were populated does not prove that text is visible, charts are legible or the right source data was used. Inspect representative rendered pages or slides and test the native artifact where relevant. When verification fails, return a precise defect with location, expected condition and observed condition. Do not ask the producer to “make it better” without preserving the failed comparison. The next run should demonstrate that the identified issue is fixed without damaging accepted regions. ## Worked example A synthetic pipeline produces a three-slide briefing. The source reader finds two supported claims and one unresolved estimate. The synthesis stage removes the estimate’s uncertainty label to fit a headline; the filler places a chart over the footnote. A visual-only verifier catches the overlap but may miss the unsupported certainty. A source-only verifier catches the claim change but may miss the hidden note. Both checks are needed, with distinct defect records. ## Exercise Specify the five handoff artifacts and route the two defects back to the correct stage. Define the source-linked finding record and the layout contract. Assign ownership of the final editable file. Write one semantic and one visual acceptance check. Describe the regression check after each correction. ## Model answer The uncertainty defect returns to synthesis with the original evidence status; the overlap returns to filling or layout integration with a render location. The final file has one integration owner. Correcting the headline requires rechecking source fidelity and layout because text length changes; moving the chart requires checking that labels and source notes remain readable. A single generic quality check cannot substitute for these boundaries. ## Rubric 3 points: All five stages have explicit, inspectable handoff artifacts. 2 points: Mutable artifact ownership and integration are unambiguous. 3 points: Semantic and visual defects have separate tests and routes. 2 points: Asset provenance and user exclusions survive the handoff. ## References S033, S034, S035, S036, S037, S001, S081, S082, S083, S058, S006, S007 # Debug the boundary between model, harness and tools Diagnose tool-using failures at the model, protocol, harness and runner boundaries instead of treating a successful connection as a working agent. ## Objectives Separate transport success from tool-protocol correctness. Use small reproducible cases to localize an integration failure. Preserve an authorized architecture while testing repairs within its boundaries. ## Evidence boundary and originating problem The Magellan/Ralph record reports a successful proxy configuration connecting a model to a tool-using coding environment, followed by tool failures and crashes on larger jobs. The user asks for repair of the same architecture rather than substitution. Earlier questions explicitly separate runner, harness and model access. The exact diagnosis proposed by the assistant was not independently verified, and no successful repair is established in the recovered record. This is a failure-analysis lesson, not a recipe for reproducing a particular current product configuration. The observed distinction is durable: a text response from an endpoint does not prove that the surrounding tool protocol works. ## Draw the boundary map before changing components Identify the runner that selects work, the harness that maintains interaction state, the model adapter, the transport or proxy, and the tool executor. Record the input and output expected at each boundary. A runner may be correct while the adapter mishandles tool calls; a model may return valid structured intent while the executor fails to report the result in the expected conversation state. Separate observations from hypotheses. “The request returned text” is an observation. “The proxy cannot support tool use” is a hypothesis until the failing boundary is isolated. Avoid replacing the user’s chosen architecture simply because an alternative is easier to demonstrate. ## Build a minimal protocol ladder Test ordinary text first, then a single harmless tool call, then tool-result continuation, then multiple calls or a longer job. Retain sanitized request/response shapes and error categories without recording credentials. The progression helps distinguish authentication, transport, schema, correlation, state and resource failures. Use a deterministic harmless tool in the fixture, such as returning a fixed arithmetic result or reading a permitted test file. A failed protocol test should not create external effects. Match each tool result to the initiating call and verify that the harness resumes the intended state rather than treating the result as an unrelated user message. ## Repair one boundary and rerun the same failure Change one adapter assumption at a time where practical. The repair claim must identify the original failing case, the change and the new observation. A larger successful job cannot prove that a specific edge case is fixed if its path was not exercised. After a narrow repair, increase workload gradually and retain the first point of divergence. Resource pressure, context truncation and state serialization can fail at different scales. Do not infer a precise root cause from the symptom “crash on larger jobs” without supporting traces. ## Worked example A synthetic adapter returns a valid call request C1. The executor runs the harmless tool but sends its result under C2. Plain-text tests pass; tool continuation fails. A schema validator that checks only required fields may also pass. The decisive test checks correlation and state transition. This fixture illustrates one possible failure class; it is not a claim about the actual cause of the reported Magellan integration failure. ## Exercise Write a protocol test ladder and a diagnosis table for the C1/C2 fixture. List the expected observation at each boundary. Separate the known mismatch from untested root-cause claims. Specify a repair and the exact regression case. State which larger-job properties remain unverified. ## Model answer The repair preserves the initiating call identity and tests that the result resumes the correct pending call. The same C1 fixture must pass after the change. Authentication, arbitrary tool behavior, parallel calls and long-job stability remain separate tests. The answer should not claim that this invented mismatch explains the original reported failure. ## Rubric 3 points: Transport, protocol, harness and runner observations are distinct. 2 points: The minimal fixture is harmless and reproducible. 3 points: Repair evidence reuses the original failure condition. 2 points: Unverified diagnoses and long-job behavior remain qualified. ## References S084, S085, S086, S087, S088, S089, S090, S091, S092, S093, S094, S095, S096, S072, S073, S074, S050, S051, S052, S064, S065, S002, S003, S004, S005, S066, S067, S068, S069, S008, S070, S071, S075, S076, S077, S078, S079, S080 # Preserve working context without laundering memory into truth Construct working context from versioned sources without allowing an agent’s memory to become self-validating evidence. ## Objectives Separate immutable source records, derived assertions and task-specific context. Preserve contradictions and human edits across revisions. Define invalidation when a summary or source changes. ## Evidence boundary and originating problem The Boris wiki specification preserves source versions, provenance, contradictions and human edits. The audio architecture similarly treats derived layers as rebuildable projections. Earlier findability work adds the practical constraint that representations have maintenance costs. These are design sources for a memory system whose usefulness depends on remaining accountable to its inputs. The existing Boris interface is user-reported; the durable wiki behavior is specified rather than runtime-verified. This lesson therefore develops a controlled fixture and acceptance tests for the proposed mechanism. ## Separate storage from context selection A source record should retain the material and its identity. An assertion records an interpretation with evidence and status. A context packet selects a subset for a particular task. The fact that an assertion is stored does not mean it is applicable to every future task, and the fact that it was selected does not increase its authority. Record the selection rule and omissions. A short context packet can legitimately omit irrelevant material, but it must not conceal a known contradiction that changes the answer. Selection is a reasoning operation with its own failure modes, not merely a token-budget convenience. ## Protect user-authored material from generated reconciliation When a generated projection and human prose coexist, define ownership at field or region level. A rebuild should not overwrite the user’s explanation simply because a newer extraction exists. Preserve the conflict and request review where the representations cannot be merged safely. Distinguish a correction to a source from a new interpretation of an unchanged source. Both may invalidate derived assertions, but they are different events. A later summary of an earlier assistant statement is still derivative evidence; repeated summarization must not turn it into independent confirmation. ## Make retrieval and memory freshness observable A context packet should identify source versions, assertion states and unresolved conflicts. If a governing source changes, mark dependent conclusions stale or rebuild them under an explicit rule. The next task must not unknowingly reuse a conclusion whose support no longer matches the current source. Test missing, inaccessible and contradictory sources. A failed retrieval is not evidence that the source contains no relevant fact. A memory system should be able to answer “not established from the accessible current evidence” without filling the gap from its own prior prose. ## Worked example A synthetic wiki stores a supported assertion that export format is CSV. A later accepted source adds JSON, while a draft proposes PDF. A generated summary says “all formats supported.” The context packet must distinguish the accepted revision from the draft and reject the summary’s overreach. If a user has written a note explaining a special CSV workflow, rebuilding the generated format list should preserve that note unless a specific authorized edit is requested. ## Exercise Design the records and invalidation behavior for the export-format fixture. Identify canonical source, assertion, projection and context-packet records. Mark the accepted revision and draft separately. Define how a human note survives a rebuild. Add a test for a derivative summary falsely appearing as independent evidence. ## Model answer The accepted current assertion is CSV plus JSON, with the draft PDF proposal retained but excluded from the accepted capability list. The summary is corrected or marked unsupported. Human-authored notes remain in their owned region. A provenance graph should show that the summary descends from existing evidence, so counting it as a second independent source is rejected. ## Rubric 3 points: Storage, interpretation and task selection have separate records. 3 points: Drafts and contradictions cannot silently become accepted memory. 2 points: Human-owned content survives generated rebuilds. 2 points: Staleness and derivative-source dependence are testable. ## References S012, S013, S097, S038, S029, S039, S040, S030, S041, S042, S043, S017 # Evaluate feature value without optimizing a proxy blindly Use typed judgments and deterministic calculations while keeping uncertainty and value assumptions visible. ## Objectives Distinguish directly measured properties from inferred feature value. Validate elicited variables before calculation. Use sensitivity analysis to reveal fragile rankings. ## Evidence boundary and originating problem The feature-value discussion explicitly rejects speculative proxies disguised as objective data. It asks for model-assisted interpretation of variables with ranges and assessment guidance, followed by deterministic Rust calculation. Related open-ended improvement requests require useful complete features, novelty checks and ordinary quality gates. The generated Rust artifacts were not independently compiled or validated in this research. The lesson is the division of labor: a model may help interpret a situation, but a deterministic function should compute from typed inputs whose origin and uncertainty remain visible. A reproducible score is not objective value inherent in the feature. ## Type the variable and its evidentiary status For every input, state its meaning, unit or scale, allowed range, source and whether it is measured, estimated or judged. “Effort = 3” is meaningless without the scale. “Confidence = 0.8” is not automatically a calibrated probability. The elicitation prompt should ask for the supporting observation and an uncertainty explanation, not only a number. Reject missing and out-of-range inputs instead of silently substituting convenient defaults. A model’s fluent rationale cannot make incompatible units comparable. If a variable cannot be responsibly estimated, preserve that state and consider whether the decision needs more evidence. ## Keep the calculation transparent and versioned The calculation should be a pure transformation of validated inputs under an explicitly chosen formula. Retain the formula version and the input record so another person can reproduce the result. Test boundary values, invalid values and ties independently of the model. Weights and formulas express a decision policy. They may be useful, but they are not discovered facts about value. State which trade-offs the policy encodes and which harms or benefits it omits. Do not label a subjective weighted score as an objective measurement merely because the arithmetic is exact. ## Test whether the ranking survives plausible uncertainty Vary uncertain inputs within justified ranges and inspect whether the preferred feature changes. A ranking that flips under a small plausible change should be presented as fragile. Consider gathering evidence, narrowing the decision or selecting a reversible experiment rather than increasing numerical precision. Keep hard constraints outside the utility score when they must not be traded away. Correctness, permission and preservation requirements should not be offset by a high estimated benefit. A feature that violates a gate is ineligible under the contract, however attractive its score. ## Worked example For an explicitly synthetic exercise, define utility = benefit × confidence − effort. Benefit and effort are dimensionless scores from 0 to 10; confidence is a judged support weight from 0 to 1, not a calibrated probability. These scales and the formula apply only to this fixture. Feature A has benefit 8, confidence 0.8 and effort 3: utility 3.4. Feature B has benefit 6, confidence 0.9 and effort 2: also 3.4. Reducing A’s confidence to 0.7 produces 2.6. The arithmetic is deterministic; the benefit and confidence values are judgments. The tie and sensitivity show why a score alone cannot establish the objectively best feature. ## Exercise Implement or calculate the fixture and write a decision note that preserves its uncertainty. Declare variable types, ranges and origin labels. Test invalid confidence, missing effort and the exact tie. Perform one sensitivity change and state its consequence. Add a non-tradeable preservation gate. ## Model answer The mathematical outputs are 3.4, 3.4 and 2.6 for the specified cases. An implementation should use exact decimal or rational arithmetic, or a declared numerical tolerance when testing binary floating-point results, so representation error does not break the mathematical tie. Confidence outside the declared interval must be rejected, not clipped without disclosure. The decision note should say the initial ranking is tied under this chosen policy and sensitive to A’s uncertain confidence. A failed preservation gate excludes the feature regardless of utility. The formula is a teaching fixture, not the source’s validated feature-value law. ## Rubric 3 points: Measured, estimated and judged inputs are visibly distinguished. 3 points: Calculations and invalid-input tests are correct and reproducible. 2 points: The tie and sensitivity are interpreted without false precision. 2 points: Hard gates cannot be compensated by utility. ## References S098, S099, S100, S101, S102, S075, S076, S077, S078, S079, S080 # Evaluate prose and experimental agent language without flattening judgment Evaluate prose loops and constrained language experiments with a mixture of mechanical checks and explicit human judgment. ## Objectives Separate formal constraints from literary quality. Design a revision rubric that preserves the intended aesthetic. Avoid treating an experimental language concept as an implemented compiler. ## Evidence boundary and originating problem The prose-loop requests prioritize writing and knowledge work, explore portable harnesses, and ask for evaluation tied to Calvino’s Six Memos and a project style guide. Categories and Likert judgments are allowed alongside more objective measures. No validated universal metric of literary merit or completed novel loop is established. The agent-language concept adds a different kind of constraint: variable-foot three-line stanzas, ellipses as the sole punctuation, and a requested Rust implementation. The detailed compiler architecture in the response is an assistant proposal; no implemented compiler was verified. These two strands teach why formal validity and aesthetic success require different evidence. ## Separate hard form constraints from interpretive criteria A mechanical checker can count lines, identify disallowed punctuation or verify a declared structural pattern. It cannot establish that a passage is vivid, light, precise or artistically successful merely by satisfying those counts. Define which criteria are hard constraints and which require a reader’s reasoned assessment. A style guide should include positive and negative examples with explanations. A number without an anchor is difficult to compare across revisions. If a rubric uses a scale, define what each level means and ask reviewers to cite textual evidence for their judgment. ## Preserve the purpose of revision across the loop A revision can improve one measurable feature while damaging voice, meaning or narrative function. Retain the original goal and the relevant source text alongside the score. Compare versions on the same passage and task. Do not let the loop optimize only the easiest-to-count signal. Record rejected changes and why they were rejected. A loop that repeatedly reintroduces an unwanted simplification has not learned from its progress record. Separate a reviewer’s preference from a factual or formal defect, and preserve disagreements rather than averaging them into a misleading certainty. ## Test experimental syntax without inventing semantics For the language concept, write a grammar fixture and distinguish parsing from execution. A three-line stanza using only allowed punctuation can be syntactically acceptable under the exercise while lacking a defined meaning. Semantic rules, error handling and runtime behavior need their own specification and tests. If the project is exploratory, say so. A generated grammar sketch, a claimed code artifact and a compiled, tested implementation are different outcomes. The course can teach how to turn a concept into rejectable tests without asserting that the original compiler exists. ## Worked example A synthetic prose revision reduces sentence length and repeats fewer words, but removes a deliberate rhythm and changes the narrator’s stance. A mechanical score improves while the intended voice deteriorates. A separate synthetic three-line stanza obeys the punctuation constraint but contains an undefined operation. The first requires evidence-based aesthetic review; the second passes a form check but fails semantic definition. Neither can be resolved by a single global “quality” number. ## Exercise Design a two-part review protocol for the prose revision and the constrained-language fixture. Define two mechanical checks and two anchored interpretive criteria. Show how a reviewer cites evidence for a voice judgment. Separate grammar acceptance from semantic/runtime acceptance. Write a stop condition for a loop that improves a proxy while damaging the purpose. ## Model answer A strong protocol reports mechanical results separately from reader judgments, with quoted short spans or precise locations supporting the latter. It can reject the prose revision despite better count-based metrics when the agreed voice criterion is violated. The stanza remains only syntactically accepted until its operations have defined semantics. The loop should pause or return to the stated goal when repeated proxy improvements fail the substantive rubric. ## Rubric 3 points: Mechanical constraints and aesthetic judgments have different evidence paths. 3 points: Interpretive ratings use anchored criteria and textual support. 2 points: Grammar, semantics and implementation claims are kept separate. 2 points: The loop has a purpose-preserving rejection or stop rule. ## References S103, S104, S105, S106, S107, S108, S109, S110, S111, S112, S113, S114, S115, S116, S117, S118 # Make actions recoverable and authorization revision-bound Recover interrupted work by separating authorization, attempted actions, observed effects and revision-specific acceptance. ## Objectives Model contract progress separately from external action state. Reconcile an uncertain side effect before retrying. Invalidate authorization and acceptance evidence when their relevant inputs change. ## Evidence boundary: a specified recovery architecture Boris supplies a concrete design for durable contracts and action ledgers. Its fast native interface and existing integrations are user-reported; the revised integration specification supplies state machines, postcondition checks and acceptance criteria. It does not supply a reproduced runtime demonstration of those mechanisms. Preserve this distinction when using Boris as an architectural example: an explicit recovery rule makes implementation testable, but does not prove recovery works. The important separation is between a phase being complete and an external action having occurred. A phase can remain unverified even when an external mutation succeeded. Conversely, a model can announce completion when no mutation occurred. Neither the conversation nor the last process exit code should become the authority for resolving that difference. ## Define two state machines and their binding records The contract phase moves through pending, ready, running and verifying before passing. The action ledger records planned, approved, dispatched, observed and reconciled states. Link the two through stable action identities and the specific criterion each action supports. Store the intended target, payload, relevant artifact revision, approval scope, execution attempt and observed result. These fields are an instructional operationalization of the specification, not an audited Boris database schema. Authorization and acceptance answer different questions. Authorization permits a particular action under particular conditions. Acceptance says that a defined postcondition has been demonstrated. Bind approval to the meaningful revision of the action: changing the recipient, output contents, target or risk invalidates permission to execute the altered action, even if its human-readable task title remains unchanged. ## Recover by observing before repeating After interruption, reconstruct state from the durable ledger and checkpoint. A dispatched action with no recorded result is unknown, not failed. Query the destination or reopen the affected artifact using the narrowest available read operation. Match a durable external receipt, a destination identifier or a resulting artifact hash to the intended action. If the result matches, reconcile it and continue verification. If absence is established, a permitted retry may be appropriate. If observation cannot decide, keep the action unresolved and ask for the missing authority or information. Idempotency keys help only where the destination honors them. A locally stored action identifier cannot force an unrelated application to deduplicate a click or send. Recovery design must therefore document which effects are safely repeatable, which have reliable lookup, and which need human reconciliation. ## Failure diagnosis: stale evidence and false completion A common defect is accepting a test receipt for an artifact that has changed. Boris explicitly requires relevant edits to invalidate acceptance evidence. Trace each receipt to the artifact revision, test configuration and criterion it examined; derive invalidation from dependencies rather than clearing every result indiscriminately. A spelling edit may leave an unrelated recovery test valid, while an action-dispatch change invalidates that path's evidence. Another defect is treating a successful click as proof of a saved document. The broker must reobserve the application, reopen or inspect the result, and link the observation to the action. A stale screen reference should fail before mutation. User takeover or cancellation must release input state while preserving enough ledger information to reconcile anything already dispatched. ## Worked example Synthetic fixture: Boris prepares report revision R4 and receives permission to save it into a designated workspace. The broker dispatches Save, then the process stops before recording a result. On restart, the destination contains a report whose hash matches R4. Meanwhile, the local editor contains R5, which adds a new section. Reconcile the R4 save as observed success; do not save R5 under R4's approval merely because the filename is unchanged. The phase can pass only the R4 criterion it actually satisfies. If the task now requires R5, evaluate the amendment, obtain any newly required permission and produce fresh evidence for the affected criteria. ## Exercise Specify the restart decision for the fixture and for an alternative in which the destination cannot be inspected. Write the contract state and action state immediately before and after restart. Identify the fields that bind permission to R4 and the observations that reconcile the save. List which acceptance criteria must be rerun for R5. Describe the user-visible state when the result cannot be established. ## Model answer The action starts dispatched/unknown and becomes observed then reconciled after the destination hash matches R4. The contract remains verifying until its required postconditions are satisfied. R5 is a changed payload, so R4's receipt cannot certify it and any authorization dependent on the contents must be reconsidered. Rerun criteria affected by the new section and its output generation, preserving unrelated valid evidence. If the destination is inaccessible, report an uncertain save and pause dependent mutations; neither retry automatically nor label the save failed. The answer should identify what observation would resolve uncertainty, rather than using a timeout as evidence of absence. ## Rubric 3 points: Separates contract completion from action state and explicitly represents unknown outcomes. 3 points: Reconciles the R4 save from destination evidence before any retry. 2 points: Binds approval and acceptance to relevant revisions without treating them as interchangeable. 2 points: Preserves the specification-versus-runtime boundary and handles the uninspectable outcome safely. ## References S012, S013, S097 # Design portable model state and reproducible execution plans Use Cassette's cartridge, revision and execution-plan concepts to distinguish portable possession from demonstrated capability. ## Objectives Separate authoritative model state from temporary execution material. Define what an execution receipt must identify to be reproducible. Reject portability and completion claims that outrun hardware-specific evidence. ## Evidence boundary: an engine report and a falsifiable blueprint Cassette's materials report completed engine milestones and a 42-of-42 arm64 macOS test result. Those tests were not independently rerun in the recovered evidence. The blueprint separately identifies live hardware, storage, model and service-quality acceptance as unproven. Therefore, teach the cartridge architecture as a documented design with reported implementation, not as demonstrated frontier-scale capability or a shipped cross-platform product. Cassette's central question is stronger than whether model files can be downloaded. It asks whether a user possesses an identifiable, usable capability, can relate it to its source and transformations, and can establish what works on a particular machine. Storage integrity and useful execution belong to the same lifecycle but require different checks. ## Keep authority outside the working set The external cartridge is intended to remain authoritative for model bytes, revisions and provenance. Bounded working material occupies machine memory during execution. Treat that memory as a disposable execution view, not the only record of what the user owns. A restart should recover identity and lineage from durable state without trusting a model's description of its previous session. A useful teaching schema separates source identity, acquired bytes, representation, child revision and execution plan. A transformed or trained descendant retains its parent relationship and transformation evidence. A plan says how a specified revision is to run under resource constraints; it is not itself a new source model. Keeping these objects distinct prevents a quantized derivative, a hardware schedule and an original checkpoint from silently sharing one ambiguous name. ## Make execution plans reproducible within declared conditions Record the model revision, representation, engine revision, hardware characteristics, storage path and configuration that materially affect a run. Add workload identity, measurement method, limits and observed result. A receipt must make it possible to distinguish an actual execution from a planned one. Mark unexecuted matrix cells accordingly; an unsupported cell is not a failure, but it cannot inherit a pass from a nearby configuration. Portability should preserve object identity, permissions and provenance even where runtime implementations differ. Moving a cartridge does not establish that another machine meets memory, latency or quality requirements. Separate tests for reading and verifying the cartridge, loading an execution plan, completing a defined workload and meeting the product's user-facing service floor. Each can pass or fail independently. ## Failure diagnosis: a complete story with incomplete artifacts Cassette's recovered corpus contains a concrete contradiction: completion prose and a coverage workbook say the package is complete while the index lists missing deliverables and the QA report gives a do-not-deliver result. This establishes an inconsistent artifact set, not the condition of an unseen corrected package. The engineering remedy is to derive the release manifest, coverage view and completion statement from one verified artifact snapshot. Another failure is substituting a small test suite for the hardware acceptance matrix. Unit tests may validate identity and scheduling logic while never exercising sustained reads from an external drive. Diagnose by asking which required behavior each test could falsify, then identify the remaining live measurements. More green tests cannot cover an experiment that was never run. ## Worked example Synthetic fixture: cartridge C7 contains source revision M2 and derived representation Q3. Engine E5 passes integrity and small-workload checks on machine A. A transfer to machine B verifies the same cartridge hashes, but the target workload exceeds B's memory limit. The correct report is that cartridge identity and transfer succeeded, while execution acceptance failed under the tested B configuration. Do not rename Q3 as M2 or claim that portability failed in every sense. A revised execution plan may be tested, but its receipt must name the changed plan and retain the unsuccessful result. These identifiers and outcomes are invented teaching data. ## Exercise Design a minimal receipt and release decision for the two-machine fixture. Draw the lineage from source M2 to representation Q3 and distinguish plan from model identity. Specify the receipt fields needed to reproduce the A and B checks. Separate passed, failed and unexecuted acceptance rows. Write a release statement that cannot contradict the artifact manifest. ## Model answer The lineage identifies Q3 as a derivative of M2 with its transformation record. Each receipt binds C7, Q3, E5, machine characteristics, execution plan, workload, configuration and measured outcome. A passes only its inspected checks; B passes identity transfer and fails the stated memory-bounded workload; all other configurations remain unexecuted. The release statement may promise verified carriage and the tested A behavior, but cannot promise B execution or general frontier performance. Generate its coverage from the same snapshot as the manifest and withhold any deliverable whose required artifacts or acceptance rows are missing. An alternate B plan is a new experiment, not a reason to overwrite the failure. ## Rubric 3 points: Preserves source, derivative, cartridge and execution-plan identities. 3 points: Receipts name enough conditions to reproduce and limit each conclusion. 2 points: Keeps failure and unexecuted states distinct from successful transfer. 2 points: Resolves completion accounting from one artifact snapshot without inventing a corrected release. ## References S009, S119, S120, S121, S122, S010, S011 # Translate accessibility commitments into acceptance tests Translate Cassette's ordinary-user accessibility requirement into observable setup, fallback and independence tests. ## Objectives Turn an accessibility commitment into a testable product contract. Separate open-engine operation from optional commercial assistance. Measure setup success and recovery without assuming expert users. ## Evidence boundary: accessibility constrains the design Cassette's requirements explicitly reject technically sophisticated Mac users as the defining first audience when that framing undermines the project's social commitment. The product remit includes multiple operating systems, agent-led setup, a knowledge workspace, an IDE and an accessible web route. These are requirements and roadmap intentions. They are not proof that the interfaces, platforms or support economics have been delivered. The blueprint also says essential open-engine operation must not depend on commercial services, with deterministic interface options alongside an agentic operator. This gives the lesson an unusually concrete architectural constraint: a polished commercial assistant may improve the experience, but must not become a hidden gate that makes the open engine unusable. ## Turn audience assumptions into acceptance conditions Begin with tasks an ordinary user must complete: recognize a suitable machine, obtain an allowed model, understand storage requirements, start useful work and recover from a comprehensible failure. Define the conditions under which the task is expected to succeed. Avoid inventing a universal speed or completion threshold; choose and justify thresholds for the intended users and supported configurations before observing the trial. Record whether users complete the task, what help they need, which terms confuse them and whether they can explain the consequential choice they just made. A scripted developer demo demonstrates a path through the interface, not ordinary-user accessibility. A setup flow that works only because the evaluator knows an undocumented command fails its own audience contract even if the engine is technically functional. ## Test commercial independence and usable fallback Trace every essential operation to its actual dependencies. Installation, model discovery, integrity verification, local execution and export may each have different requirements. For every dependency, ask whether it is unavoidable, optional, or merely convenient. A disabled paid account must not silently block a documented open operation. If online access is genuinely required for acquisition, disclose that bounded requirement without pretending all operation is offline. The deterministic fallback should expose the same important domain objects and effects as the agentic interface. It need not reproduce every conversational convenience. Test that the user can identify the selected model, inspect the intended action, see a useful failure message and recover without relying on the assistant's prose. A fallback that requires editing undocumented internal state is a development escape hatch, not a satisfactory user path. ## Failure diagnosis: audience substitution and false platform coverage A team can satisfy a technical checklist while replacing the intended audience with users who already understand storage, model formats and terminal tooling. Diagnose this by comparing recruitment criteria, support assumptions and the task contract. Do not let successful expert setup stand in for evidence about ordinary users. Likewise, a shared object model does not establish equal runtime performance on every platform. Cassette's design keeps permissions, identity and provenance portable while allowing implementation differences. Report platform-specific support honestly. An accessible web route may widen access, but its costs, data flow and dependence on remote services need their own acceptance tests. Pricing tiers, social benefit and customer retention remain claims to test, rather than conclusions obtained from an attractive architecture. ## Worked example Synthetic fixture: a participant can launch a local model with guided assistance. When the commercial assistant is unavailable, the TUI displays model filenames but no indication of which representation fits the machine. A developer succeeds by recognizing a naming convention. The engine-independence test may show that execution is technically possible; the ordinary-user fallback test still fails. A targeted repair adds a readable capability check and a deterministic selection path tied to the cartridge's metadata. Re-test with participants who have not seen the earlier flow, including recovery from an unsuitable model choice. This fixture proposes a test, not an observed Cassette usability result. ## Exercise Write an acceptance plan for essential open operation and ordinary-user recovery. Define one participant task and its supported hardware assumptions. Identify optional commercial dependencies and test their removal. Specify how the user recognizes an unsuitable model and recovers. State what evidence would justify a platform claim and what remains unproven. ## Model answer A strong plan asks a nonexpert participant to select and run a compatible model on a declared configuration, with observable completion criteria and recorded assistance. Disable the commercial assistant while leaving the permitted acquisition and local tools available; verify that no essential action secretly requires a paid credential. The fallback should explain incompatibility, preserve the user's work and offer a deterministic compatible choice without undocumented commands. A successful trial supports only that task, participant population and platform configuration. Cross-platform operation, service quality, retention and social impact require separate evidence. Report expert intervention as assistance, never as independent user success. ## Rubric 3 points: Operationalizes the intended ordinary-user audience rather than substituting experts. 3 points: Tests essential open operation with commercial assistance unavailable. 2 points: Provides observable incompatibility and recovery criteria. 2 points: Bounds platform, usability and commercial claims to the actual evidence. ## References S009, S119, S120, S121, S122, S010, S011 # Turn inbox capture into governed knowledge Preserve the meaning of attention actions while turning selected inbox material into source-backed, revisable knowledge. ## Objectives Distinguish archive, delete, defer, pin and propose in an attention workflow. Promote a message into a claim without converting its contents into authority. Handle synchronization and human edits before publishing derived knowledge. ## Evidence boundary: a product specification atop Boris Inbox for Boris is a requested Python TUI product specification, with documentation explicitly preceding implementation mechanics. The delivery report describes extensive requirements and acceptance scenarios, but the full artifact was not audited in the recovered evidence and no implemented client was verified. Its reported behavior includes Done as archive, pin protection for bulk actions, and visible recovery for interrupted sending and synchronization conflicts. Boris separately specifies versioned sources, supporting spans, contradictions, permissions and protected human edits in its knowledge workspace. Combining those ideas into an inbox-to-knowledge workflow is an instructional synthesis. It must not be presented as a shipped Inbox feature or a demonstrated integration. ## Separate attention state from source and claim state An inbox action answers what should happen to an item in the user's attention system. A knowledge operation answers whether some content should become a supported assertion. Archiving a message does not endorse its statements, delete its source or resolve its commitments. Deferring it does not change when the statement applies. A pin protects visibility; it does not make the content authoritative. Represent the provider message identity, source revision, attention state and derived-claim identity separately. If a message says a proposal is approved, retain who said so, which proposal revision they meant and any limits in the message. The body may contain quoted third-party instructions or stale facts. Extraction is a proposal to represent the content, not permission to execute instructions found within it. ## Promote selected material through a governed update Capture only what the task needs, with a link or stable reference to the source and the supporting passage. Preserve uncertainty, conflicting replies and the difference between an assertion being made and the assertion being established. Proposed wiki edits should state which existing claim they supplement, revise or contradict. A human correction must not disappear during the next regeneration. Boris's design propagates source permissions into derivatives and distinguishes publication, observation and applicability. In an undated teaching fixture, these become three separate clock fields rather than one overloaded timestamp. A recently received forward can contain an old policy whose applicability has ended. Retrieval should expose that distinction and prevent a visually fresh inbox item from becoming misleadingly current knowledge. ## Failure diagnosis: destructive simplification and duplicate effects The first defect is interpreting Done as deletion or treating every item in a bulk operation alike. Preserve operation semantics and pin protection in the selection and confirmation path. Test the resulting provider state, not merely disappearance from the visible list. A filtered view can hide an item without changing its underlying state at all. The second defect is resending after a timeout or accepting a stale local snapshot over a remote change. Use Boris-style action reconciliation to establish whether sending occurred before retrying. For knowledge publication, compare the proposed edit's base revision with the current page. A conflict should produce a reviewable merge proposal; it should not silently replace the user's revised wording with an older machine extraction. ## Worked example Synthetic fixture: message M1 says brochure B2 may be reused internally. A reply M2 narrows that permission to one named team. The inbox bundle contains M1, M2 and a pinned reminder. The user marks the unpinned bundle Done and asks for a knowledge note. Correct behavior archives the selected messages, preserves the pinned reminder, and proposes a source-linked note that includes M2's limitation. It does not delete the messages, share B2, or infer broad reuse rights from M1 alone. If a human has already edited the note's scope, the proposal must compare against that edit before applying any change. ## Exercise Specify the state transitions and proposed knowledge note for the bundle. List attention, provider and knowledge states separately for each item. Identify the support and limitation that must survive extraction. Describe the conflict check against a human-edited note. Add an interrupted-send case and define its recovery observation. ## Model answer M1 and M2 become archived in the provider after the action is verified; the pinned reminder remains visible and protected. The knowledge proposal says that internal reuse is limited to the named team, citing both messages and preserving B2's identity. The note remains proposed until the update's permissions and revision checks pass. Compare its base revision with the current human-edited note and surface any scope conflict instead of overwriting it. An uncertain send requires lookup of the intended outbound message or provider receipt before any retry. No attention-state transition establishes the truth of the extracted assertion or authorizes sharing the brochure. ## Rubric 3 points: Keeps attention, provider and knowledge states distinct, including pin protection. 3 points: Carries source identity, conflicting evidence and restricted scope into the note. 2 points: Preserves human edits through revision-aware conflict handling. 2 points: Treats the combined workflow as proposed and reconciles uncertain sends before retrying. ## References S123, S012, S013, S097 # Separate personal and distributable product responsibilities Use Glassport's parallel versions to define shared behavior, explicit variants and separate release evidence. ## Objectives Create a change-propagation contract for personal and distributable variants. Distinguish packaging artifacts from install, launch and release evidence. Prevent personal-environment assumptions from becoming silent distribution dependencies. ## Evidence boundary: parallel responsibilities, limited release proof Glassport's direct requirement is that a personally optimized version and an App Store-targeted version receive aligned improvements. The inspected material establishes icon delivery and the dual-development requirement. Installation, Desktop launching and repository-instruction verification remain outstanding in that source. An icon file is evidence of a packaging asset; it cannot establish an installed application or release readiness. The source does not specify a verified branching strategy, distribution pipeline or current store status. The procedures in this lesson are design exercises derived from the parallel-version problem. Do not retrofit them into claims about Glassport's actual repository or imply that the two variants have already passed equivalent tests. ## Define alignment at the behavior boundary Aligned improvements need not mean identical implementation. Identify the shared user behavior, the variant-specific constraint and the evidence required in each version. A personal integration may legitimately depend on a local service or customized workflow that cannot be assumed for another user. Keep that dependency explicit rather than hiding it inside a shared feature's success criterion. For every change, create a propagation record: shared intent, affected components, personal acceptance, distribution acceptance, known divergence and decision owner. A change is accounted for only when both versions have a disposition: implemented and verified, implemented but unverified, deliberately different with a reason, or blocked. A merged commit in one branch should never silently close the other version's work. ## Separate code, packaging and real launch checks A packaging asset must exist in the required form and be referenced by the intended application build. That check differs from installing the build, launching it through the intended route and using the changed behavior. Test each boundary with its own receipt tied to the build and configuration. A filesystem icon can be correct while the installed application still displays a cached or unrelated asset. For a distributable variant, add clean-environment tests appropriate to its declared audience. Inspect assumptions about local paths, configuration, credentials and permissions. This is a proposed readiness checklist, not an assertion about particular store rules. The objective is to find which parts of personal success rely on state that a new installation will not possess. ## Failure diagnosis: drift disguised as shared progress One failure mode is testing the personal build and then describing the distributable version as improved because they share a feature name. Compare behavior and relevant source revisions, not labels. Another is treating a cherry-picked patch as proof that configuration and packaging changes travelled with it. Cross-version regression tests should exercise the user-visible effect and report the exact build under test. A third defect is overwriting deliberate personal behavior in pursuit of superficial uniformity. Require a reason for divergence, then test the common contract and the permitted difference separately. If one variant is blocked, report that variant and blocker plainly. Alignment is maintained through explicit decisions and evidence, not by claiming both versions are complete whenever either one progresses. ## Worked example Synthetic fixture: a launch-panel improvement uses a custom local configuration in the personal version. The distribution build compiles, but on a clean profile its launch action has no configured destination. Updated icon files exist for both builds. The correct disposition is personal behavior verified under its custom configuration; distribution behavior blocked by missing first-run configuration; packaging assets present but installed icon behavior still untested. A repair could provide a supported configuration flow or a clear unavailable state. The fixture does not establish Glassport's actual launch mechanism, repository layout or store requirements. ## Exercise Create a propagation record and release note for this change. State the shared user outcome and each variant's preconditions. Identify code, packaging, installation and launch checks separately. Choose a deliberate distribution behavior when no destination is configured. Write the narrowest truthful completion statement for each version. ## Model answer The shared outcome is a discoverable launch action with a correct, understandable result. The personal variant may retain its explicit custom destination; the distribution variant needs a first-run configuration path or a clear non-destructive unavailable state. Separate receipts cover asset inclusion, installation, launch and the changed behavior, each naming the build. The note says the personal behavior is verified only in its tested configuration, distribution support remains blocked, and icons are delivered but not yet checked in an installed application. Both variants stay on the propagation record until the distribution disposition is implemented and tested or an authorized scope change resolves it. ## Rubric 3 points: Defines alignment through shared behavior and explicit variant-specific assumptions. 3 points: Separates asset delivery, installation, launch and feature verification. 2 points: Keeps a blocked variant open without erasing legitimate personal customization. 2 points: Avoids inventing repository, store or release facts absent from the Glassport sources. ## References S124 # Build narrow, checkable certificates from WITNESS Turn a disputed drawing interpretation into a bounded image predicate, a checkable certificate and an explicit unresolved remainder. ## Objectives Distinguish stored, visible and interpreted drawing content. Apply lower and upper masks to a three-valued connectivity decision. State exactly what a reproduced WITNESS kernel result does and does not establish. ## Evidence boundary: a reproduced kernel, not a complete drawing reader WITNESS proposes a source-linked experiment around a disputed drawing claim. A surviving import-free kernel supplies mathematical and image-evidence operations, and its self-tests were independently reproduced: 8,523 checks, 1,458 interval cases and examination of 8,192 intermediate masks. These are real reproduced outcomes within a small synthetic test domain. They do not prove a complete PDF interpreter, reliable engineering judgment or measured improvement in a model's drawing comprehension. The useful mechanism is to request evidence that distinguishes alternatives. Rather than asking several agents to reconsider the same ambiguous crop, identify an observable predicate, the source region it concerns and the result that would reject an interpretation. Leave the semantic interpretation with an explicitly accountable reasoning step. ## Choose the layer and predicate before computing WITNESS separates stored, decoded, executed, visible and interpreted content. A PDF object may exist but be covered; visible text can disagree with a hidden text layer; a repeated drawing occurrence need not represent another physical component. A certificate should name the layer it examines so that a convenient parser representation cannot silently become an engineering conclusion. For image topology, specify the source identity, crop, coordinate transform, endpoints, threshold assumptions and neighborhood rule. Exact rational transforms can preserve coordinate relationships without rounding drift, but they cannot restore detail absent from the image or validate a wrong source crop. The predicate might be foreground connectivity between two pixels. That is deliberately narrower than whether two pipes, wires or structural elements are functionally connected. ## Use lower and upper masks without forcing an answer A lower mask contains foreground admitted with certainty under the declared uncertainty model; an upper mask contains every pixel that could be foreground. Every admissible image lies between them. If the endpoints connect in the lower mask, the path also exists in every admissible intermediate mask. If the endpoints do not connect in the upper mask, no admissible intermediate mask connects them. Otherwise, return undetermined. A positive certificate can carry a path that an independent checker validates against the appropriate mask and adjacency rule. A negative certificate must establish the relevant separation under the upper mask. Keep checker inputs tied to the source and assumptions. The certificate is valuable because its narrow assertion can be checked independently of the model's explanation, not because the explanation sounds mathematically confident. ## Failure diagnosis: scope inflation and expensive irrelevant evidence The most consequential error is converting certified ink connectivity into engineering connectivity. A connected stroke may be an annotation, overlap or symbol. Keep the image result separate from hypotheses about the drawing's meaning. Another error is treating undetermined as weakly positive. It is a precise statement that the current evidence admits both outcomes; acquiring a better crop or different source may be justified. WITNESS also selects evidence with shared context costs. Its greedy procedure is not a proof of a globally smallest packet. Evaluate whether a proposed observation distinguishes the live alternatives and include the contextual material needed to interpret it. A highly detailed crop that omits a legend or endpoint can be larger and still less useful than a smaller, well-chosen packet. ## Worked example Synthetic fixture: in a three-pixel row, the left and right endpoint pixels belong to both masks. The middle pixel belongs only to the upper mask. Under horizontal nearest-neighbor adjacency, the lower mask has no connecting path and the upper mask does. The correct outcome is undetermined. If another observation establishes the middle pixel as foreground, a lower-mask path becomes checkable. Even then, the result says that these foreground pixels connect under the stated rule. A separate interpretation must decide whether the mark represents a component connection, annotation or overlap. None of these fixture outcomes is a test of a real drawing. ## Exercise Write the certificate contract and the next evidence request for the three-pixel fixture. Identify the predicate, source binding, masks, endpoints and adjacency rule. Derive the initial result without guessing the middle pixel. Specify the smallest observation that could resolve the topology question. List the engineering interpretations that the certificate cannot settle. ## Model answer The initial certificate contract binds the source revision and crop to endpoint connectivity under the declared horizontal adjacency and lower/upper masks. Its result is undetermined because the lower mask disconnects and the upper mask connects. An observation that classifies the middle pixel within the same uncertainty model can resolve the image predicate: certain foreground yields a positive path; certain background yields upper-mask separation. The certificate does not establish physical, electrical or structural connection, component identity or annotation scope. The next request should target the ambiguous pixel plus sufficient surrounding context, while the semantic question remains explicit and separately reviewed. ## Rubric 3 points: States a source-bound image predicate and its neighborhood and uncertainty assumptions. 3 points: Derives all three-valued outcomes correctly and preserves the initial undetermined state. 2 points: Keeps engineering meaning separate from certified foreground connectivity. 2 points: Reports the reproduced kernel result without implying full PDF or real-drawing validation. ## References S014, S015, S016, S018 # Test visual preservation and asset completeness Use the Ralph UI failure and Infini-Wizard asset concerns to test preservation and visible completeness separately from build success. ## Objectives Define preservation requirements before an interface change. Trace assets from acquisition through runtime use and visible states. Distinguish reported implementation from inspected visual acceptance. ## Evidence boundary: two concrete quality problems The Ralph record includes a reported app-page refactor that damaged the existing interface. The response was a requirement to inspect the interface first, change only the intended area and verify preservation. A browser-integrated specification exists, but the recovered evidence does not demonstrate that the proposed integration worked. Infini-Wizard provides a complementary case: a dependency-ordered 36-task Swift game PRD included build, lint, file-quality and asset requirements. Completion and asset import were user-reported, followed by concern about remaining placeholder art. Neither a repository audit nor a device or release test was independently verified. The teaching point is concrete: imported files and passed implementation tasks can coexist with visibly unfinished output. ## Specify the allowed visual change and the preserved contract Before editing, identify the target region, relevant screens, states, viewport conditions and interaction sequence. Capture a baseline that includes the surrounding interface, not just the element to be changed. State which behavior and appearance must remain stable and which changes are authorized. This turns preservation into an acceptance condition rather than a vague request to make the page better. Compare meaningful states under comparable conditions. Loading, empty, error, populated and selected states can reveal different defects. Control dynamic content or declare narrow comparison masks before reviewing the candidate. A broad exclusion added after a regression appears can hide the failure. Visual differences need interpretation: expected text wrapping or animation is not automatically a defect, while a missing button can be consequential even if the pixel difference is small. ## Trace an asset to its actual rendered use Create a manifest that connects each required asset to its permitted source, local identity, application reference and visible role. Acquisition answers whether the file exists. Integration answers whether the code or asset catalog references it correctly. Runtime coverage answers whether the intended state actually uses it. Visual acceptance answers whether the result is appropriate and complete. Test each boundary rather than treating the first as evidence for all four. For a game, include scene, character state, resolution and fallback behavior where relevant. A populated asset folder can coexist with hard-coded placeholder colors or a fallback image triggered by a misspelled identifier. Detect placeholders through both repository inspection and rendered-state checks; neither alone guarantees that every state has been exercised. ## Failure diagnosis: green gates that omit the user-visible defect A build gate fails only on the conditions it checks. It can pass while a layout is damaged or a placeholder remains. Diagnose the defect by locating the missing acceptance condition rather than simply adding more generic tests. A visual baseline may catch unintended layout movement; an asset-reference check may catch a missing mapping; a device-level interaction test may reveal a state that never appears in a static preview. Keep quality claims revision-bound. If an agent changes shared styling after screenshots are approved, those screenshots no longer establish preservation for the changed screens. Likewise, do not expand a target such as a large usability improvement into an achieved result without a locked baseline and comparable measurement. Ralph's proposed outcome goals remain proposals unless supported by results. ## Worked example Synthetic fixture: a game package contains four final images, but the character selector references only three. The fourth character renders a placeholder through a fallback path. The build passes, and a screenshot of the default character looks complete. A separate settings-panel refactor also moves a navigation control outside the visible area on a narrow viewport. The asset-import criterion passes; runtime asset coverage and visual preservation fail. The repair must bind the fourth image correctly and restore the control, then revisit both the selector's relevant states and the affected viewport. These are constructed failures modeled on the source's quality concerns. ## Exercise Write two independently rejectable acceptance stories for the fixture. Define the asset manifest and the character states to inspect. Define the allowed settings-panel change and the preserved navigation behavior. Identify which existing passing checks remain informative and which are insufficient. Specify the screenshots, interactions and revision bindings needed to close each story. ## Model answer One story requires all four final assets to exist, be referenced and appear in their intended character states, with no unintended placeholder fallback. The other permits the settings-panel change while requiring the navigation control to remain visible and usable at the declared narrow viewport and other supported states. Build success remains useful but cannot close either visual criterion. The default-character screenshot covers only that state. Capture and inspect the fourth character, compare the settings view against its baseline, exercise navigation and tie the receipts to the repaired build. Relevant subsequent asset or layout edits reopen the affected acceptance checks. ## Rubric 3 points: Separates asset acquisition, application references, runtime coverage and visual acceptance. 3 points: Defines a bounded change and explicit preserved behavior across relevant states. 2 points: Explains why passing build and default-state screenshots cannot close the defects. 2 points: Binds evidence to the repaired revision and preserves the sources' reported-outcome limits. ## References S054, S055, S056, S057, S064, S065, S002, S003, S004, S005, S066, S067, S068, S069, S008, S070, S071, S072, S073, S074, S075, S076, S077, S078, S079, S080 # Test the boundary of a software-readiness claim Build a readiness argument whose individual claims can fail independently and whose unverified areas remain visible. ## Objectives Separate test execution, behavioral coverage, interface use and release readiness. Challenge generated tests with distinct failure conditions. Write a bounded software-readiness statement tied to actual evidence. ## Evidence boundary: a method supported by unequal examples The evidence-bearing workflow pattern joins different kinds of material. Ralph contributes a reported UI failure and requested quality gates. Boris contributes specified revision-bound receipts and broker verification. WITNESS contributes a narrowly reproduced kernel result. The published readiness method calls for changed-behavior checks, meaningful assertions, recovery tests, real-interface use and disclosure of unverified areas. These sources support a comparison, not proof of a shared implementation or universal compliance. A readiness claim must therefore say what was checked, how and against which artifact. An assertion that tests exist is weaker than a receipt showing execution. A passing suite is narrower than a claim that the product serves its intended users under realistic conditions. ## Decompose readiness into claims with different failure modes Start from the requested user outcome and list the claims needed to support it. A changed feature behaves correctly; unchanged behavior remains intact; failure recovery preserves state; permissions are respected; the real interface exposes the feature; the delivered package is the tested package. Define a check for each claim that could reject it independently. Do not create an unweighted pile of checks and call its size confidence. A missing recovery test cannot be compensated for by additional formatting tests. Likewise, a maintainability rule such as a function-length limit is a bounded quality condition, not a security proof. The record requests such limits but does not establish the assistant's suggested numeric defaults as universal adopted standards. ## Challenge the test oracle and the execution path Generated tests are work products that require review. A test may copy the implementation's mistaken assumption, assert only that no exception occurred, mock away the changed behavior or never execute the modified branch. Inspect whether its expected result follows from the task contract independently of the code it checks. Introduce a controlled incorrect result in a disposable fixture to see whether the assertion can reject it. Then examine the path from test to product. A direct function call may verify logic while missing an inaccessible button, stale UI state or incorrect authorization boundary. Add a real-interface test where that path matters. Record mocks and exclusions explicitly. Independence here means different ways of being wrong, not simply asking a second agent to restate the same conclusion. ## Failure diagnosis: evidence that survives a relevant change A green receipt can become stale after the underlying code, configuration, dependency or artifact changes. Boris's design makes that relationship explicit. Track the relevant dependencies of each acceptance result and require reruns when those inputs change. A release package assembled from another build cannot inherit a test badge by filename alone. Readiness also has operational dimensions that isolated tests may never exercise: interruption, retry, user takeover, unavailable services and contradictory data. State these gaps rather than disguising them with a broad ready label. Usage telemetry measures activity, not demonstrated productivity or quality. WITNESS's 8,523 checks remain a strong kernel result precisely because the claim stops short of full drawing interpretation and product readiness. ## Worked example Synthetic fixture: a report exporter passes unit tests for correct text and file naming. The tests mock the save dialog. In the application, the export button is disabled after a canceled save, so the user cannot try again. A developer fixes that state transition, then changes the output-path configuration without rerunning export tests. The first build lacks demonstrated cancellation recovery; the second has stale evidence for output placement. Neither should be described as fully verified export functionality. The evidence ledger should preserve the passing text checks while identifying the failed interaction and unverified configuration separately. ## Exercise Write a defensible readiness report and a minimum completion plan for the exporter. Map each readiness claim to its current supporting or missing evidence. Explain which mocked behavior conceals the user-facing defect. Identify the receipt dependencies invalidated by the path change. State the remaining checks and a release decision without broadening the evidence. ## Model answer The report says text generation and naming passed under the tested unit configuration, while cancel-and-retry behavior failed through the real interface. The save-dialog mock concealed the state transition. The recovery fix needs an interface test that cancels, retries and confirms the resulting file. The output-path change invalidates checks whose expected destination depends on that configuration, even if text-only tests remain valid. Withhold a complete exporter-readiness claim until the delivered revision passes those affected checks. Disclose any untested environments or interruption cases. Passing maintainability gates or increased test counts cannot substitute for the missing behavioral evidence. ## Rubric 3 points: Decomposes readiness into independently rejectable claims rather than a single test-count proxy. 3 points: Examines the test oracle, mocks and actual interface path. 2 points: Invalidates only relevant stale evidence and binds release claims to the delivered artifact. 2 points: States unverified areas and preserves the distinction between reported, specified and reproduced evidence. ## References S002, S003, S004, S005, S050, S051, S052, S013, S015, S018, S053, S022 # Bound a service agent through routing and escalation Build a library-service agent whose routing, answer grounding and human handoff have separately testable boundaries. ## Objectives Separate deterministic service routing from grounded question answering. Handle an intent change without carrying stale request fields into a handoff. Test configured escalation rather than assuming that a prompt instruction works. ## Public teaching scope and evidence boundary This public service-routing scenario combines rules-based resource selection, sourced question answering, licensed-content boundaries and escalation. It does not describe a named employer’s configuration or claim a production deployment. The exercise tests two concrete flow problems: multiple user responses must form a coherent structured handoff, and a changed intent must update both routing and stored state. Detecting the new topic does not prove that stale fields have been removed. ## Route the request before generating an answer Classify the current request into an explicit service path such as article, book, standard, market research or general question. Deterministic rules can then select the appropriate resource route and required fields. Reserve generated answers for questions that can be grounded in the approved knowledge base. The visible result should be a usable resource, a sourced answer or an actionable human intake route. Keep confidence and scope visible. If sources do not support an answer, do not manufacture a purchase promise or substitute unsupported general advice for the requested service. Licensed material may require linking or an access route rather than reproducing its contents. Routing determines what the agent may help prepare; it does not itself authorize a transaction or an external message. ## Treat intent changes as state transitions A conversation can move from finding a free article to requesting a paid standard. Re-evaluate the route and the meaning of collected fields when that happens. Retain useful context while clearing or marking fields that no longer apply. A title captured for an article must not silently become the standard identifier in a handoff email. A structured handoff needs defined fields, reliable rendering and review of missing or conflicting values. Broadening triggers can help detect a changed topic, but it does not prove stored state is correct. Test the entire sequence through the rendered payload, including backtracking and corrections. ## Failure diagnosis: an escalation policy with nowhere to go A prompt that says escalate can still fail if the intake URL is a placeholder or the message body loses required fields. Validate destinations and configuration separately from language behavior. A broken route should produce a truthful blocker and a supported alternative if one exists, not a claim that the request reached the team. Do not promote a presentation's finished-solution language over its explicit pending tests. Acceptance needs evidence that a person can start the request, change intent, inspect the handoff and reach the configured endpoint. Add negative cases for unsupported questions, unavailable licensed sources, paid requests and missing identifiers. Treat a simulated handoff and an actual authorized delivery as different results, with different evidence and consequences. ## Worked example Synthetic fixture: the user supplies an article title, then says they actually need a paid standard and gives its identifier. The free-resource route has already stored the article title in a global field. A broad trigger now selects the standards route, but the assembled intake body still contains the article title as the requested standard. Correct behavior detects the changed intent, preserves the old title only as superseded context, and confirms the standard identifier in a reviewable handoff. If the intake destination is still a placeholder, the agent reports that configuration blocker and does not claim submission or purchase. ## Exercise Design the routing decision and end-to-end acceptance case for this conversation. Identify the route and field meanings before and after the intent change. Define the complete handoff payload and its required review checks. Test a missing identifier and an unconfigured destination. State the evidence needed to distinguish prepared, submitted and fulfilled requests. ## Model answer The route changes from article discovery to paid-standard human intake. The article title is removed from the active request field or clearly marked superseded; the new standard identifier, user's request and necessary context populate the handoff. A missing identifier prompts a targeted question. A placeholder destination blocks submission and remains visible as a configuration defect. A prepared payload proves only preparation, an authorized send receipt proves delivery to the intake channel, and a separate service response would be needed to establish fulfillment. The agent cannot promise a purchase. Test the full user interaction and rendered payload rather than accepting trigger matching alone. ## Rubric 3 points: Separates deterministic routing, grounded answers, licensed access and purchase boundaries. 3 points: Revalidates stored fields after intent changes and checks the actual handoff payload. 2 points: Treats placeholders and missing identifiers as explicit blockers. 2 points: Distinguishes retained design, reported fixes, tested delivery and service fulfillment. ## References S020, S021, S022, S023, S026 # Convert a failure record into a reusable tool safely Turn Magellan's refactor experience into a bounded reusable check without converting a success report into a security guarantee. ## Objectives Extract a tool contract from a concrete failure and repair record. Separate repeatable detection from environment-specific remediation. Retest reusable tooling against negative fixtures and the real invocation path. ## Evidence boundary: reusable-tool creation is reported Magellan's record reports that security-refactor logs were turned into a reusable tool. It also reports successful configuration of a model through LiteLLM into Claude Code for a Ralph loop, while another report describes pervasive tool failures and crashes on larger jobs. The combined evidence supports reported customization and an unresolved reliability problem, not a verified stable repair. The exact proposed diagnosis remains a hypothesis. The source also distinguishes present capability from desired autonomy: manual tool building is described, while idle improvement that pauses for the user is an ambition. A requested browser-extension deprecation and hardening plan does not establish that the attack surface was eliminated. Preserve these boundaries when deriving a reusable procedure. ## Extract decisions and checks, not an unedited command transcript A log contains observations, failed hypotheses, commands, contextual assumptions and results. Separate them before making a tool. For each retained step, identify the condition it detects, the evidence it consumes, the output it produces and the state it might change. A command that fixed one repository can be unsafe or irrelevant in another. Preserve the reason for the change and the preconditions under which it applies. Build a read-only diagnostic contract first where possible. It should return findings with locations, evidence and bounded conclusions. Keep remediation as a separate, explicitly scoped operation that checks current state and can be reviewed or reversed where appropriate. Remove sensitive values from reusable fixtures while preserving the structural feature that caused the failure. A log is evidence about an encounter, not a blanket authorization for future actions. ## Turn lessons into falsifiable, portable gates Magellan's gate requests emphasize binary outcomes, cross-language applicability and genuine baselines. Translate those demands into separate rules with explicit inputs and failure conditions. A check for an exposed credential pattern, excessive permissions or an unsafe tool boundary differs from a maintainability check. Avoid labeling a repository secure merely because the chosen checks pass. Use a small fixture family: a known bad case, a corrected case, an irrelevant case and a case the tool cannot interpret. The unknown case should remain unsupported rather than pass silently. Compare behavior across the intended stacks through shared invariants, allowing stack-specific inspection adapters. A performance comparison requires an actual baseline and matched workload; an invented reference value cannot support an improvement claim. ## Failure diagnosis: a successful configuration mistaken for reliable tooling A tool may work when invoked directly but fail through the model, gateway and harness. Preserve the actual invocation path in acceptance. Check argument serialization, returned tool identities, result consumption, cancellation and larger-workload behavior without assuming which layer caused the source's reported failures. A reproducible minimal failure gives a narrower diagnostic target than another broad repair prompt. If a requested change removes an attack path, test that the old path is actually unavailable and that legitimate work still has an approved route. A rename or cosmetic replacement is not evidence of hardening. Any future self-improvement loop must remain bounded by allowed actions, observable outcomes and user interruption; log-derived tooling does not grant the agent permission to expand its own access. ## Worked example Synthetic fixture: a refactor log identifies a local configuration file that accidentally exposes a credential to a tool's output. A proposed reusable checker simply searches for that original filename and deletes matches. This is an unsafe generalization: it confuses one symptom with the exposure mechanism and couples detection to destructive remediation. A better diagnostic inspects the declared output path for credential-bearing fields using sanitized fixtures, reports the relevant flow and refuses unsupported formats. A separate repair proposal removes or masks the field under explicit scope. Then test the result through the actual tool wrapper, because direct function success may miss serialization exposure. ## Exercise Specify a reusable diagnostic and a separate repair acceptance plan. Extract the invariant violated by the fixture instead of retaining its filename-specific shortcut. Define diagnostic inputs, outputs, supported formats and unknown behavior. Create bad, corrected, irrelevant and unsupported fixtures. Specify an actual-wrapper test and a bounded repair postcondition. ## Model answer The invariant is that sensitive credential fields must not reach the tool's declared user-visible output. The diagnostic reads supported configuration and output mappings, reports an evidence-linked exposure finding and never deletes files. Corrected and irrelevant fixtures must not produce the same finding; unsupported formats return an explicit unknown result. The repair is separately authorized and scoped to preventing that field's exposure while preserving legitimate output. Acceptance exercises the real wrapper and serialized response, not only the detector's function. Report the selected tests passed, with remaining unsupported paths, rather than claiming the whole agent is secure or that Magellan's reported reliability issue is resolved. ## Rubric 3 points: Extracts a mechanism-level invariant instead of replaying context-specific log commands. 3 points: Separates read-only diagnosis, authorized repair and unsupported cases. 2 points: Uses discriminating fixtures and tests the actual harness invocation path. 2 points: Avoids security guarantees and preserves Magellan's reported-success and unresolved-failure boundaries. ## References S084, S085, S086, S087, S088, S089, S090, S091, S092, S093, S094, S095, S096, S072, S073, S074, S050, S051, S052 # Build a maintained knowledge-curation supply chain Treat the curation service's answer quality as dependent on a maintained knowledge supply chain, while keeping proposed controls separate from established practice. ## Objectives Trace a chatbot answer back through embedding and curation dependencies. Design maintenance and withdrawal tests for derived knowledge. Avoid attributing an invented operating model to the teaching scenario. ## Public teaching scope and evidence boundary A knowledge assistant depends on a maintained supply of curated resources. The public knowledge-engineering methods treat source purpose, provenance, applicability and revision as part of the system. An embedding pipeline does not remove those responsibilities. This lesson develops a teaching design for collection maintenance. Its owners, metadata fields, review triggers and withdrawal paths are proposed controls to test, not assertions about a named organization’s current service or staffing. ## Trace the answer through its supply dependencies An embedding process can make a resource retrievable without establishing that the resource is accurate, applicable or permitted for the user. Trace an answer through the selected passage, source revision, collection inclusion decision and responsible disciplinary knowledge context. Each boundary can introduce a different error: unsuitable source, incorrect extraction, stale representation, poor retrieval or unsupported generation. Diagnose at the earliest unsupported transition. If an approved source is absent from the collection, tuning the answer prompt will not restore it. If two sources conflict, ranking one higher may conceal the issue rather than resolve it. If the correct passage is retrieved but its qualification is omitted, the problem lies in answer construction. Keep those failures separate so maintenance effort addresses the cause. ## Design maintenance as an explicit proposed contract For a teaching implementation, propose an owner, source-status field, update trigger and withdrawal path for each collection. Label these as design choices to validate with the organization. Define how a changed source affects extracted passages, embeddings, caches and already published derivative material. A search index can be rebuilt; the authoritative source and its revision record should not disappear simply because an embedding is replaced. Include an unresolved state. A resource can be useful yet lack an accountable owner or clear applicability. Decide whether that limits retrieval, adds a warning or blocks a particular high-consequence use. The choice should depend on the task and be reviewable. Do not treat every uncertainty as identical, or disguise a missing governance decision as a model-confidence score. ## Failure diagnosis: better answers from worse knowledge A convincing answer can conceal stale or unauthorized material. Test with a withdrawn resource, a superseding revision, two conflicting disciplinary sources and a user whose access differs from the curator's. Evaluate whether the answer preserves the relevant qualification and whether the system can show why a source was selected. More indexed documents or more chatbot activity cannot establish correct task completion. Maintenance remains necessary when feature work stops. A continuing service needs explicit responsibility for source updates, failures and withdrawal. End the exercise with a proposed maintenance contract and the observations needed to accept it; do not invent institutional success. ## Worked example Synthetic fixture: a curated discipline collection contains guidance G1. Its curator replaces it with G2 because the applicability has narrowed, but the answer index still retrieves a G1 passage and a cached response omits the new limitation. The system's response becomes more fluent after a prompt change while remaining wrong. Trace the defect through source revision, embedding generation and cache identity. A repair must invalidate or refresh the affected derivatives and preserve the limitation in the answer. Removing G1 from a folder alone is insufficient if cached material can still support a current answer. The fixture is a proposed maintenance test, not an observed service incident. ## Exercise Write a proposed curation and withdrawal contract for the fixture. Map source, passage, embedding, cache and answer identities. Assign proposed responsibilities for source decisions and derivative refresh. Define the expected result for a query whose answer changes under G2. State what evidence would establish maintenance success and what the source does not prove. ## Model answer The contract links every derivative to G1 or G2 and its generation, with an explicit invalidation path from the curator's revision decision. The proposed source owner determines applicability; the proposed service owner ensures affected index and cache entries are refreshed or withheld. A query requiring the old broad answer must now receive the narrower supported answer or an explicit unresolved response. Verify that stale G1 material cannot silently support a current result, while retaining provenance for inspection. Successful maintenance requires observed behavior across those paths, not merely a completed upload. These roles and tests remain proposals; this teaching scenario establishes no organization-specific implementation. ## Rubric 3 points: Locates answer failures within distinct source, representation, retrieval and generation stages. 3 points: Defines source-revision propagation and withdrawal checks across derivatives. 2 points: Labels added ownership and maintenance arrangements as proposed rather than established source facts. 2 points: Distinguishes service continuation, feature investment and demonstrated answer quality. ## References S020, S021, S022, S023, S026 # Design organizational collaboration without inventing a product Use the proposed operating model and the collaboration model to design collaboration around engineering delivery without turning an organizational layer into an imaginary application. ## Objectives Distinguish a delivered product, an operating model and a collaboration layer. Specify cross-role handoffs with artifacts, evidence and decisions. Evaluate a collaboration proposal without claiming organizational adoption or workforce effects. ## Public teaching scope and evidence boundary Organizational collaboration can be designed without inventing a separate software product. The object of the exercise is a dependable transition from project work to reusable knowledge. Data, AI, learning and knowledge responsibilities support that transition; a new label does not establish adoption. A strategic proposal establishes an articulated intention. It does not prove that governance was accepted, interfaces worked or projected benefits occurred. Keep engineering delivery as the outcome and evaluate coordination through observable handoffs. ## Model work transitions rather than an invented product surface Map a concrete project journey: understand the commission, form a team, develop the design, assure it, coordinate disciplines, manage risk, deliver and return learning to reusable knowledge. Choose a costly transition and identify the work products that cross it. The collaboration layer should make responsibilities, evidence and decisions clear at that boundary. For each handoff, specify the producer, recipient, artifact, required context, acceptance decision and correction route. A dashboard may help expose these things, but it is not the organizational arrangement itself. A graph may help locate precedent, but it does not decide whether the precedent applies. Avoid inventing a login, central repository or autonomous agent simply because a named collaboration effort sounds like a software product. ## Allocate production, assurance and learning explicitly Distinguish who creates a candidate artifact from who can judge its fitness for the next use. A model may draft a project summary; a discipline specialist may assess technical applicability; a delivery owner may accept a handoff; a knowledge curator may decide what becomes reusable. These are proposed roles for an exercise, not an actual organizational staffing chart. Give disagreement a visible route. If a proposed reusable lesson is disputed, retain the supporting project evidence, applicability limits and unresolved question. Do not force consensus merely to complete a handoff. Closing the delivery task and publishing reusable knowledge can require different approvals and different evidence. The coordination design should prevent one team's completion label from being inherited automatically by another team's acceptance decision. ## Failure diagnosis: communication polish mistaken for operating change A diagram can explain an arrangement beautifully while leaving every decision owner undefined. Diagnose that failure by asking what happens to a real disputed artifact: who receives it, who can reject it, what evidence they inspect and what happens next. If the answer is another abstract capability box, the design has not reached an operational handoff. A second error is inferring productivity or staffing outcomes from an aspirational model. Measure selected transitions through completion quality, rework, waiting and review burden under declared conditions. Activity counts or speculative workforce estimates cannot establish gains. A bounded trial may show that one handoff improved; it does not establish adoption across the organization or validate a long-range scenario. ## Worked example Synthetic fixture: a project team produces a reusable design rationale. A data specialist can extract its entities, an AI assistant can draft a summary, and a learning lead wants to publish it. The discipline reviewer finds that a critical site constraint is missing. A collaboration-layer design returns the candidate with the missing constraint and preserves its draft status; it does not mark the knowledge enterprise-ready merely because the summary and graph are complete. The accepted artifact must state its applicability and supporting evidence. No new software product is required to define or test this decision path. ## Exercise Design the smallest collaboration trial that could reject the proposed handoff method. Identify the actual delivery outcome and the single transition under test. Assign proposed production, review, acceptance and publication responsibilities. Specify the artifact and evidence needed to resolve the missing constraint. Define a comparison that includes rework and review burden, then bound the conclusion. ## Model answer The trial examines whether a design rationale can move from project production to reusable knowledge without losing applicability. The team produces it; extraction and summarization assist; a discipline reviewer assesses the constraint; a designated knowledge owner accepts publication only after correction. Preserve the draft, cited evidence and rejection reason until the missing constraint is supplied and checked. Compare similar handoffs using correctness, rework, waiting and review effort, with confounding differences recorded. A successful trial supports the tested coordination method for that transition. It does not show a collaboration application was built, that the firm adopted the operating model or that staffing can be reduced. ## Rubric 3 points: Keeps engineering delivery, proposed operating model and collaboration layer in separate categories. 3 points: Defines a rejectable handoff with accountable roles, artifacts and correction routes. 2 points: Preserves applicability and disagreement through the publication decision. 2 points: Bounds trial conclusions and avoids invented adoption, product or workforce claims. ## References S020, S021, S022, S023, S026 # Separate a graph roadmap from operational readiness Turn a proposed graph roadmap into evidence-gated decisions about operational readiness. ## Objectives Distinguish roadmap milestones, reported PoC results and verified operating evidence. Build a dependency-based release argument rather than a target-driven status label. Define a partner handoff whose missing prerequisites stay visible. ## Public teaching scope and evidence boundary A graph roadmap connects source curation, semantics, governance, delivery and human accountability. It establishes intended work and priorities, not release receipts. In the fictional readiness review used here, a demonstration, a code handoff and a production target each support a different claim. A proof of concept can answer selected questions while leaving formal benchmarks and operating responsibilities unresolved. The exercise tests what additional evidence a release argument needs; it does not disclose the status of any named organizational programme. ## Convert each milestone into a claim and an acceptance event A roadmap entry such as an MVP being delivered is still a planned milestone when it appears in a future-work structure. Recast it as a precise claim: which users can do what, through which interface, under which access and support conditions? Identify the artifact or observation that would establish the claim and the role empowered to accept it. Use separate states for intended, reported, inspected and operationally verified. These states describe evidence, not a universal maturity ladder. A thoroughly inspected plan can remain unimplemented; a working PoC may lack the controls required for production. Preserve the source of every status assertion so a report about a completed review cannot silently become direct evidence of a running service. ## Make release dependencies explicit and owned In the fictional review, list the prerequisites for the intended release: an approved operating environment, appropriate access, representative tests and accountable service ownership. Turn each condition into a dependency with an owner, acceptance evidence and impact on the release. Some conditions may block all operation; others may block only a specific audience or capability. State that scope rather than collapsing every issue into a generic red flag. A partner handoff must transfer more than code access. Include the supported knowledge sources, source-update behavior, environment assumptions, test evidence, unresolved risks, operational responsibilities and rollback or suspension decisions. A review approval may cover a design scope without covering its implementation or operating environment. Record what the approval actually assessed before using it in the release argument. ## Failure diagnosis: target pressure replacing evidence A target can coordinate effort, but cannot close an unresolved dependency. Diagnose false readiness by looking for status words that changed without corresponding artifacts or observations. A demonstration may become more polished while formal benchmarks, access controls or service ownership remain incomplete. Keep these gaps visible even when a target approaches. A proposed service transition must define source and user migration, continuity and responsibilities explicitly. A change in development priorities does not establish that one service has replaced another. Without evidence of those transitions, record unresolved conditions. ## Worked example Synthetic fixture: a Graph PoC passes a design review and answers a small set of project questions during a partner demonstration. The internal landing zone remains unapproved, the test corpus excludes conflicting sources, and no party owns production incident response. A slide labels the MVP ready. The evidence supports the bounded demonstration and reported review, while production readiness remains blocked by the operating environment, missing failure coverage and unassigned support. The corrective action is to close or explicitly limit those conditions, not to add another successful demonstration of the same questions. These facts are invented exercise data for this public teaching scenario. ## Exercise Prepare a readiness decision and a partner handoff checklist for the fixture. Classify the slide, demonstration and design review by what each can establish. List the three unresolved dependencies and their effect on the release claim. Define the next acceptance evidence for each dependency. Write a status statement that preserves PoC value without claiming production delivery. ## Model answer The slide is a status assertion, the demonstration supports only its tested questions and environment, and the design review supports only its declared review scope. Production readiness is blocked by the unapproved landing zone and absent incident ownership; missing conflicting-source tests leave answer behavior unverified for that failure class. Obtain environment approval and inspected configuration, assign and exercise the incident route, and execute discriminating conflict tests against the intended build. The handoff includes source, configuration, test and ownership records. Report a useful PoC with specific production prerequisites outstanding. Do not convert the target, code access or another polished demonstration into a release receipt. ## Rubric 3 points: Distinguishes roadmap intent, mediated reports, review scope and direct operating evidence. 3 points: Turns dependencies into owned, evidence-bearing acceptance conditions. 2 points: Transfers operational and knowledge responsibilities as well as code. 2 points: Rejects target-driven readiness and unsupported service-replacement claims. ## References S020, S021, S022, S023, S026 # Make governance decisions inspectable across roles Make concept and relationship decisions understandable, challengeable and revision-aware for specialists, builders and nontechnical reviewers. ## Objectives Define concept identity and relationship meaning before accepting generated graph edges. Provide role-appropriate views without changing the underlying evidence. Record objections and revisions so governance can be inspected rather than merely asserted. ## Evidence boundary: requirements and published method The knowledge-engineering sources connect taxonomy-tool specification, SKOS/OWL-informed relationship naming, graph reasoning and workshops on governance and knowledge architecture. They establish requirements and intellectual positions, not a verified semantic platform. The workshop requirement is instructional: participants lack context, and the session should help them understand rather than serve as a pitch. The published method treats machine-generated concepts and relations as proposals requiring purpose, identity, definitions, evidence and revision. It also distinguishes machine representation from the human view needed to judge a particular question. Conditional scenarios about future production and assurance do not establish current organizational roles or reliable workforce forecasts. Use the principles to design a reviewable decision process, not to invent adoption. ## Separate labels, concepts, resources and relations A label is a way to name something; a concept is the identified thing the system means; a resource can discuss or exemplify it. Similar labels do not establish identity, and co-occurrence does not establish a meaningful relation. Define what the concept includes, excludes and is used for before merging candidate nodes. For each proposed relation, state its meaning, direction, allowed endpoint types and supporting evidence. A broader-topic relation differs from part-of, about, produced-by or cites. Choosing an impressive formal vocabulary cannot repair a mistaken relation. Keep extraction confidence separate from governance acceptance: the model may confidently detect a phrase while the organization has not decided what concept it should denote or whether the proposed relationship is useful. ## Give each reviewer a view suited to the decision A machine-facing graph may need stable identifiers and typed edges. A domain reviewer may need a side-by-side comparison of definitions and source passages. A delivery owner may need a dependency view showing which answers or workflows change if a relation is accepted. A newcomer may need a concrete example before seeing the graph. These can be different views of the same versioned evidence. Preserve the route from every displayed conclusion to the object, revision and supporting passage. Visual prominence and ordering can hide qualifications even when the data model retains them. Put unresolved objections where they affect the decision, not behind a generic information icon. The goal is practical examinability: a reviewer should be able to identify what is proposed, why it matters and how to challenge it. ## Failure diagnosis: approval without an inspectable decision A governance workshop can produce broad agreement while leaving concept boundaries and ownership ambiguous. Diagnose this by asking a participant to evaluate a difficult example using the accepted definition. If two reviewers reach incompatible decisions, preserve the disagreement and refine the rule or scope; do not conceal it behind a unanimous meeting summary. Maintain a decision record with the proposal, evidence, objections, disposition, responsible role and affected revision. An accepted relation may become inappropriate when its source or intended use changes. Determine which downstream representations need reconsideration and expose superseded decisions rather than deleting their rationale. Governance is a continuing revision path, not a permanent approval badge attached to a model-generated graph. ## Worked example Synthetic fixture: extraction creates nodes named Bridge design and Bridge design guide, then links the guide as a narrower concept because the labels overlap. Inspection shows the second node is a document about the first topic. A specialist rejects the narrower-concept edge and proposes a resource-about-topic relation. The developer needs endpoint types and stable identifiers; the newcomer needs the source passage and a plain-language explanation. An answer system using the old edge must be assessed for impact after correction. This is a constructed taxonomy exercise, not a finding about the implemented taxonomy visualizer or an organizational graph. ## Exercise Produce a governance decision packet for the proposed correction. Define the two node identities and the meanings of the competing relations. Specify the source evidence and the objection that rejects the narrower-concept edge. Design specialist, developer and newcomer views of the same decision. Record the revision and identify downstream checks that the change should trigger. ## Model answer Bridge design is a topic; Bridge design guide is a resource discussing that topic. Their relationship is aboutness, not a narrower concept within a taxonomy. The packet cites the guide's identity and relevant passage, preserves the rejected edge and explains the endpoint-type error. The specialist view compares meanings and evidence, the developer view specifies identifiers and types, and the newcomer view uses a plain example without hiding the qualification. Record the accepted correction and its responsible role, then check affected navigation, retrieval and generated answers. Do not claim that choosing a formal vocabulary alone proves the graph correct or that workshop agreement establishes implementation. ## Rubric 3 points: Separates labels, concepts and resources and defines relation meaning and endpoint types. 3 points: Provides evidence-linked, role-appropriate views of one consistent decision. 2 points: Preserves objections, revisions and downstream impact checks. 2 points: Keeps requirements, proposals and conditional scenarios distinct from deployed governance. ## References S060, S059, S061, S062, S063, S023, S024, S025, S026 # Integrate a source-to-action workflow and defend its limits Integrate source selection, analysis, draft production and authorized action into one defensible workflow without hiding the gaps between projects. ## Objectives Design a source-to-action contract with explicit interfaces between retrieval, analysis and execution. Carry revisions, permissions and unresolved claims through a composite workflow. Defend a completion statement using component-specific evidence and visible exclusions. ## Public teaching scope and evidence boundary This capstone combines a local analysis workspace, a proposed metadata-enrichment stage, a source-linked response draft and Boris-style durable action contracts. The workspace components are an instructional design. Boris contributes specified revision, reconciliation and evidence mechanisms; the combination does not establish a demonstrated integration. Keep the local export, proposed metadata changes and external communication as separate effects. Each needs its own acceptance evidence. The exercise is a design problem, not a claim that a unified autonomous platform exists. ## Define the source-to-draft interfaces Start with one RFP section and a bounded approved source collection. Record the question, source identities, applicable revisions and intended output. Taxonomy, tagging and findability help determine which material is suitable before generation. Folder membership alone does not establish currency or relevance. Use the illustrative local analysis stage to extract candidate evidence into a reviewable table. Each row includes the proposition, exact source location, qualification, applicability and unresolved conflict. Local spreadsheet output does not upload itself. The synthesis consumes reviewed rows; structured output does not turn every sentence into an accepted fact. ## Separate proposed knowledge updates from authorized effects A metadata writeback is a mutation, unlike a local analysis export. A Boris-style contract distinguishes proposed changes, required approval, dispatched actions and verified results. Carry target, payload revision and permission scope into the action record. Source content is evidence to interpret, not authority for new actions. Keep the human-readable draft and machine-readable evidence aligned through stable claim and artifact identities. Preserve reviewer corrections and invalidate affected acceptance results. If source permissions narrow, revisit the draft and derivatives. An architecture drawing does not itself establish access control, monitoring or credential protection. ## Failure diagnosis: successful components, unsupported whole The composite workflow can fail at its seams even when every component produces an output. A static dashboard can conceal stale coverage; a local spreadsheet can contain unsupported claims; a plausible draft can inherit obsolete permissions; a send can succeed without its receipt being saved. Test the interfaces with contradictory sources, missing passages, human edits, permission changes and interrupted side effects. Completion requires the approved artifact revision, the required evidence and verified authorized effects. Keep economic claims separate: qualitative testimonials and promotional projections do not establish measured savings or bid-win improvement. Keep architectural claims separate too: module names and queues do not prove autonomous multi-agent behavior. A defensible capstone ends with a scoped result, unresolved areas and a clear account of which parts were simulated. ## Worked example Synthetic fixture: an RFP asks for a capability and a response time. Source A supports the capability but is restricted to internal use. Source B is approved for external use and supports a narrower capability. A static dashboard lists an older response-time promise, while the source collection contains no current support for it. The workflow produces a local evidence table and a draft using B's narrower statement, with response time marked unresolved. It proposes an internal clarification and a metadata correction but does not execute either automatically. Before authorized release, the draft changes from D3 to D4. D3's review and send permission must be checked against the changed contents, and any uncertain send must be reconciled before retrying. ## Exercise Produce a reviewable capstone contract, evidence table specification and final status statement. Map each RFP assertion to its source, revision, permission and qualification. Specify local outputs, proposed writebacks and external communications as different operations. Define tests for missing response-time evidence, permission narrowing, human edits and an interrupted send. Identify which D3 acceptance and approval records remain applicable to D4. Write the strongest justified completion statement and name the unverified integration boundaries. ## Model answer The evidence table supports only the externally permitted narrower capability from B; A can inform internal review but cannot be copied into the external draft under the fixture's restriction. The old dashboard is insufficient support for response time, which remains unresolved. Local analysis completes independently of proposed metadata changes or communications. Each mutation requires its own applicable scope, revision and result evidence. Review D4's changes against D3's approval and rerun affected checks before release; reconcile an uncertain send against destination evidence rather than resending blindly. The status may report a completed, reviewed draft with one unresolved requirement, but not a complete submitted response until authorized delivery is verified. The integration remains a teaching design, and Boris remains a specified design; the other components are illustrative. ## Rubric 3 points: Carries source support, revision, qualification and permission through the whole workflow. 3 points: Separates local analysis, proposed knowledge mutation and authorized communication. 2 points: Handles missing evidence, revision changes and interrupted effects with explicit tests. 2 points: Defends a bounded completion claim without inventing deployment, interoperability or economic outcomes. ## References S020, S021, S022, S023, S026 # Measure the information gap before adding automation Use observed information requests to identify a service problem. Group queries by intended task, inspect failed or abandoned routes, and distinguish a missing synonym from a missing document, unclear ownership or genuinely absent knowledge. A query-log case links repeated terms to navigation changes, better retrieval matches and knowledge-gap filling [S029]. The transferable method is a feedback loop: observe demand, classify the failure, make one bounded change and retest comparable requests. Query volume measures demand, not the value of the intervention. Preserve the denominator and sampling method before claiming an improvement. # Separate structural counts from semantic correctness A taxonomy can pass a file-level audit and remain conceptually wrong. Count records, unique concept IDs, relation types and unmapped terms separately. Then sample definitions, broader/narrower relations, duplicate meanings and category assignments against an explicit expert rubric. A recovered classification workbook contains 1,184 rows but 1,182 distinct concept strings; its summary and mapping sheets each contain 36 rows [S027]. Those are inspectable structural facts. They do not measure the accuracy of the classifications. A useful exercise is to seed one duplicate, one unsupported relation and one valid but differently labelled synonym: the structural audit and semantic review should catch different problems. # Budget for upkeep and use fallback retrieval A navigational map creates maintenance work whenever concepts, ownership or source locations change. Record who maintains each structure, which signals trigger review and how a user reaches the source when the map is stale. A findability case reports that comprehensive link maps became difficult to maintain and that bounded snapshots remained useful [S030]. Choose an index or graph because it improves a defined task after maintenance cost, not because its visual density looks comprehensive. Compare direct lookup, navigation and search on the same questions, retaining unresolved cases and the cost of repairing stale paths. # Make research chains preserve negative evidence A research chain can extract passages, classify them, synthesize a finding and render a dashboard while losing the absence of supporting evidence. Keep the question, extraction category, source span, surrounding context and explicit no-match state through every stage. The client-segmentation method proposes categorical extraction with nearby sentences and source checks before synthesis [S031, S032]. The RFP-analysis case reports outputs linked to direct supporting quotations [S028]. Treat those as design/report evidence, then test your own chain with an irrelevant document, a misleading keyword match and a passage that supports the opposite conclusion. No-match is a valid output, not an invitation to invent a plausible paragraph. # Audit the gap between the architecture diagram and the runnable boundary For each component in a proposed architecture, label its state: observed, reported, specified or proposed. Trace a real action through identity, input, model call, output store and verification. In this fictional review, the drawing promises hosted controls while the runnable baseline has only local output. Identify the missing receipts rather than treating the drawing as implementation. Exercise: remove a promised component and identify which requirements fail. Several component names do not prove multi-agent autonomy. # Evaluate useful work rather than activity Keep quality, elapsed time, review effort, error severity and operating cost separate. Count comparable completed tasks, not messages, logins or generated tokens. Design an evaluation that measures source checking, substantive feedback and repair of discovered gaps. Define a baseline and record costs that move from production to review. A proposed comparison supplies a study design, not a measured benefit. Retain unresolved cases; skipping a hard case is not success. # Constrain service routing and escalation Use deterministic routing for clear service intents and grounded answers where interpretation is needed. In this fictional service contract, distinguish resource routing, sourced answers, licensed-content boundaries and escalation; purchase commitments remain outside the agent’s scope. Test a changed intent midway through the conversation so the original route and stale fields cannot silently govern the new request. A prepared handoff is different from verified delivery or fulfilled service. # Source register S001 — Retained reference: Write a bounded task contract from a product goal Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S002 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S003 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S004 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S005 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S006 — Retained reference: Write a bounded task contract from a product goal Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S007 — Retained reference: Write a bounded task contract from a product goal Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S008 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S009 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S010 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S011 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S012 — Retained reference: Preserve working context without laundering memory into truth Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S013 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S014 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S015 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S016 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S017 — Retained reference: Model recorded media as temporal, revisable assertions Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S018 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S019 — Retained reference: Model recorded media as temporal, revisable assertions Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S020 — Enterprise Search Needs to Know When to Look Again Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/enterprise-search-needs-a-researcher/ S021 — Stop Retrieving. Start Navigating. Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/stop-retrieving-start-navigating/ S022 — Agent-Generated Tests Do Not Prove Software Is Ready Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/agent-generated-tests-software-readiness/ S023 — One Body of Knowledge, Two Different Views Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/knowledge-engineering-two-representations/ S024 — The Next Decade of Knowledge Engineering Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/knowledge-engineering-the-next-decade/ S025 — What Knowledge Engineering Might Become Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/knowledge-engineering-after-the-interface/ S026 — An AI Still Needs to Know What You Mean Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/ontology-ai-library-principles/ S027 — Retained reference: Expand a taxonomy without confusing volume with correctness Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S028 — Retained reference: Turn RFP questions into quotation-backed analysis Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S029 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S030 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S031 — Retained reference: Preserve negative evidence in chained research Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S032 — Retained reference: Preserve negative evidence in chained research Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S033 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S034 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S035 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S036 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S037 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S038 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S039 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S040 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S041 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S042 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S043 — Retained reference: Measure the information problem before automating Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S044 — Retained reference: Turn RFP questions into quotation-backed analysis Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S045 — Retained reference: Turn RFP questions into quotation-backed analysis Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S046 — Retained reference: Expand a taxonomy without confusing volume with correctness Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S047 — Retained reference: Expand a taxonomy without confusing volume with correctness Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S048 — Retained reference: Expand a taxonomy without confusing volume with correctness Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S049 — Retained reference: Expand a taxonomy without confusing volume with correctness Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S050 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S051 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S052 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S053 — Codex Is Becoming a Delegation Layer, Not Just a Coding Tool Public methodology reference. Its arguments do not independently establish implementation or outcomes for the course projects. https://recluse.studio/blog/delegation-changes-the-shape-of-work/ S054 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S055 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S056 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S057 — Retained reference: Define what each kind of evidence can establish Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S058 — Retained reference: Write a bounded task contract from a product goal Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S059 — Retained reference: Govern concepts, labels and graph relationships Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S060 — Retained reference: Govern concepts, labels and graph relationships Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S061 — Retained reference: Govern concepts, labels and graph relationships Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S062 — Retained reference: Govern concepts, labels and graph relationships Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S063 — Retained reference: Govern concepts, labels and graph relationships Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S064 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S065 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S066 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S067 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S068 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S069 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S070 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S071 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S072 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S073 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S074 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S075 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S076 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S077 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S078 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S079 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S080 — Retained reference: Decompose Ralph work into independently rejectable stories Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S081 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S082 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S083 — Retained reference: Coordinate parallel agents through interfaces and file ownership Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S084 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S085 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S086 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S087 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S088 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S089 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S090 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S091 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S092 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S093 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S094 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S095 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S096 — Retained reference: Debug the boundary between model, harness and tools Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S097 — Retained reference: Preserve working context without laundering memory into truth Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S098 — Retained reference: Evaluate feature value without optimizing a proxy blindly Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S099 — Retained reference: Evaluate feature value without optimizing a proxy blindly Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S100 — Retained reference: Evaluate feature value without optimizing a proxy blindly Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S101 — Retained reference: Evaluate feature value without optimizing a proxy blindly Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S102 — Retained reference: Evaluate feature value without optimizing a proxy blindly Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S103 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S104 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S105 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S106 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S107 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S108 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S109 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S110 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S111 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S112 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S113 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S114 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S115 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S116 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S117 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S118 — Retained reference: Evaluate prose and experimental agent language without flattening judgment Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S119 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S120 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S121 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S122 — Retained reference: Design portable model state and reproducible execution plans Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S123 — Retained reference: Turn inbox capture into governed knowledge Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. S124 — Retained reference: Separate personal and distributable product responsibilities Underlying conversations and documents are not reproduced or publicly linked. The lesson explicitly distinguishes specifications, author reports, inspected artifacts and reproduced tests. This reference is not independently accessible evidence for the reader. # Scholarly foundation connections Full chapter text and answer keys: [foundations.md](foundations.md). ## Measure the information problem before automating A ten-document test collection contains four relevant documents. Returning three documents with two relevant gives precision 2/3 and recall 1/2. These denominators expose different errors, but neither establishes whether the controlling exception was found. Compare the retrieved evidence with the task the query expresses, and retain uncertainty about that interpretation. ## Turn RFP questions into quotation-backed analysis An RFP is a request for suppliers to describe what they can provide. Its clauses may apply to different submission stages. Represent support as a relation among a claim, source version and applicable condition; “every claim has some source” does not mean one source supports every claim. Test a changed exception while holding the rest of the passage fixed. ## Expand a taxonomy without confusing volume with correctness A label is a string used to name a concept; it need not identify the concept uniquely. Give generated candidates stable identifiers and explicit scope notes before reviewing their relations. Repeating a classification over the same fixed set satisfies A ∩ A = A, but does not produce an independent confirmation of the classification. ## Define what each kind of evidence can establish All approved records may be verified while some verified records remain unapproved. A four-record set can make the premise true and the proposed converse false. This countermodel rejects the inference without disputing the premise. Then inspect whether the premise itself is supported by current evidence; logical validity and empirical warrant require separate checks. ## Write a bounded task contract from a product goal A precondition states what must hold before a procedure starts; a postcondition states what must hold if it finishes correctly. Preserving an invariant through each step supports partial correctness. A separate decreasing measure can establish termination under stated assumptions. Give the contract both obligations rather than allowing “keep improving” to serve as a stop condition. ## Build section-wise document review with a human boundary A reviewer needs enough time and visible evidence to identify an error and reject the proposed change. Bainbridge’s analysis of automation describes why monitoring and recovery can become difficult precisely when routine work has been automated. Test a planted defect with an unfamiliar reviewer and record the assistance they needed, alongside the final approval. ## Carry evidence through work-winning research and drafting A source can be used by a drafting activity without supporting every sentence it produces. Separate quotations, paraphrases and new inferences, then retain a source version and applicable span for each consequential claim. Two derivative summaries of one original do not provide two independent evidential origins. Trace the draft’s claims back before counting citations. ## Preserve negative evidence in chained research A document listing CSV and JSON support does not establish that PDF is unsupported unless the list is declared exhaustive. Open-world reasoning keeps absence separate from falsity. Record whether a negative finding came from an explicit prohibition, an inspected complete list or a source that simply says nothing about the feature. ## Model recorded media as temporal, revisable assertions A recorded statement has a speaker, a position in a recording, a source version and a later interpretation. Those fields support different checks. PROV distinguishes entities, activities and attribution; a transcript correction should create a revised record and identify affected assertions without silently replacing the history that explains the correction. ## Govern concepts, labels and graph relationships SKOS permits hierarchical cycles even when a browsing application forbids them. State the local acyclic policy and show the offending path. Direct broader edges also differ from transitive reachability: A broader B and B broader C do not require a direct A broader C assertion. Preserve asserted and derived edges separately. ## Maintain human and machine views of the same knowledge Exporting a concept graph as a table or diagram may retain labels while losing relation direction, proposed status or source identity. Gruber’s portability question is therefore semantic as well as syntactic. Define a round-trip test over the distinctions needed by the next reader, and mark any properties the simpler view cannot represent. ## Decompose Ralph work into independently rejectable stories For a fixed collection of N records, a loop index i can mark the examined prefix while N−i decreases after each terminating examination. The argument establishes eventual coverage under those assumptions. It does not establish correct classifications, and it fails if the input grows without a bound or a worker can wait forever. ## Coordinate parallel agents through interfaces and file ownership Parnas’s information-hiding criterion organizes modules around decisions likely to change. Two workers with different titles may still depend on the same mutable schema or asset. Specify read/write footprints and stable handoff records, then change one adapter and inspect which components require repair. File ownership alone does not prove semantic independence. ## Debug the boundary between model, harness and tools A tool request may parse correctly while naming an unavailable document or the wrong pending call. Types can exclude specified classes of mismatch within a formal system, but a valid SourceSpan can still cite the wrong source. Retain the first failing layer so that a schema repair is not mistaken for evidence of a correct external effect. ## Preserve working context without laundering memory into truth A memory summary is a representation selected for later use. It may omit the condition that made a claim true. Record its source version and dependencies, and mark it for review after source withdrawal or revision. A belief-state model requires explicit transition and observation assumptions; a natural-language summary is not automatically a sufficient statistic. ## Evaluate feature value without optimizing a proxy blindly A numerical benefit score requires an agreed scale and a maintenance horizon. Before maximizing it, remove alternatives that violate non-tradeable constraints. Then vary uncertain estimates and inspect whether the preferred choice changes. Enumeration over a few indivisible features is a finite allocation problem, not evidence that arbitrary product design is convex optimization. ## Evaluate prose and experimental agent language without flattening judgment A grammar recognizes permitted strings; an interpreter assigns operations to their structure. Test unfamiliar combinations after teaching every component, so that missing vocabulary is not confused with compositional failure. The proposed agent language also needs execution evidence and an independent aesthetic judgment; successful parsing establishes neither of those results. ## Make actions recoverable and authorization revision-bound A lost acknowledgement is compatible with a remote action occurring or failing to occur. Repeating the action can duplicate an effect. Preserve a dispatched-but-unresolved state, correlate the intended revision with the request, and inspect the destination before deciding whether a retry is appropriate. Clock order alone does not establish the causal relation among approval, dispatch and observation. ## Design portable model state and reproducible execution plans A portable file can preserve bytes while losing the interpretation of a state code, a dependency or a permission record. A two-bit encoding of four states needs a shared codebook. Include versioned semantics and a reproducible execution fixture when assessing portability; successful serialization answers only part of the question. ## Translate accessibility commitments into acceptance tests A designer’s successful rescue of an interface does not establish that an ordinary user can recover. Human–AI interaction research motivates visible capability limits, correction and dismissal. Ask an unfamiliar reader to complete the acceptance task and log each hint, failed attempt and workaround so that assisted success is not reported as unassisted success. ## Turn inbox capture into governed knowledge “Please export,” “I will export,” and “the source says export” have related content but different functions. Searle’s speech-act distinctions and historical FIPA message definitions make this separation explicit. An inbox record needs to preserve whether a sentence requests, commits, reports or quotes before any authorized effect is considered. ## Separate personal and distributable product responsibilities Personal and distributable versions can have different obligations while sharing code. Identify which module owns source identity, permissions and schema evolution, then test a change at that boundary. Separate folders do not guarantee separation when a shared build step or generated asset can modify both products. ## Build narrow, checkable certificates from WITNESS With fixed endpoints and adjacency, a path in a lower foreground mask remains a path in any containing mask. If even an upper mask has no path, every contained actual mask lacks that path. The remaining case is undetermined. A certificate checks this bounded connectivity claim; it does not establish the physical meaning of the drawing. ## Test visual preservation and asset completeness A completed transfer or successful render command may still leave the reader with missing assets or illegible labels. End-to-end reasoning asks which property the recipient actually needs and where it can be checked. Retain the rendered view, dimensions and comparison conditions rather than inferring visual preservation from file existence alone. ## Test the boundary of a software-readiness claim A passing execution, an exhaustive finite-state check and a proof over a formal model establish different things. Zero failures in a selected regression suite does not directly estimate deployment risk. State the sampling procedure, assumptions, tested revisions and unmodeled conditions before using a pass count to support a readiness claim. ## Bound a service agent through routing and escalation The same probability of a wrong answer can justify different actions when error costs differ. In a constructed model where a wrong release costs100 and perfect review costs4, automatic release has lower expected loss only above0.96 correctness. The calculation depends on its assumptions; apply required permission and review constraints before ranking feasible actions. ## Convert a failure record into a reusable tool safely A repaired failure case is valuable regression evidence, but tuning repeatedly against it can overfit the repair. Preserve that known case for development and add a separately governed transfer test. Record the changed condition that the reusable tool claims to handle and require the resulting artifact, not only a reassuring execution transcript. ## Build a maintained knowledge-curation supply chain A curation update changes the collection, while task outcomes may arrive later and include noise. In a toy recurrence, e(t+1)=(1−k)e(t), convergence for every initial error requires0