AI Can Help Build Knowledge Resources, but Someone Must Test Them

Give a language model a folder of product documents and ask it to propose a classification. It can produce names, group similar items, identify possible synonyms, and suggest relationships before a person could type the first complete hierarchy. It may also merge products whose differences matter, invent a category the business does not use, or treat one ambiguous sentence as a settled rule.

This is the two-sided change generative AI brings to knowledge engineering. AI applications need reliable external knowledge, and AI can help produce some of the resources those applications use. Current employer roles already include model building alongside generative-AI-assisted creation and evaluation. That is evidence of a changing workflow, not proof that the whole discipline has been automated. (Amazon, Ontologist—Product Knowledge)

A proposed relationship is not a fact

Suppose the model reads ten catalogues and proposes that two terms mean the same thing. It may be right. Or one term may name a family while the other names a specific model. The knowledge engineer needs to inspect the source passages, ask a subject specialist when the distinction is consequential, and test how the mapping behaves in actual searches and decisions.

The work is not best measured by how quickly the first taxonomy appears. A rapid draft can save time if review is faster than construction and if errors are discoverable before they affect users. If review requires checking every item from scratch, or if plausible errors survive into production, the apparent speed may be an accounting trick. An AI-assisted method needs evaluation on difficult cases, exceptions, multilingual terms, and changes over time.

The same applies to extracted rules. A model might turn “normally inspect after 500 hours” into an unconditional 500-hour requirement. The adverb could be carrying the exception that matters most. The resulting rule may pass a schema check and still misstate the source. Keeping the source passage, extraction method, reviewer, and approved wording together makes correction possible.

The engineer’s work changes shape

If machines produce first drafts of classifications and mappings, the human task may shift toward defining acceptable sources, setting review priorities, constructing representative tests, and investigating consequential failures. Less time may be spent entering individual relationships; more may be spent deciding whether a model represents the field faithfully enough for a particular use.

This does not make the specialist’s judgment optional. It changes when that judgment is applied. A subject expert may no longer spend an hour describing routine categories but may need to examine the cases where two categories nearly overlap. A knowledge engineer may spend less time typing nodes and more time tracing why an application applied the wrong definition. The field’s old maintenance problem does not vanish when a model can propose updates; it may arrive more frequently.

There is a training question as well. Junior practitioners have often learned by constructing records and seeing how experts correct them. If routine construction is automated, teams will need deliberate routes through case review, source analysis, and model failure so newcomers can acquire judgment rather than only approve machine output.

The system is now capable of helping build its own knowledge supply. That makes the quality of evaluation especially important. It also makes the human-facing representation more consequential. A reviewer cannot inspect thousands of proposals as a flat list. They need a design that reveals patterns, disagreements, and effects of change. To understand that design problem, the next part of the series steps backward through the older history of making knowledge visible.