The Knowledge Acquisition Bottleneck Was Not Just an Interview Problem
Expert-system builders found that obtaining and maintaining specialist rules required more than extracting statements from experts.
Suppose a knowledge engineer asks a specialist how to decide whether a component can be reused. The specialist gives a clear answer. The engineer turns it into a rule. Then a real case arrives, and the specialist rejects the component for a reason they did not mention. Asked why, they say the exception was obvious from the drawing.
The engineer did not fail to take notes. The conversation exposed only part of the work. Expert-system projects encountered this problem often enough that knowledge acquisition became known as a bottleneck: getting useful specialist knowledge into a working system was slower and harder than the ambition to apply it. The label can make it sound as though the solution is simply to interview faster. The difficulties run deeper. (Studer, Benjamins, and Fensel, 1998)
An expert’s explanation is evidence, not a transfer
A specialist may omit a condition because the people around them already know it. They may act on a visual pattern before they can name it. They may use a local term whose meaning changes across departments. Another specialist may reach the same decision for a different reason or reach a different decision on the same case. None of this makes the specialists unreliable. It means practical expertise is situated in tasks, materials, and institutions.
The original knowledge-acquisition metaphor encouraged an overly clean sequence. First, find the expert. Second, extract what the expert knows. Third, encode it. The system then possesses the knowledge. DENDRAL and MYCIN had already shown a more iterative reality: cases exposed missing distinctions and required the engineer and specialist to return to the work. Commercial systems made the cost of repeating that cycle impossible to ignore.
The engineer’s questions also shape the result. Ask “What rule do you follow?” and a person may give the official rule. Show them three awkward cases and ask what they would do, and the differences among the cases may reveal what the rule left out. Observe the work and a gesture toward a drawing may matter more than another paragraph of explanation. The method used to investigate expertise changes which expertise becomes available.
More rules can make the system worse
The usual repair for a missed case is tempting: add a rule. But rules interact. A new condition may overlap an old one, contradict a previously settled exception, or apply more broadly than its author intended. A system can accumulate a large amount of correct-looking local advice while losing a coherent account of the task it performs.
Consider the reused component again. The question may depend on version, intended use, loads, applicable standard, inspection history, and whether the evidence remains valid. Encoding one exception without representing these distinctions could make the immediate case pass while creating several new mistakes. The structure of the domain matters as much as the wording of any one rule.
Maintenance therefore belongs inside knowledge acquisition, not after it. When a standard changes, someone needs to know which rules depended on it. When two teams use the word “approved” differently, a shared system must not silently treat their records as equivalent. When an expert leaves, the organization needs more than their last interview transcript; it needs a way to test whether the system still makes sound distinctions.
The question behind the bottleneck
By the late 1980s, the field was confronting a problem its first successes had partly concealed. Expert knowledge was not an object waiting in a person’s head for transfer into code. The engineer had to construct a representation for a particular purpose, with the specialist, the cases, and the institution all affecting what it meant.
That realization takes us somewhere older than computing. People had distinguished knowing a fact, knowing how to make something, and judging what to do in a particular situation long before expert systems existed. The bottleneck becomes easier to understand once those forms of knowing are brought back into view.
