The Expert-System Problem Is Older Than the Computer

The early expert-system builder asked a reasonable question: what does the specialist know that the program does not? The answer became troublesome when the specialist could perform a task, explain part of it, and still fail to supply a complete rule for the program. The difficulty was not invented by computing. People had been distinguishing among ways of knowing for more than two thousand years.

Aristotle’s account is worth visiting at this point in the story because it breaks the assumption that knowledge is one substance that can be moved from a person to a database. In Book VI of the Nicomachean Ethics, he distinguishes epistēmē, commonly rendered as scientific knowledge; technē, skilled making; phronēsis, practical judgment; and sophia, a kind of theoretical wisdom. These are not modern software categories, and treating them as such would flatten the argument. They name different human capacities that may be involved in one piece of work. (Aristotle, Nicomachean Ethics, Book VI)

The instruction is not the craft

A shipbuilder can know why a timber behaves in a particular way, know how to make a sound joint with it, and judge whether that joint is suitable for this vessel under these conditions. A written specification can preserve dimensions and requirements. It may be excellent. Another person could memorize every line and still be unable to build the ship.

This is not a romantic argument that craft cannot be taught. It is a distinction about what the document does. The apprentice needs instruction, material practice, correction, and the ability to notice when the timber in front of them differs from the timber assumed by the specification. The master builder may use a drawing to explain a joint, then correct the apprentice’s hand at the workbench. Knowledge is being transmitted through more than sentences.

The same problem confronted knowledge engineers. A rule from a medical specialist could state when a finding supported a diagnosis. It might not contain the specialist’s ability to recognize a poor sample or decide which question to ask next when the case does not fit. A configuration rule could forbid one combination of computer parts while leaving out why a seemingly similar combination was safe. The system could still be useful, provided its builders were exact about which part of the work it represented.

Knowing a fact is not judging its use

The difference between technē and phronēsis is particularly useful here. Skilled making concerns the production of something. Practical judgment concerns what should be done in a situation where the circumstances could be otherwise. The same person may exercise both. A specialist can know how to perform a procedure and decide that this is not the moment to perform it.

This distinction does not produce a simple rule that humans must always decide and machines must only advise. It asks a more precise question: which conditions for action have been represented, which remain outside the system, and who is responsible for decisions at that boundary? A well-designed program can apply a requirement consistently. It may not have the evidence or authority to determine whether the requirement should govern an unusual case.

There is also knowing through acquaintance: knowing a place, a person, or the smell of cedar. Reading about cedar does not provide the sensory acquaintance. Such knowledge may still matter in a task, and its absence may become visible only when a system encounters a case that a practitioner would recognize immediately. (Stanford Encyclopedia of Philosophy, acquaintance and description)

The question becomes sharper

I do not think Aristotle solves the design of an expert system from a distance of two millennia. He gives us a useful refusal. A system’s stored propositions, a person’s practiced skill, and a person’s judgment under changing conditions should not be called the same thing merely because all three support an answer.

The early builders were selecting parts of expertise for representation, making those parts operational, and testing the result against cases. Once that is clear, the next problem follows. Some expertise can be stated but has not been written down. Some cannot be fully stated even by the person who uses it. The difference would become central to knowledge management and to the limits of knowledge engineering.