Deleting a source record and removing its learned effects

A file disappears from a collection. A system trained with that file may still contain effects of having learned from it. Those are separate conditions, and a promise about one should not silently become a promise about both.

Machine unlearning studies how a trained model can be changed after some training data must be removed. For librarians evaluating an AI service or managing a research collection used in training, the practical question is precise: what does the provider mean by deletion, and what evidence supports the particular result it claims?

The Atlas treats this as a reference case that tests its broader theory of unfinished passage. The source may be gone while a derivative remains operative. That does not make every learned effect a harm or every unlearning method impossible; it requires the promised destination to be defined.

A model is a different object from its training collection

Antonio Ginart, Melody Guan, Gregory Valiant, and James Zou’s 2019 paper formulates the problem of deleting data from trained models. Their efficient methods are demonstrated for a particular clustering setting, rather than for every kind of AI system.

The useful distinction is between changing an input collection and changing what a completed training process produced. A library could remove a document from a dataset it controls while lacking the model, training procedure, or authority needed to determine whether a supplier’s derived system has changed.

I would separate those claims before discussing a service with a contributor. Removing the stored source, preventing future training use, changing an already trained model, and addressing copies held elsewhere are different actions. Their completion may require different parties and different evidence.

That separation also keeps the technical vocabulary manageable. The issue does not require a reader to inspect millions of numerical parameters. It requires an institution to identify which artifact its deletion claim covers and whether the claimed operation reaches the artifact whose continuing use concerns the person.

A successful output is not a complete audit

Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Papernot’s 2022 USENIX paper examines the definition and auditability of unlearning. They show why inspecting a resulting model cannot, under the conditions they study, establish the complete history of which data produced it.

Their argument places the auditable algorithmic process at the center of a credible claim. It is a limit on particular forms of proof, rather than a demonstration that every attempt to remove learned influence is useless or that no bounded technical guarantee can exist.

A simple demonstration might ask a model for material associated with a removed record and receive no such material. That result describes the attempted query. It does not by itself establish that every relevant influence has disappeared, because the query and its observed answer cover only a limited part of the model’s possible behavior.

The opposite observation also needs care. A later output resembling the removed material may require investigation of duplicates, other sources, or the scope of the removal process. One observation cannot supply an unexamined account of the entire training history.

This is where the Atlas’s question about completion becomes useful. A visible interface event is easy to recognize. An absence defined against a model that never encountered the data requires a more carefully specified standard and an evidence chain capable of supporting it.

Positive guarantees have declared boundaries

Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens van der Maaten’s 2020 paper on certified removal provides an important positive result. The authors define a strong removal guarantee and develop a mechanism for linear classifiers.

That result prevents an easy fatalism about deletion. A bounded method can have a meaningful guarantee while still requiring the model class, assumptions, and process described by its authors, so a result for a linear classifier cannot be transferred automatically to a supplier’s generative model.

For a library, the appropriate procurement question is therefore about the actual system being offered. Which data and derivatives are covered? What procedure is used? Which guarantee is claimed? Who can inspect the evidence, and what remains outside its scope?

The NISO privacy principles provide professional grounds for requiring a clear account of collection, use, retention, and responsibility. The technical papers supply distinctions needed to evaluate a particular model-level promise. Neither source substitutes for a verified account of the contracted service.

I would document those answers before describing deletion to contributors or users. A library should be able to say that it can remove a controlled source without implying that this action also certifies changes to systems it does not control. If a supplier supports a further operation, its scope and proof should remain attached to that claim.

The review can stay bounded. It need not reproduce an entire research field to establish that a provider’s answer fails to identify the artifact being changed or the process that supports its guarantee. That missing information is already a precise unresolved condition.

Before marking a removal request complete, compare each promised action with its retained evidence. Record source removal, future-use restrictions, model changes, and external copies separately, then tell the requester exactly which results the institution has established and which remain unresolved.