Synthetic Data Needs Verification and Variation

Scope note: This review covers seven recent studies of recursive synthetic training, verification, data ratios, preference diversity, and source-grounded generation. It does not estimate how much synthetic data any named frontier model uses.

Synthetic data is not one thing.

It can be a model repeating its own output without review. It can be a verified solution to a problem with a known answer. It can be a faithful reformulation of human material. It can also be a curated set selected by one reward function until the output becomes less varied.

Seven recent preprints explain why those differences matter. Their results do not support the simple claim that synthetic data inevitably causes model collapse. They support a conditional claim: recursive training degrades when the process removes information, variation, or connection to an external standard.

Verification changes recursive training

Escaping Model Collapse via Synthetic Data Verification analyzes repeated training when an external verifier selects generated data. In theory and in experiments with regression, images, and text summarization, verification prevented the usual collapse and could produce early improvement. The longer-term result depended on verifier quality. If the verifier was imperfect, the model eventually approached the verifier’s own limits, and early gains could stop or reverse.

This is a useful correction to the idea that filtering automatically solves the problem. A verifier adds information only to the extent that it can distinguish better output. Repeated selection can reproduce the verifier’s errors and preferences.

Seed2Scale reports a practical version for embodied AI. A small action model gathers trajectories, a larger vision-language model judges success and quality, and the target model learns from the selected examples. Starting from four demonstrations, the reported target success rate improved by 131.2 percent across iterations.

The system succeeds by separating generation, evaluation, and learning. The same model is not solely responsible for creating and approving its next training set.

One reward can reduce variation

Curated Synthetic Data Doesn’t Have to Collapse studies curation under several reward functions. A fixed reward tends to concentrate probability on a small set of highly rewarded outputs. With several preferences, the theoretical process can preserve probability across several useful regions.

This has direct consequences for alignment data. If one judge consistently rewards a particular tone, argument form, or answer length, repeated training can reduce alternatives that remain valid. Multiple independent criteria do more than improve average quality. They can preserve differences that the task requires.

The Necessity of Imperfection makes a related argument from human text. The authors claim that standard synthetic generation removes irregularities associated with human cognitive limits. Their system attempts to reintroduce structured variation and reports closer distributional similarity to human text and gains in a financial stress test.

I would treat the strong financial result cautiously because it comes from one specialized evaluation. The broader idea is still worth testing: uniform fluency can remove signals that matter to downstream use.

Ratios and task type matter

Characterizing Model Behavior Under Synthetic Data Training trains models from 410 million to 12 billion parameters with synthetic proportions from zero to 50 percent. Performance remained stable up to about 20 percent in the reported settings, then declined faster beyond 30 percent. Larger models tolerated more synthetic data. Calibration worsened before accuracy, and reasoning tasks degraded faster than retrieval tasks.

Calibration may therefore provide an earlier warning than benchmark accuracy. A model can retain the right average answer while becoming less accurate about its own uncertainty.

Language Generation with Replay provides a learning-theory account of generated text returning to later training data. The authors show that replay can be harmless under a strong uniform learning condition and harmful under weaker conditions. Their positive results correspond to practical controls such as cleaning, watermarking, and filtering, while their negative results define cases where those controls remain insufficient.

The paper does not predict inevitable collapse of internet-trained models. It defines conditions under which repeated generated data changes what can be learned.

Synthetic generation can preserve source information

Generating Pretraining Tokens from Organic Data starts with limited human material and creates faithful rephrasings and reformatted versions. The generators receive rewards for quality, fidelity, and whether the examples teach material the target model has not yet learned. In 400-million and 1.1-billion-parameter models, the method produced 3.7 to 5.2 times the effective value of simply repeating the original tokens.

This use differs materially from ungrounded self-training. The generated examples reorganize existing information and remain checked against the organic source.

My standard for synthetic data

I would document the original information source, generator, selection method, verifier, synthetic-to-external ratio, and number of recursive generations. I would measure calibration and output diversity as well as task accuracy. I would also keep a fixed human or environmental evaluation set that the generation process cannot alter.

For subjective tasks, I would use several evaluation criteria and inspect which valid outputs each criterion removes. For tasks with executable answers, I would prefer direct verification over model preference. For source transformations, I would test factual fidelity to the original material.

Synthetic data can extend limited data, target difficult cases, and reduce collection cost. It can also reproduce the generator’s limitations at increasing scale. The deciding factors are verification, variation, external information, and transparent measurement.