Even supplemented synthetic data causes model collapse.

Garbage in, garbage out. Data pollution is ruining generative AI's future. A recent study by researchers in Canada and the U.K. explained the phenomenon of model collapse. Model collapse occurs when new generative models train on AI-generated content and gradually degenerate as a result. In this scenario, models start to forget the true underlying data distribution, even if the distribution does not change. This means that the models begin to lose information about the less common -- but still important -- aspects of the data. As generations of AI models progress, models start producing increasingly similar and less diverse outputs. Generative AI models need to train on human-produced data to function. When trained on model-generated content, new models exhibit irreversible defects. Their outputs become increasingly "wrong" and homogenous. Researchers found that even in the best learning conditions, model collapse was inevitable. Why is model collapse important? Model collapse is important because generative AI is poised to bring about significant change in digital content. More and more online communications are being partially or completely generated using AI tools. Generally, this phenomenon has the potential to create data pollution on a large scale. Although creating large quantities of text is more efficient than ever, model collapse states that none of this data will be valuable to train the next generation of AI models. More AI-generated data pollution makes data from human interactions with systems harder to find and more valuable. Companies and platforms with access to human-generated data will be more likely to create high-quality AI-generated models. Companies that were able to scrape the web before AI pollution will have an advantage over those scraping the post-ChatGPT web for quality training data. How does model collapse occur? Model collapse happens when new AI models are trained on generated or synthetic data from older models. The new models become too dependent on patterns in the generated data. Model collapse is based on the principle that generative models are replicating patterns that they have already seen, and there is only so much information that can be pulled from those patterns. In model collapse, probable events are overestimated and improbable events are underestimated. Through repeated generations, probable events poison the data set, and tails shrink. Tails are the improbable but important parts of the data set that help maintain model accuracy and output variance. Over generations, models compound errors and more drastically misinterpret data. Researchers define two types of model collapse: early and late. In early model collapse, the model begins to lose information about probability tails. In late model collapse, the model blends together what should be distinct patterns in the data. Eventually, the outputs become increasingly similar to each other with little resemblance to the original data. The previously mentioned study, "The Curse of Recursion: Training on Generated Data Makes Models Forget," tested three AI model types by repeatedly feeding them model-generated data. In all three cases, the researchers found instances of model collapse: Gaussian mixture model (GMM). A GMM is designed to separate data into clusters using a Gaussian distribution. Within 50 re-generations, the data distribution completely changed. By generation 2,000, there was no longer any variance in the data. Variational autoencoder (VAE). The VAE was trained on real data and used to generate images of handwritten digits. The next generations were trained on model-generated data. As the generations progressed, the images got progressively blurrier until each digit resembled a roughly uniform smudge.