Ai data poisoning magnified; Solvency is key.

An AI can unintentionally swallow a poison pill of its own making to induce model collapse is the conclusion of a new research paper that raises new concerns about AI training and long-term accuracy while highlighting a need for an image classification system that may save AI from itself. Once the poison is consumed, the AI never fully heals. The research paper called “Nepotistically Trained Generative-AI Model Collapse” is authored by Matyas Bohacek of Stanford University and Hany Farid of the University of California, Berkeley. The key conclusion is that when an AI is trained on even small amounts of data of its own creation, generative AI produces distorted images. Given that generative AI text-to-image models are trained in data scrapped from the internet, future data scrapping will likely involve material created by the AI itself. When this happens, the researchers discovered, it’s as if the AI has consumed a poison pill. The researchers found that when an AI is trained on its own images, the result, while initially yielding a small improvement in quality, quickly turned into a highly distorted picture, a process they call “model poisoning.” The model collapse persisted even when the self-generated images the AI inadvertently ”retrained” on were as little as 3 percent. The researchers found that “the popular open-source model Stable Diffusion (SD) is highly vulnerable to data poisoning.”