artificial intelligence collapse risks

AI training of AI in LLMs may result in model collapse, researchers suggest

A study published in Nature warns that using AI-generated datasets to train subsequent machine learning models may lead to model collapse, polluting their outputs. The research indicates that, after a few generations, original content is supplanted by unrelated gibberish, underscoring the necessity of reliable data for AI training.

Generative AI tools, including Large Language Models (LLMs), have gained widespread popularity, primarily being trained on human-generated inputs. However, as these AI models become more prevalent on the internet, there is a risk of computer-generated content being used to train other AI models, or even themselves, in a recursive manner.

Ilia Shumailov and his team have developed mathematical models to illustrate the phenomenon of model collapse in AI systems. Their research shows that AI models may disregard certain outputs, such as infrequent lines of text in training data, leading to self-training on a limited subset of the dataset.

Shumailov and his team examined the responses of AI models to training datasets primarily generated by artificial intelligence. Their findings reveal that using AI-generated data leads to a degradation in learning capabilities over successive generations, culminating in model collapse.

The majority of recursively trained language models analyzed showed a pattern of generating repetitive phrases. As an example, when medieval architecture text was used as the initial input, the ninth generation's output consisted of a list of jackrabbits.

According to the authors, model collapse is an inevitable result of using training datasets produced by earlier generations of AI models. They suggest that successful training with AI-generated data is possible if stringent data filtering measures are implemented.

Simultaneously, firms leveraging human-produced content for AI training could develop models that outperform those of their rivals.

Further detail: In the paper 'AI Models Collapse When Trained on Recursively Generated Data,' Ilia Shumailov et al., Nature, 2024.

Source: