Topic: Model Collapse

The program's "Value for Value" model is explained, where listeners contribute time, talent, or treasure. The hosts review recent AI-generated artwork submitted by producers, noting signs of "model collapse" and "muddy" images. They provide tips for artists on using Photoshop to enhance their submissions and maintain the quality of the No Agenda Art Generator.

A critique of listener-submitted AI art focuses on the repetitive use of the color orange and cartoonish styles, which the hosts suggest is a sign of "model collapse." They speculate that AI models might be storing metadata in specific colors like orange, leading to a lack of original or exciting visual content from "prompt jockeys."

The hosts review listener-submitted artwork for the episode, noting a perceived decline in quality due to the widespread use of AI tools. They discuss the concept of "model collapse," where AI systems begin to degrade by training on their own output, leading to repetitive and "cartoony" styles. Specific pieces by artists like Digital 2112 Man and Scaramanga are analyzed for their creative concepts despite the limitations of AI generation.

A discussion on the "DH Unplugged" podcast regarding AI-generated art suggested that the technology is experiencing "entropy" or "model collapse" as it ingests its own previous outputs. This phenomenon results in muddier colors, a lack of true whites or blacks, and a general loss of luminosity in generated images. The hosts noted that many "prompt jockeys" are resorting to cartoonish styles to mask these technical deficiencies as the AI models become increasingly repetitive.

The intentional flooding of the internet with AI-generated "slop" is proposed as a method to cause model collapse and bankrupt AI companies. High operational costs, such as Anthropic's reported 35 cents per API call, make the industry vulnerable to energy and compute shortages. While companies like Meta release open-source models like Llama to compete, the proliferation of low-quality AI content threatens the integrity of future training data.

The concept of entropy, or the state of disorder and deterioration in a system, is being applied to the current state of the internet and AI development. Google's Notebook LM is cited as an example of how AI models may face "model collapse" if they are trained primarily on AI-generated content rather than human sources. This deterioration is compared to the decline of Google Search quality.

A demonstration of AI "entropy" was conducted by feeding a transcript of a previous *No Agenda* episode into Notebook LM's "Deep Dive" podcast generator. When the resulting AI podcast was fed back into the system to generate a second iteration, the AI began to hallucinate, falsely claiming that vintage Walkmans were exploding in Lebanon. This illustrates the rapid degradation of accuracy when AI consumes its own output.

A study published in Nature magazine reveals that AI models fed on AI-generated data quickly "collapse" into nonsense. Researchers found that by the ninth generation of training on synthetic text, the output becomes incoherent. This "model collapse" theory suggests that the cost of training models will increase while quality decreases if they cannot access original human-generated content.

A segment from NPR's Marketplace explains the concept of "model collapse," where AI systems trained on AI-generated content become increasingly deranged over generations. This recursive feedback loop is compared to photocopying a photocopy until the original image is unrecognizable. The only proposed solution is hiring humans to write original "prose" to freshen the training data.