Topic: Data Models

AI companies are reportedly hitting a "data plateau" as they run out of high-quality internet data to train large language models. The industry is shifting focus from "pre-training" to "inference," which requires different types of chips and could open the door for competitors to Nvidia. This shift is also driving a massive demand for data centers located near high-capacity power transformers.

A study published in Nature magazine reveals that AI models fed on AI-generated data quickly "collapse" into nonsense. Researchers found that by the ninth generation of training on synthetic text, the output becomes incoherent. This "model collapse" theory suggests that the cost of training models will increase while quality decreases if they cannot access original human-generated content.

A segment from NPR's Marketplace explains the concept of "model collapse," where AI systems trained on AI-generated content become increasingly deranged over generations. This recursive feedback loop is compared to photocopying a photocopy until the original image is unrecognizable. The only proposed solution is hiring humans to write original "prose" to freshen the training data.

Meta's release of the "Segment Anything Model" (SAM) for object identification in images and videos is discussed. The hosts debunk the "magic" of the AI by explaining the underlying labor involved, where thousands of workers in the Philippines and India perform manual data labeling for micropayments. They characterize the hype surrounding computer vision as a waste of corporate resources that relies on human-in-the-loop training rather than true autonomous intelligence.

Critics challenge the validity of COVID-19 data models and the shifting death toll projections provided by Dr. Fauci and Dr. Birx. A rhythmic parody focuses on the repetitive advice to "wash your hands" more often than dreamed possible. These public service announcements specifically target "communities of color" and urge citizens to step up for their elderly relatives.

A discussion on climate forecasting reveals that of the 32 families of computer models used by the UN, only the Russian model accurately tracks real-world temperature data. The other models are criticized for being "parameterized" or "fudged" to predict excessive warming. Experts explain that these models were tuned to simulate early 20th-century warming that occurred before significant CO2 increases, leading to systematic errors in current predictions.

Dr. Anthony Fauci defended the use of computer models in pandemic planning while acknowledging they are an "imperfect science." He argued that the decision to implement strong mitigation in the U.S. was based more on the observed reality in China and Northern Italy than on specific numerical projections.

Fred Van Leeuwen from Hilversum, Netherlands, donated while expressing skepticism regarding both political leaders and scientific data models. He argued that the failure of COVID-19 models justifies a more critical view of climate change projections.

A discussion on the inherent flaws of computer modeling references a past C-SPAN lecture by a modeling expert who warned against using such tools for social or environmental policy. The hosts argue that models used for both climate change and pandemic projections are frequently inaccurate and should not be used to drive public fear or policy decisions.

A comparison of weather forecasting technology reveals that the European model is often considered superior to American models due to higher resolution and better supercomputing power. The hosts question why American meteorological technology has seemingly lagged behind since the 1980s. The segment concludes with a brief critique of how news algorithms and supercomputers are used to influence public opinion and maintain a state of fear.