Topic: Inference

The AI industry is shifting from training large language models to "inference," which involves deploying models at the "edge" for business operations. Dan Armada, CEO of Armada, describes his company as a "hyperscaler for the edge," building modular data centers. Market trends show high spot prices for Nvidia's H100 and H200 chips, which are optimized for these inference tasks rather than initial training.

Industry insights from a data center developer suggest a significant downturn in the AI infrastructure market, with companies like Microsoft reportedly canceling contracts. The emergence of the Chinese DeepSeek model has shifted expectations toward cheaper training methods, moving the industry focus from remote training centers to low-latency "inference" hubs. Many struggling data centers are being repurposed for Bitcoin mining, while major firms like KKR and BlackRock have already secured exits from these investments.

AI companies are reportedly hitting a "data plateau" as they run out of high-quality internet data to train large language models. The industry is shifting focus from "pre-training" to "inference," which requires different types of chips and could open the door for competitors to Nvidia. This shift is also driving a massive demand for data centers located near high-capacity power transformers.