Thoughts on LLM inference cost vs. revenue for Q4
Been thinking a lot about the push towards cheaper inference for LLMs, especially with some of the recent model releases. It feels like the market is underestimating how quickly this will impact the bottom line for some of the key players. We've seen a ton of focus on training costs, but as these models get deployed at scale, inference becomes the real cost driver. I'm putting a 60% probability on at least two major AI-centric companies reporting a significant year-over-year decrease in their blended average inference cost per query and a corresponding increase in inference-related revenue for Q4. The reasoning is largely based on new hardware efficiencies finally coming online and the aggressive competition driving down API pricing, which will force innovation in the underlying cost structure. If they don't show that, it's a red flag for me on their ability to scale profitably.
That's a really interesting point about the shift from training to inference as the main cost. I hadn't thought about it in terms of how quickly it could impact revenue. Are there any specific companies you think are particularly vulnerable or well-positioned?