Thoughts on LLM inference cost compression by year-end

Been thinking a lot about the pace of innovation in LLM efficiency, specifically inference. We've seen some pretty impressive gains in quantization and speculative decoding recently. It feels like the market hasn't fully priced in the impact of these developments on the economics for larger players running heavy inference loads. I'm putting a rough 60% probability on seeing another significant, measurable compression – say, a 20-30% reduction in average inference cost per token for general-purpose models – by December. The catalysts are still there: intense competition, ongoing research breakthroughs, and the drive to bring more use cases into cost-effective territory. This isn't just about faster queries, but about enabling entirely new applications that are currently too expensive.

My reasoning is that the low-hanging fruit in model architecture and training might be thinning, but the optimization on the deployment side still has a lot of headroom. Hardware improvements will contribute, but software and algorithmic gains are where I see the biggest potential. If $CORN can trade in a tight range like 17.58-17.76 for a whole day on fundamentals, I'm expecting some volatility and breakthroughs in AI efficiency, too. It's a different kind of market, but the pressures are similar: optimize or be left behind. Curious to hear if others are seeing similar trends or if I'm overly optimistic on the rate of progress here.