At some point, if we reach stability on the models, we'll start getting silicon optimized for individual models. They are optimizing for time to market, not efficiency right now. I don't know how much it will move the needle on the cost math, but at this scale any improvement has a crazy multiplier.
But that means going back to "80% profit margin" Jensen and further digging your capex hole. The benefits would have to pay not only for current capex but also past capex.
But by then, I will be able to go one line down in my dropdown menu to switch to a newer LLM provider who doesn't have to amortize those past capex.
Yes, the whole thing feels dot comish. I’m betting there will be a few (maybe only 1-2, like what happened to search) winners, and everyone else is in for a bad time. It’s the same dynamic too: the winners are going to win so big that everyone wants to get their money in for a chance at a piece of the pie.
