Coding with LLMs (Claude Code, OpenAI Codex) is often presented as the ‘killer app’ for Generative AI. But looking at data, it seems the one piece of the puzzle missing is actual cost. …
Both Uber and Spotify (and AWS too) had economics of scale going for them - the more users they have, the more the infrastructure could be leveraged. This does NOT work for LLMs. More users means using more compute, more advanced tasks (like coding) uses exponential amounts of compute. A single user running a complex task can make 8 Blackwell GPUs run full tilt, and you don’t even have any guarantee that the output will be useable.
There are a few narrow areas where LLMs might be successful, like scanning for security vulnerabilities or searching large amounts of documents. The massive amount of money invested will never be recouped with these usage scenarios.
I don’t think anyone is assuming it will stay at its current efficiency and there will be zero improvements. A lot of the everyday AI use cases will likely be pushed to someone’s personal device aka your phone. In the same way a lot of Uber and Spotify is handled by your personal device today. What we’ve seen for years now is the development of these gargantuan models that are then condensed down into much smaller models with 90%+ of the same effectiveness. Simultaneously we will see and are seeing devices sold with better NPU’s for edge compute for AI the same we’ve seen the push for more edge compute to manage other services such as Uber and Spotify.
Across this thread and others there’s like this implicit assumption AI will never progress beyond where it is right now in spite of the evidence of its almost exponential growth. It’s really interesting.
Both Uber and Spotify (and AWS too) had economics of scale going for them - the more users they have, the more the infrastructure could be leveraged. This does NOT work for LLMs. More users means using more compute, more advanced tasks (like coding) uses exponential amounts of compute. A single user running a complex task can make 8 Blackwell GPUs run full tilt, and you don’t even have any guarantee that the output will be useable.
There are a few narrow areas where LLMs might be successful, like scanning for security vulnerabilities or searching large amounts of documents. The massive amount of money invested will never be recouped with these usage scenarios.
I don’t think anyone is assuming it will stay at its current efficiency and there will be zero improvements. A lot of the everyday AI use cases will likely be pushed to someone’s personal device aka your phone. In the same way a lot of Uber and Spotify is handled by your personal device today. What we’ve seen for years now is the development of these gargantuan models that are then condensed down into much smaller models with 90%+ of the same effectiveness. Simultaneously we will see and are seeing devices sold with better NPU’s for edge compute for AI the same we’ve seen the push for more edge compute to manage other services such as Uber and Spotify.
Across this thread and others there’s like this implicit assumption AI will never progress beyond where it is right now in spite of the evidence of its almost exponential growth. It’s really interesting.