Oops. Now that users are being to made to pay something closer to the true cost of AI inference, no one will like using it anymore. Could this be what ultimately sets off the bubble *collapse?
Users are paying way more than the cost of inference. Look up inference prices of high end open weight models vs claude or gpt. Cheaper by an order of magnitude.
It’s the constant training of new models that’s losing them money. New version is out every month.
To be clear, We don’t really know the architecture or model size of claude or chatgpt. If opus is a 1 trillion parameter dense model, then yes of course it’s going to be way more expensive to run than deepseak which is an 862 billion parameter MoE model.
American models have focused on being the best regardless of cost where chinese model makers were forced to focus on efficiency because of lack of access to chips. Which will probably give them an advantage as the free investor money runs out for the american companies.
The constant training is simply how AI works and will continue to work chinese or american. That cost will never go away. Anytime you need your model to learn new skills or gain new knowledge, it needs to be trained.
To clarify, AI companies charging the cost that would make the inference profitable for them, against the operating costs and financing costs on new capital expenditures (new data centres, new compute and new model training*), is more than what most people appear to be willing to pay. That cost is indeed more than just the cost of inference incurred by the AI company.
*(I’m being generous and including model training as capex for the sake of argument, even if I personally think to continue the hypetrain, continuous model improvements are core to AI companies’ operation.)
Oops. Now that users are being to made to pay something closer to the true cost of AI inference, no one will like using it anymore. Could this be what ultimately sets off the bubble *collapse?
Users are paying way more than the cost of inference. Look up inference prices of high end open weight models vs claude or gpt. Cheaper by an order of magnitude.
It’s the constant training of new models that’s losing them money. New version is out every month.
To be clear, We don’t really know the architecture or model size of claude or chatgpt. If opus is a 1 trillion parameter dense model, then yes of course it’s going to be way more expensive to run than deepseak which is an 862 billion parameter MoE model. American models have focused on being the best regardless of cost where chinese model makers were forced to focus on efficiency because of lack of access to chips. Which will probably give them an advantage as the free investor money runs out for the american companies.
The constant training is simply how AI works and will continue to work chinese or american. That cost will never go away. Anytime you need your model to learn new skills or gain new knowledge, it needs to be trained.
To clarify, AI companies charging the cost that would make the inference profitable for them, against the operating costs and financing costs on new capital expenditures (new data centres, new compute and new model training*), is more than what most people appear to be willing to pay. That cost is indeed more than just the cost of inference incurred by the AI company.
*(I’m being generous and including model training as capex for the sake of argument, even if I personally think to continue the hypetrain, continuous model improvements are core to AI companies’ operation.)