• 0 Posts
  • 4 Comments
Joined 3 years ago
cake
Cake day: June 12th, 2023

help-circle
  • While this advice is true for all models, when it comes to agentic tasks (add this small feature/write this test harness/find bugs/suggest improvements), open source models are still way behind, vibe code or not.

    Claude Fable or even Opus in an editor like Zed have a 1 million token context window and will “think” through the goals of the application, test their changes, work through debugging processes the way a programmer would, stop to ask for clarification, check diagnostic tools and linters, prompt to run test code, etc.

    Llama, Gemma and Qwen etc. Do lack a lot of the world knowledge to get the goals of the application, but they also just don’t have the debugging skills, won’t test their code, don’t always tool call correctly, get confused as the context increases and nobody has enough vram to run on large context sizes locally.

    They can do autocomplete on small functions but aren’t really there for more complex tasks.

    On top of that, the biggest problem is that the best open source models are trained and released by the same giant tech conglomerates that have an interest in not competing with their own products. Qwen is Alibaba, Llama is Meta, gpt-oss is OpenAI. Even the more “independent” ones, kimi (Moonshot) and GLM (z.ai) are mostly funded by Alibaba and Tencent. They’re released for research and marketing purposes and to please their corporate backers with inflated stock. Almost nobody has the resources to train new models from scratch. People make lots of merges and fine tunes but AI is not democratised the way that traditional programming tools have been.

    Maybe some day there will be enough cheap compute for open source communities to pool together resources to build competing models but they’re not really there yet :(


  • That’s fair. The nuance that people lose is more that people are often painting them all with the same brush. Protesting any datacenter regardless of impact.

    It becomes something like: “datacenters are evil and are a symbol of techno fascist distopia! If they build a datacenter in my city, the taps will run dry and Elon Musk will use it to make ai porn of my children!” Even if it’s a small solar powered closed loop that provides VPS, storage and web hosting for nerds and small businesses.

    I also do think there’s also a scale of evil there. Some environmental impacts are not immediately obvious and might not be known about during planning. Some were built a long time ago with older tech and are a bit shitty but have a plan to transition to be more sustainable, etc.

    The world is full of “alright but a little bit shit.” It’s not all perfect angels and mustache twirling villains.

    I don’t want to detract too much from the real villains though. Nobody needs a 9GW datacity for military ai.


  • Not all data centres are evil and the issue is nuanced. This one sounds pretty evil though.

    9GW is totally insane and they’re building a gas plant for it instead of renewables (although there’s some solar too). It’s closed loop so the water use fears once it’s running are probably a bit overblown, but the construction itself is going to be ecologically insane. The thing is basically a data city, 162 square km is even larger than a lot of cities and involves building an entire power plant and new energy infrastructure. Building it is a full megaproject and even just noise pollution and the construction impacts will mess with bird migration etc. Obviously the whole thing isn’t going to be full of data centre, some of that space is empty but still.

    It’s also going to have the US military as a major client so… Pretty high up there on the evil scale IMO.