Another good option is Qwen3.5-9B. I’d try a Q4 or Q5.
Whichever model you choose, I’d also suggest a few MCP servers to give it access to reliable information rather than rely entirely on its internal knowledge.
Microsoft Learn MCP: Official PowerShell, Windows and network documentation. Will help with your scripting stuff.
I’d start with Microsoft Learn and Exa, then add the others as needed. You’ll need an MCP-capable client (which pretty much all of them are now - eg: llama.cpp web ui, OpenWebUI, Light Local Chat etc), and I’d keep file access read-only initially.
I’d personally use McProxy instead of adding MCPs singularly, but I fear I’m already over loading you with information.
EDIT: The PolyStrata recommendation assumes you have a discrete GPU with sufficient VRAM (unclear if you do?). If not, disregard Ornith and try Tiny Ling 3 Q6 or Qwen3.5-9B with ik_llama.cpp instead.
Either way, both smaller models should be capable of the tasks you’ve described, particularly with access to the right documentation. Just verify any network or firewall changes before applying them.
Assist with writing letters, home assistant code, light powrshell scripting.
Edit: helping segment my network
That should be well within reach of a local LLM.
If you’ve got a discrete GPU with sufficient VRAM, something like Ornith running on PolyStrata could be worth a shot. Try a Q3 size.
https://huggingface.co/bartowski/Ornith-1.5-35B-A3B-GGUF
https://github.com/VecSzn/PolyStrata
Hell, Tiny Ling 3 might be able to do it. Try a Q6.
https://huggingface.co/bartowski/Ling-3.0-tiny-GGUF
Another good option is Qwen3.5-9B. I’d try a Q4 or Q5.
Whichever model you choose, I’d also suggest a few MCP servers to give it access to reliable information rather than rely entirely on its internal knowledge.
https://learn.microsoft.com/en-us/training/support/mcp
https://github.com/exa-labs/exa-mcp-server
https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem
If you’re actually running Home Assistant, its official MCP server is worth a look too.
https://www.home-assistant.io/integrations/mcp_server/
I’d start with Microsoft Learn and Exa, then add the others as needed. You’ll need an MCP-capable client (which pretty much all of them are now - eg: llama.cpp web ui, OpenWebUI, Light Local Chat etc), and I’d keep file access read-only initially.
I’d personally use McProxy instead of adding MCPs singularly, but I fear I’m already over loading you with information.
EDIT: The PolyStrata recommendation assumes you have a discrete GPU with sufficient VRAM (unclear if you do?). If not, disregard Ornith and try Tiny Ling 3 Q6 or Qwen3.5-9B with ik_llama.cpp instead.
https://github.com/ikawrakow/ik_llama.cpp
Either way, both smaller models should be capable of the tasks you’ve described, particularly with access to the right documentation. Just verify any network or firewall changes before applying them.