Deepseek and Agent Coordination

There's a bit of catching up to do here, which I'll dole out over the next week little by little. First of all - an objective I have is to build a cost-effective way of serving a high quality LLM. Searching around, the obvious option is Kimi K3.1, which can be run on 16 sparks (which would cost ~$100k) - but this would be quite slow. I think I'll have to shelf that for the time being in hopes that someday, somehow, compute gets a little cheaper and I can pick up a 8xB300 node or something. Probably pipe dreams.. :(

So for now I'll fall back to a smaller, but still supposedly great model - DeepSeek Flash 4.1. This can run on 4 sparks. I dutifully plopped down the cash for the hardware, got it in a few days and had Claude set everything up for me. I'm pretty happy with the initial benchmarks - 80toks/s single request and up to 252toks/s at a batch size of 16. That's definitely enough to get some work done for the time being.

One immediate goal I have is to have allow multiple instances of codex/claude code/deepseek to work together within my home network. I started down the path of having Claude build an "agent forum" for user-directed agent collaboration. After some thinking today I decided that probably the better approach is to just work inside of a chat app like Slack. There's something called "Zulip" which I can deploy locally on one of my servers. The vision is that I'll let all of my agents collaborate with each other and my family over Zulip. This should enable me to mux together multiple computers on the network, as well as share some LLM accounts with my family members. Working on that now.

It's pretty nice to be booting up my old hardware again. I like the physicality of getting my hands on these computers. For now. :)