- Agents want to use some sort of API to work with zulip, which means they are headless and can't use computers. They offer workarounds, but they all suck. What I actually want is for the agents to live in the codex/claude app and respond into Zulip. Codex seems willing to make this work, Claude does not.
- Figuring out how to allow some semblence of privacy for humans seems important. For example I wouldn't want a conversation about a Christmas present for my wife being visible to my wife. It's easy enough to implement something that has the veneer of privacy but this gets complicated because these agents all effectively have what amounts to root access to Zulip and can just hack their way in to access private messages if prompted. Still mulling over this.. Maybe some sort of device level security is the answer (e.g. agents can live on my laptop and respect my privacy when they do so, but also contribute to zulip?)
- Juggling between agent interrupts and queueing messages also seems conceptually hard in this interface. Luckily agents are pretty good at being interrupted these days. Still, I see that they sometimes forget to check up Zulip messages between when they start a task and finish a task and I imagine this could get troublesome in some cases.
- Monitoring usage and making sure things don't totally break when I exhaust quota on Claude, Codex or wherever else
- Managing orchestration of message receipt across multiple computers that host agents
- Code quality and refactors - I often ask my agents to clean up after themselves at a pretty regular cadence, but I no longer have that optionality as easily when there are so many agents to manage. I kinda think I'll have some agent who uses free quota at the end of each reset cycle to perform refactors. Not sure yet.
It strikes me that maybe I'm overdesigning? Surely one agent can do all of the things I'm asking here, and maybe separate responsibilities are not useful? Or maybe the right thing to do is use some sort of multi-agent harness and let it figure out how to self-organize? But then the organization is more opaque. I really don't know. I'll carry forward with this for awhile and see how things evolve. One really nice thing working with agents like this is that we can mutate this whole framework pretty easily as things progress.