r/LocalLLaMA
· Communities
Llama-CPP Parallel Agents –> fine for decode, but one agent’s prefill will grind all other agents to a halt
Testing with 3-5 agents. Decode performance is superb, however if one performs a web search and needs to process a few thousand tokens, ALL other agents will grind to a halt: I've tried tuning a little bit, but no luck. example command of mine (this server is ONLY used for the sub-agents): ./llama-server --model /mod