r/LocalLLaMA
· Communities
What are people using for multi-model backends? What about swapping configs?
I am trying to plan and deploy a machine that serves models for coding, Hermes, and whatever else. It's got multiple GPUs in it, and I want the flexibility to run different configurations (i.e. I might want to run two smaller models when I'm using Hermes and doing some less-intensive coding, swap to one big model acros