r/LocalLLaMA
· Communities
Llama.cpp PR 8% speed boost
Llama.cpp currently uses cpu based sampling for user with mtp enabled. The PR moves sampling to the gpu, which on a 5090 boasts an 8% increase in tok/s for qwen3.6:35b. I tested it on my P40 and observed a 4% increase inference speed boost. Pretty exciting to see 84 tok/s max on a nvidia p40 for me. Backend sampling sh