r/LocalLLaMA
· Communities
Running Qwen3 30B A3B at 50 tok/s on RTX 5060 Ti
Experimented with some custom CUDA and C++ code that can now run a Qwen3-30B-A3B at 50-54 tok/s at float 8 on an RTX 5060 Ti with only 16 GB of VRAM. This speed is roughly 50% improvement to llama.cpp which runs at around 33-34 tok/s (with n-cpu-moe). These speedups come mostly from combining SOTA solutions I saw in pa