r/LocalLLaMA
· Communities
**Your $80 Tesla P100 has been doing silently noisy math in llama.cpp for years. Three lines fix it, for free.**
## TLDR; Shipped — in turboquant v0.3.0, downloadable now. https://github.com/TheTom/llama-cpp-turboquant/releases/tag/tqp-v0.3.0 llama.cpp's CUDA code has a flag that means "this GPU is fast at fp16, so do the math in fp16." The GTX 10-series and P40's (sm_61) were exempted from it long ago. The P100 (sm_60) was not,