r/LocalLLaMA
· Communities
ggml-cpu/ops: vectorize flash-attention V-cache F16 to F32 conversion by jinzihao · Pull Request #26947 · ggml-org/llama.cpp
Overview ggml_cpu_fp16_to_fp32 leverages hardware F16C intrinsics (AVX-512, AVX2, etc.), faster than the software-only ggml_fp16_to_fp32_row, bringing 17-31% gain in prompt processing rate for a smaller model like qwen3:4b. Wish the PR had few additional models(recent ones like Qwen3.5/3.6 & Gemma-4 models) with t/s st