r/LocalLLaMA
· Communities
We quantized DeepSeek V4 0731 and benchmarked it against popular quants on 8× RTX 5090
We converted the model from the original safetensors and found two issues. The first one made our quantization fail several times, the second one does not fail at all, it just quietly ruins the base 1) You must use the --no-lazy option, otherwise token_embd.weight will take on the value NaN. 2) By default, the converte