r/LocalLLaMA
· Communities
DSv4-0731 on 32gb RAM – 70tok/s prefill, 1.5tok/s decode
The past week I've been running DSv4 inference on my laptop by keeping everything RAM-resident except the MXFP4-experts (since expert pool is ~147GB and won't fit) TL;DR - read speed is the limiter more than the kernels; repacking to enable sequential reads rather than random reads works, pipelining to hide reads behin