r/LocalLLaMA
· Communities
300b on 32gb MoE-streaming findings + optimisations
The past week I've been running DSv4 inference on my laptop by keeping everything RAM-resident except the MXFP4-experts (since expert pool is ~147GB and won't fit) TL;DR - read speed is the limiter more than the kernels; repacking to enable sequential reads rather than random reads works, pipelining to hide reads behin