r/LocalLLaMA
· Communities
DeepSeek v4 Flash 0731 4bit ~50tps prefill, ~1tps decode on M5 Air 32gb
Currently running some experiments using the streamed experts trick that's been floating around this sub as well as some of my own trickery to get prefill to run a bit faster. It's been quite a bit of fun so far - just getting a 300b model to run at all on an Air is itself equal parts silly and satisfying Repo isn't in