Deepseek-V4-Flash-0731 Dwarfstar on Mac
Here is the prefill performance in an M2 Ultra with 192GB of RAM. For decode, at the following depth: Start: 28 t/s 45k: 23.5 t/s 192k:…
Latest
What are the best ways for managing context across a massive project?
I have a bunch of PDFs, word docs, and excel files that relate to a project of interest. I am trying to figure out what is…
my tps is suddenly halved and I do not know why.
previously on my 3070, 32gb ddr4 and i711700 I used this command for months and got 26-30 tps: "C:Program Filesllama cppllama-server.exe" ^ -m "C:Program Filesllama cppmodelsQwen3.6-35B-A3B-UD-Q4_K_XL.gguf"…
Conclusion: r/LocalLLaMA still has brilliant open-weight research, but finding it requires wading through endless benchmark drama, non-local Discussion Points and repetitive hardware flexes.
I let Gemma4-31b run on my laptop for like almost a day using a heavily altered pi to do a deep dive on our beloved Llama…
It’s more likely I’m stupid than it’s a great conspiracy but…
How is it possible for such an active group like Unsloth to quantize so many models, and yet Hy3, which came out at the start of…
Best model <3B for multilingual understanding/ instruction following?
I know qwen 3.5 4b is great but a bit too large and miniPCM5 1b is great for agentic use but not so great for multilingual…
Has Qwen 3.8 has dropped yet? Day 90…
submitted by /u/mailto_devnull [link] [comments]
llama.cpp just added MTP / DSpark support for DeepSeek V4 Flash
submitted by /u/rmhubbert [link] [comments]
Vacuum 16T
https://huggingface.co/tsfrm/vacuum-16t A 16.5-trillion-parameter model that contains nothing. This model is just a ████ you to the labs and companies who say that "haha I have the…
How do you test your setup?
We all have been there, tinkering around with models is fun but we rarely do it with research precision and issues are often subtle and hard…