Question about Quant versus Size.
Sorry if this is asked a lot, but I was wondering if there is any clear winner on the Quantization versus Model Size debate? I can…
Latest
Anyone Used MiniMAx H3 yet? Open Weights are out today!
I am curious if anyone have used it. I would love to feed it key frames and test if it can create in-between frames between my…
You really should not quantize KV Cache for DeepSeek V4 Flash
I don't think anyone should quantize the KV with DS4F. I checked the the quality impact (PPL, KLD, Same TopP) for swhitching from BF16 KV to…
China’s DFSX Offers 2x The Memory Bandwidth Of NVIDIA’s GB200
submitted by /u/MundanePercentage674 [link] [comments]
PSA: llama.app, Mac app and llama serve from llama.cpp
https://llama.app/ Been using llama.cpp for years now and im on here all the time (im a mod..), but somehow I totally missed that llama.app exists and…
https://huggingface.co/poolside/Laguna-S-2.1-NVFP4
Updated release (August 2026). This is a new checkpoint that supersedes the earlier version of this repository. The weights have changed, not only the config, so…
DeepSeek V4 @ IQ3XXS on M1 Ultra 128GB- 16 tok/s in LM Studio after patch
M1 Ultra 128GB, Unsloth UD-IQ3_XXS, wired limit at 120GB. I was at 5-6 tok/s before the patch. Getting 15-16 tok/s now with the patched engine, and…
[Paper] EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation
The EdgeRazor method uses an entropy-guided distillation process to better translate a teacher model's logit probability distributions into the student model's low-bit / mixed-precision hidden-layer features,…
Parlor v2: best-effort fully local GPT-Live clone on an M3 Pro
GPT-Live is so good that I use it almost every day. I've been wanting to replicate it since it was released. My first attempt was to…
Try handling complex tasks to your local models with GraphARC, graph engineering yes !
🚀 We just built our first real-time implementation of Graph Engineering, inspired by our experience building graph tooling used by 4,000+ developers. 🔗 Repo: https://github.com/CodeGraphContext/grapharc Have…