Gemma4 26B QAT double calling tools?
Just something I've noticed in my harnesses. It likes to call tools twice. I'm wondering if anyone has noticed anything similar with it, and if so,…
Latest
LokalBot: macOS app to supercharge your work with local LLMs on device
This app can replace Granola, Wispr Flow and Cotypist for free if you can spare a bit of RAM to run the models on device. I…
Llama.cpp PR 8% speed boost
Llama.cpp currently uses cpu based sampling for user with mtp enabled. The PR moves sampling to the gpu, which on a 5090 boasts an 8% increase…
DeepSeek V4 Flash 0731GGUFs with updated template (supports reasoning levels)
submitted by /u/tarruda [link] [comments]
Why are Chinese models better* at Frontend than the western top labs?
I use A LOT both openAI and Anthropic products. When I need some frontend work (pure web dev) (or answer that feel less verbose and more…
DeepSeek v4 Flash 0731 4bit ~50tps prefill, ~1tps decode on M5 Air 32gb
Currently running some experiments using the streamed experts trick that's been floating around this sub as well as some of my own trickery to get prefill…
Probably the best way to run DS4 flash on a mac right now (192gb+ vram)
Found this quant, so thought I would share, since its the best I've found so far for running on my mac (m3 ultra). It's got dspark/mtp…
Qwen3.8-Max just took #2 in Vision Arena, trailing Claude by only 13 points. Open-source vision is getting crazy close.
submitted by /u/kevin_cn_ai [link] [comments]
Optimised DSv4-Flash for 2x GH200: 10,000 tok/s PP, >300 tok/s TG on SGLang
There are some PRs to use and a nice trick to speed up PP on really longs contexts in my write up. Hope it helps! TL;DR:…
Time to finally migrate from LM Studio -> llama.cpp, your experience?
Has anyone moved from LM Studio to llama.cpp? What was your experience like? What did you have to learn in order to recreate your experience? Which…