Unsloth has uploaded several sizes of Deepseek-V4-Flash GGUF’s
submitted by /u/ForsookComparison [link] [comments]
Latest
I built an Android app for Qwen3 TTS on-device
It's more of a showcase demo at the moment, but Qwen3TTS 0.6B runs usable fast in Q4. On my Galaxy S25 I get around 0.5 realtime…
I built a tiny proxy that gives GLM 5.2 vision (or any text LLM) – MIT
VisionBridge lets you give text-only LLMs vision. It's tiny OpenAI-compatible proxy that lets reasoning models (DeepSeek, Qwen, GLM…) see images by querying a separate vision model…
GLM-5.2 on 8xB200: the deployment math nobody spells out – NVFP4 + 2x TP=4 replicas should beat TP=8 by ~2x. Full config guidance inside.
We have 8xB200 nodes and users keep asking us how to serve GLM-5.2 on them. Our engineering team went through everything published so far, and the…
Anthropic Research – "Verbalizable Representations Form a Global Workspace in Language Models"
submitted by /u/cuolong [link] [comments]
local already feels good enough
This is specifically for coding, technical planning, and hardware setup. _____ The only times Qwen 3.6 35B A3B has let me down, it has been something…
Gepard : 0.6B streaming TTS built for real-time dialogue – 20× realtime factor, ~50ms time-to-first-audio, vLLM-native, Apache 2.0
We just open-sourced Gepard 1.0, a TTS model built for real-time conversation. It’s streaming-first: audio starts the moment text arrives, generated frame by frame instead of…
I tested freshly merged DFlash in llama.cpp on Qwen 3.6 27B Local AI win. 4.44x faster at 36K context. Here are my findings RTX 6000 PRO.
Hey guys, A month ago I posted my MTP benchmarks here (3.34x on Gemma 4). DFlash support just merged into llama.cpp (PR #22105), so I ran…
Qwen3.6-27B – Effect of KV quantization on KLD – Q8, Q6, Q5 (bartowski)
Lower is better - Quantization increases from right to left I recently made a post here about how I squeezed more context into a Q8 model…
How are people hosting random GGUF / open models behind an API?
I keep running into this annoying gap: A model exists on Hugging Face. Sometimes it has GGUFs. It runs fine locally in Ollama / llama.cpp /…