Running GLM 5.2 on 4xGB10 with a 100G Switch, 330k ctx, ~25 t/s tg, ~650 t/s pp
TP4+DCP2 for a ~360k kV pool. Prefill increases to 900-1000 t/s with longer prompts. You can also run DCP4 for 660k, but prefill gets shaved to…
Latest
HF Viewer tons of new features!
I'm glad to announce that we have completely revamped the site, so that you can now search the over 2000 viewable models instantly! You can also…
Seasonic PSU calculator now mentions RTX 5080 SUPER (24GB), RTX 5070 Ti SUPER (24GB) and RTX 5070 SUPER (18GB)
submitted by /u/panchovix [link] [comments]
Qwen3.6-27b does not understand software architechure.
Been using this for real software development for a commercial app. i.e. Not a single file HTML app. I mean a large scale 100k+ loc project…
How big of a model do you guys think google overview model is?
It's perhaps the model AI model in the world right know, and I just wanted to understand, if you are serving a billion plus requests a…
The standard free ChatGPT LLM you get after a few messages HAS to be some sub-20b model with online search enabled, no other way to explain how awful it is
Recently bought into the local LLM hype by buying a 32gb vram gpu and holy shit, gemma 4 31b at 5bits blows the standard ChatGPT model…
QLLM, no transformer, no mamba and new noval architecture with O(1) inference is finally out as model
okay so you might be following me or not.. but I have been working in AI since last 10+ years and our first product in AI…
llama.cpp fix for DeepSeek V4 Flash crash and stall
Hey all, I've been working with DSV4 on my Frankenstein rig (5-GPU box: 2x3090 + 5060 Ti + 2x4060 Ti, 96 GiB VRAM, 125 GiB DDR4)…
Tess-4-27B by Migel Tissera
submitted by /u/beneath_steel_sky [link] [comments]
Ternary Bonsai 1.58-bit models – ggml: add Q2_0 quantization support (CPU) by khosravipasha · Pull Request #24448 · ggml-org/llama.cpp
This PR adds Q2_0 support for CPU. Main motivation is to support Ternary Bonsai models (1.7B, 4B, 8B) and upcoming models. This PR is CPU only…