Longcat 2 model weights have been published
https://huggingface.co/meituan-longcat/LongCat-2.0-INT8 https://huggingface.co/meituan-longcat/LongCat-2.0-FP8 submitted by /u/RhubarbSimilar1683 [link] [comments]
Latest
Dario’s favorite model
found this here https://huggingface.co/ideogram-ai/ideogram-4-fp8/discussions/3#6a2070c21bfe5300af2887b2 submitted by /u/po_stulate [link] [comments]
Need some help figuring which quant sizes of some of the big MoEs would fit properly (with how much context + unquantized KV) on a future 256GB or 512GB dram + 48GB VRAM rig I might build later on, since I want to download and save them now (not later on when I have the rig)
I plan to download a few of them in full 16-bit safetensors (so I'd be able to turn those into whatever quant size/quant-style I want later…
For RAG specifically, prefill speed matters more than decode and why Strix Halo struggles for interactive use
Seeing a lot of "what hardware for local RAG" threads lately, and the framing that keeps getting missed is: decode tok/s is not the bottleneck for…
Any idea why bartowski claims DeepSeek-V4-Flash is MXFP4?
https://huggingface.co/bartowski/DeepSeek-V4-Flash-GGUF This model is in MXFP4 and as such has only been provided in MXFP4 format! Original model: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash But on original's page none of tensors…
Help using llama.cpp with intel n100?
I have an intel n100 mini pc that's on 24/7 running proxmox. I want to use llama.cpp server with gemma 4 E2B for small tasks. Should…
Qwen 27B
Just a datapoint I wanted to share.Qwen 27b, at q6kxl, with multi-token prediction, on a 4090+3090 system, using lcpp, puts out 50-90 tokens/s decode and 1500-2200…
Mongo with vector search performance
MongoDB has vector search and hybrid capabilities in the community (free, not cloud) build. Has anybody tried it in the real world and seen how it…
The Hugging Bay – model governance has entered its torrent era
The fact that this feels less like a joke and more like a roadmap is the funniest part. Edit: Sorry, it seems the comment function is…
Portugal just released their own LLM Amalia (9B)!
I didnt see any mention here. Source: https://portugal.gov.pt/en/gc25/communication/news/llm-amalia-shows-portugals-potential HF link SFT: https://huggingface.co/amalia-llm/AMALIA-9B-0626-SFT HF link DFO (Direct Preference Optimization): amalia-llm/AMALIA-9B-0626-DPO · Hugging Face Paper: https://arxiv.org/pdf/2603.265