RTX 4070, 32gb system ram, Linux. NVIDIA-SMI 610.43.03, KMD Version: 610.43.03, CUDA UMD Version: 13.3 Systemd service modifications: [Service] Environment="OLLAMA_HOST=0.0.0.0:11434" Environment="OLLAMA_FLASH_ATTENTION=true" Environment="OLLAMA_KV_CACHE_TYPE=q8_0" Environment="OLLAMA_NUM_PARALLEL=1" Environment="OLLAM
https://preview.redd.it/wbq8ullnbafh1.png?width=1409&format=png&auto=webp&s=e6d2fe2b1c87c724bc64003c25f917dcee53260f I finally got round to tweaking this, with a bit of help from GLM5.2. The trick to getting it running with vLLM (which I couldn't…
In a previous post (https://www.reddit.com/r/LocalLLaMA/comments/1utefpr/running_qwen3_30b_a3b_at_50_toks_on_rtx_5060_ti/) there seemed to be great demand for bringing in Qwen3.5 35B. Some Gated Delta Network kernels later and here it is.…
Forgive me naiveness, I’m a little lost. Why must a model be jack of all trades? Why can’t we have a model expert in a single…
Shoutout to this awesome guy - https://www.reddit.com/r/LLM/s/IDUyU3v9ap Thanks to his project, BigMoeOnEdge https://github.com/Helldez/BigMoeOnEdge, I managed to successfully run a 35B MoE model on just 12GB of…
Just saw this on huggingface: https://huggingface.co/ProCreations/grug-27b The benchmarks claim that it's quite a bit better than qwen3.6 27B original and that they reduced the amount of…
TL;DR it's pretty goddamned fast; 69 tps decode at near max (256k) context with MTP on. prefill numbers went down to 893 at max context with…
RT Nathan LambertNew podcast with @xeophon on all things open models. More on Kimi K3, Qwen 3.8, GLM-5.2, Xi's WAIC speech, distillation, the open-closed Gap, and…
A podcast with Florian Brand.
Laguna-S-2.1 dropped few hours ago and as I am in the market for an upgrade to trusty qwen3.6 dense and the current daily 122B, I ran…
Kind of unexpected. Happy for Gemma-4/Google, big win for us, LocalLLMers. Yet Qwen3.6 still does better in Hermes than Gemma-4 somehow. We need Gemma-4.1 fine-tuned on…
No weights yet. I feel sad for them, that training run cost maybe 100s of thousands of dollars and they didn't even beat GPT-OSS 20B in…
I've been using the GGUF version of Qwen3.6-27B for a while, so when Unsloth released an NVFP4 version I wanted to see what it could do…
Alibaba's Qwen Audio 3.0 TTS Plus tops Artificial Analysis' Speech Arena leaderboard. The model supports 16 languages and lets users control speaking style with natural language…
https://github.com/woct0rdho/transformers5-qwen3.5-recipe It's time for GGUF to replace bitsandbytes as the base model format for low-VRAM LoRA training. It's actively supporting new model types such as MoE,…
I know DavidAU gets a bad rap, and rightfully so. I've tried some of his fine tunes in the past and they have been... Interesting. I…
https://preview.redd.it/wluwwp6s4heh1.png?width=1248&format=png&auto=webp&s=6e95d963645c5a0ef750bf81324a6d4dcbc0389e Spent the last few days measuring speculative decoding on Qwen3.6-27B (dense, NVFP4) on one RTX PRO 6000 Max-Q, comparing vLLM and SGLang across MTP, DFlash,…
Trying to see what I can scrounge together bare minimum hardware requirements to get up to that rough speed. Right now I'm running an RX6600XT and…
IntroductionI started learning about interpretability late February of this year. I’ve been a full stack dev for a non profit for a few years now, developing…
During Preview, Qwen3.8 is getting better by the day. Latest version is live now, with broad gains and a big step up on web frontend. Thank…
RT AiBattle- Qwen and Kimi have both undergone epic-level improvements. Does GLM still stand a chance?- Tang Jie (Founder of http://z.ai): Epic-level PlusReally excited for the…
My earlier post was mistitled as Qwen 3.7 Max Preview. Well, whatever but in that post I compared Qwen 3.8 Max Preview to Minimax M3 and…
Article URL: https://www.emergingtrajectories.com/lh/frontier-lab-economics/ Comments URL: https://news.ycombinator.com/item?id=48980019 Points: 14 # Comments: 4
An example TL;DR I have open-sourced NInfer, a from-scratch C++/CUDA inference engine currently specialized for two exact Qwen3.6 checkpoints on a single RTX 5090. Both the…
Qwen is Alibaba's open-weight model family — Qwen3 (text), Qwen3-VL (vision), Qwen3-Coder, QwQ (reasoning). Qwen3 sits near the top of every open-weight benchmark in 2026 and ships under a permissive licence.
Owner: Alibaba. We have 350 stories indexed for this model, auto-tagged from titles across every tracked source — official announcements, papers, GitHub release notes, and third-party press. The CTA on each card links to the original; the official site is qwen.ai.
Related text models: GPT, Claude, Gemini, Gemma, Llama, Mistral, Grok, DeepSeek.