Fastest speculative decoding for qwen
Hey guys, we’ve built fastest speculative decoding for Qwen at least. To run in sglang you can use our fork Hf link Would appreciate your feedback…
Communities 6954 stories
FCC Approves Test of Space Mirror to Light Night Sky Despite Outcry
Article URL: https://www.nytimes.com/2026/07/10/climate/fcc-space-mirror.html Comments URL: https://news.ycombinator.com/item?id=48871367 Points: 34 # Comments: 20
Boko Haram uses ChatGPT and Claude, which local models do you think would lead BokoHaramBench?
submitted by /u/JayoTree [link] [comments]
The U.S. tech industry is increasingly anxious about the rising power and competitive price of open-source AI models from China — and whether the Trump administration will respond with yet another executive order | Politico
Politico: Wall Street’s new obsession: Which CEOs have Trump’s ear?: https://www.politico.com/newsletters/politico-influence/2026/07/10/wall-streets-new-obsession-which-ceos-have-trumps-ear-00993324 submitted by /u/Nunki08 [link] [comments]
Withdraw from ACL ARR and resubmit to a workshop? [D]
Hey guys, I received mediocre scores for my EMNLP paper during the May ACL ARR cycle: 2.5/3, 3/4, 2.5/4. The paper is in the Interpretability track.…
Your code is fast – if you're lucky
Article URL: https://tiki.li/blog/lucky_code.html Comments URL: https://news.ycombinator.com/item?id=48870799 Points: 13 # Comments: 0
Show HN: Richest people in the world by wealth creation instead of ownership
Article URL: https://anti-forbes-list.vercel.app Comments URL: https://news.ycombinator.com/item?id=48870472 Points: 14 # Comments: 4
A font that humans can read but AI cannot
Article URL: https://www.mixfont.com/ghost-font Comments URL: https://news.ycombinator.com/item?id=48870381 Points: 51 # Comments: 48
Almost $1B Later, the US Still Can't Make a Medical Glove
Article URL: https://www.bloomberg.com/news/features/2026-07-07/why-it-s-so-difficult-to-produce-100-american-made-medical-gloves Comments URL: https://news.ycombinator.com/item?id=48870258 Points: 11 # Comments: 2
Companies are scrambling to curtail soaring AI costs
Article URL: https://www.economist.com/business/2026/06/14/companies-are-scrambling-to-curtail-soaring-ai-costs Comments URL: https://news.ycombinator.com/item?id=48870035 Points: 19 # Comments: 6
Running Qwen3 30B A3B at 50 tok/s on RTX 5060 Ti
Experimented with some custom CUDA and C++ code that can now run a Qwen3-30B-A3B at 50-54 tok/s at float 8 on an RTX 5060 Ti with…
Predicting human preference for generated image pairs using HPSv3 [P]
Hey! I'm looking for ways to predict human preference for a project I'm building. (imagebench.ai) I've tryed HPSv3, https://github.com/MizzenAI/HPSv3 and made post about it here: https://imagebench.ai/blog/does-the-score-match-your-eye…
Can we reconstruct a closed-source LLM tokenizer using only two oracles from the chat API?
From the chat APIs alone, we can extract two useful oracles: 1. Token length oracle: Given any string s, return len(tokenize(s)). Prefix token oracle: Given a…
What's the best way to do authentication in modern applications
Article URL: https://neciudan.dev/most-secure-way-to-store-auth-token Comments URL: https://news.ycombinator.com/item?id=48869243 Points: 3 # Comments: 0
How can i limit reasoning effort on the qwen3.5 and gemma4 models?
So, umhh, I am working on an agentic coding platform, and I need to make qwen3.5 and gemma4 models out of controlled reasoning chains. For example,…
Qwen3.6 35B-A3B (Q8_0, no KV quant) single prompt in opencode: "Create a beautiful, relaxing flight simulator in a single html file with mountains, clouds, and endless procedural terrain"
Told it that in plan mode, then told it to implement with no changes to the plan. This model punches far, far above its weight. I…
The Vintage Beauty of Soviet Control Rooms
Article URL: https://designyoutrust.com/2018/01/vintage-beauty-soviet-control-rooms/ Comments URL: https://news.ycombinator.com/item?id=48868996 Points: 18 # Comments: 3
Measuring Is Not Enough Anymore
Is measuring general AI capabilities a good strategy to reduce AI existential risk, compared to other strategies? A disclaimer upfront: To answer that, one organization in…
Some testing on RTX Pro 4500 (With Oculink) on PrismaQuant, INT4 Autoround and NVFP4 W4A4 quantized model
This little beast has been around for a while after asking about whether it's possible to set up in this subreddit. Beelink SER 8 8745 HS,…
Grok Build CLI uploads your whole repo — full git history + .env secrets — to xAI’s cloud, and the opt-out doesn’t stop it (wire-captured)
I ran Grok Build CLI (v0.2.93) through mitmproxy. It uploads your entire repo as a git bundle (full history) to xAI's Google Cloud — independent of…
Don’t bring an AI detector to a deepfake fight: proving reality through multimodal provenance
Epistemic status: confident on the framing, speculative on the implementationTL;DR: Fake media detectors are on the losing end of an arms race. Instead of trying to…
A Simple Model of AI "Psychosis"
"AI Psychosis" has gone from an evocative term for people undergoing extreme delusions, sometimes even culminating in suicide, to the now colloquial insult for anyone who's…
Benchmark of the new unsloth/Qwen3.6-27B-NVFP4 on 4x 5060 ti’s with P2P and PP=4 at 1,4,8,12, and 16 concurrency.
Been trying to troubleshoot prefill around when I saw the newer (supposibly faster) nvfp4 quant. For those curious of prefill issues on 4x5060 ti's, hopefully this…
The Termination Circuit (how reasoning models stop thinking).
Reasoning models since the dawn of o1 and R1 have a tendency to overthink. Despite a lot of work on early-exit methods and steering, open-weight and…