How does MTP actually improve performance? Haven’t understood the verification process
Hi there, does anyone know how MTP works? I was trying to wrap my head around it, but things aren't adding up. For example, the input…
Latest
Spiritbuun’s VBR (Variable Bit Rate) KV cache — first impressions
Well, this is an appreciation post on the spiritbuun llama.cpp fork. Some weeks ago I was testing forks and configurations to find which one was the…
[2607.07508] Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
Reinforcement learning (RL) is becoming increasingly important for post-training large language models (LLMs). Previous RL pipelines for LLMs were mostly synchronous and batch-interleaved, which is inefficient…
I RL-trained Qwen3.6-35B-A3B to RL-train small task-specific Qwen models. Fully open source! 🤓
👋 Training my first RL model last year was super fun, now I've RL-trained a model that RL-trains other models... wild times! The agent gets a…
Nemotron-3-Embed 1B/8B
https://huggingface.co/nvidia/Nemotron-3-Embed-8B-BF16 https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16 Nemotron-3-Embed-8B-BF16 is a versatile text embedding model trained by NVIDIA and optimized for retrieval and semantic similarity tasks. It provides strong multilingual and cross-lingual…
Using local computer vision to perform mouse actions via keyboard
Hello, I created this when I was suffering from hand pain due to mouse usage. It runs 100% locally. I've optimized it as best as I…
llama.cpp milestone
Thanks to all contributors for our local inference! submitted by /u/sergeysi [link] [comments]
Good podcasts
I'm heading on vacation soon and want to download a few good podcasts about local LLMs, open-weight models, inference, tooling and the broader open-source AI ecosystem.…
Google DeepMind’s Demis Hassabis calls for U.S.-led global AI watchdog
From Demis Hassabis on Substack: A Framework for Frontier AI and the Dawning of a New Age: https://demishassabis.substack.com/p/a-framework-for-frontier-ai-and-the-dawning-of-a-new-age Article on 𝕏: https://x.com/demishassabis/status/2076957440109625718 Axios: https://www.axios.com/2026/07/14/demis-hassabis-ai-regulati
Pydantic AI structured outputs and evals · coles.codes
Even at temperature zero the same prompt can give you different answers, because inference stacks arent batch-invariant and on a hosted API you dont control the…