Anyone tried using the new (ish) Gemma diffusion model as a speculative model?
It seems that MTP is the gold standard for speed up but still suffers from having to choose between regressive and parallel drafters that come with…
Latest
Mistral released Leanstral-1.5-119B-A6B
Leanstral 1.5, a free Apache-2.0 licensed model with 6B active parameters, delivers a major performance upgrade in formal verification, saturating miniF2F, solving 587/672 PutnamBench problems, and…
BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding
Speculative decoding accelerates inference by using a lightweight draft model to generate candidate tokens in parallel, and are then verified by the target model, enabling lossless…
ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
In prefill-decode (PD) disaggregated LLM serving, each request is assigned to a decode worker after prefill. Existing decode routers balance only load; for mixture-of-experts (MoE) models…
GLM 5.2 is really good!
I'm probably late to the party when it comes to reviewing GLM 5.2, but I've been using it recently and I'm impressed. My use case: I…
Micro-World – Action-controlled Interactive world model – AMD
In this work, we introduce Micro-World, an action-controlled interactive world model designed to generate high-quality, open-domain scenes. Built on top of the Wan2.1 family of models,…
Gemma Avatar: Talk to Gemma 4-31B face to face
This is a voice chat with Gemma 4 31B where you talk to a 3D avatar. It listens while you speak, answers with a voice and…
My DeepSeek V4 Pro at home got faster again
You may remember my earlier posts about DeepSeek V4 Pro at home. Today I checked the performance in my llama.cpp branch that contains various fixes and…
Anyone looked into "Thermo-NN" optimizing model architecture via thermodynamic/Landauer cost instead of pure FLOPs?
Came across a project framing architecture search around thermodynamic information cost (Landauer's principle) rather than the usual FLOPs/params tradeoff, the pitch is causal derivation before implementation,…
GLM5.2 on 5x Pro 6000s and a 5090, an expensive journey
This started as something I thought was reasonable. I already had a 5090 for my gaming machine, and I thought a second 5090 would make me…