Prefill vs. decoding and local LLM ROI: is prefill underrated?
I'm trying to understand why, when people discuss the ROI of running LLMs locally, they almost always focus on output speed (decoding) and rarely on input…
Latest
The cyber shelf – 4x 16gb home lab
Been a long time lurker of this subreddit, learned a whole lot from here and Gemini. I've finally got my rig somewhere I feel I could…
RTX 5080 + 32gb DDR4
Looking for some suggestions on what I should be realistically aiming for. Recently I've used qwen 3.6 27b at Q3KM, but I've also been able to…
I told Gemma 4 12B (Q8_0, no cache quant) to write a single-file 3D bowling simulator in WebGL. It’s terrible, but honestly better than I expected.
Just sharing some slop. Used opencode as the harness. I know this model isn't really recommended for coding, but I was just curious how it would…
Got my Ascent GX10 two days ago, ran REAP-pruned NVFP4 DeepSeek-V4-Flash on a single Spark, and it stays consistent at long context
Got my Ascent GX10 two days ago and spent the last couple of days pushing a REAP-pruned NVFP4 DeepSeek-V4-Flash setup on a single Spark by patching…
OpenComputer | An Open Source Computer Built For Agents.
Open Computer running in an isolated VM with inference running M4 Pro via LM Studio Gemma 4 13B QAT Hey everyone, Tim from AnythingLLM, where we…
Impact of dual PCIe 5 x8 vs dual PCIe 5 x16 for dual Rtx 6000 pro max-q
i have an existing system with: Asus ROG MAXIMUS Z790 DARK HERO LGA1700 Intel Core i9-14900K G.Skill Trident Z5 RGB 2x48GB DDR5 6800MHz CL34 (x2, 4…
Ant Group released LingBot-Vision: DINO-family vision backbones in 4 sizes, and the 0.3B ViT-L matches DINOv3-7B on NYUv2 depth with ~23x fewer params
Weights, all 4 sizes, Apache-2.0 (ViT-S / ViT-B / ViT-L / ViT-g): https://huggingface.co/collections/robbyant/lingbot-vision Code: https://github.com/robbyant/lingbot-vision Project page: https://technology.robbyant.com/lingbot-vision Self-supervised DINO-family backbone, but the masking is boundary-driven:…
ThinkingCap-Qwen3.6-27B: same accuracy as base Qwen3.6 with ~50% fewer thinking
We rigorously evaluate the resulting checkpoint across general reasoning, non-reasoning multiple-choice question answering, everyday multi-turn conversations, system prompt adherence, safety, math, code and agentic use cases.…
New strix halo box: GMKtec EVO-X3, superior cooling to avoid thermal throttling, $3,600
https://www.gmktec.com/products/gmktec-evo-x3-ai-mini-pc-amd-ryzen-ai-max-395 Features: USB 4. Adds a dedicated OCuLink port. OCuLink provides a direct, high-speed cable connection to a desktop graphics card, minimizing data loss and improving…