A community project to build open source models
This is probably a dumb question, but I am going to ask it anyway. In light of open source models getting so popular, the chance any…
Latest
What do you guys use local models for?
I previously built a telegram bot that chatted with people. But it was never put in production. Besides that I often find that even the cheap…
Llama-Server is Throwing Away Your Perfectly Good KV Caches, and How to Fix It
Introduction I watched 2.49 GB of state restore from disk in 1.23 seconds — and then get thrown away. llama-server's slot save/restore promises exactly what long-context…
I built a free desktop app to make running local models dead simple (Athanor Lite)
Been lurking here for a while and using local models for client work. Kept hitting the same wall, the people who would benefit most from local…
I asked Codex to optimize DeepSeek V4 Flash 8-bit MLX on oMLX. Got ~1.6x prefill and ~3x decode speedup.
Follow-up to my earlier posts: Should I sell my Mac Studio? https://www.reddit.com/r/MacStudio/s/GK7QP8Lg87 Kimi benchmark: https://www.reddit.com/r/LocalLLaMA/s/ujBsYLYmpd Short version: my Mac Studio was sitting mostly idle, and from…
GLM 5.2 FP8 with FP8 KV – Terminal-Bench 2.1 = 79.8 (with one time-out that I didnt re-run)
I wanted to test the official results vs fp8 + fp8 kv. basic sglang setup on H200. If anyone wants one of the official tests do…
Supra Reasoning Summarizer — a tiny model to summarize thinking traces from coding agents
Hi, r/LocalLLaMA! SupraLabs just released a model called: SupraLabs/reasoning-summarizer-800m-pre-gguf It is a thought trace summarizer. Basically, the user/dev can send a reasoning + tool calls (if…
Who builds a better tank in Box3D?
Working on a new benchmark suite that tests how well agents can design physical objects in Box3D. tasks include designing cars, trucks, cranes, etc. submitted by…
New toy to test.
A friend has been kind to let me test his new device. 125 ddr5, AMD Ryzen 395. What models you guys recommend for coding that I…
LivePortrait distilled model that can run at 25fps in the browser
This began as me attempting to run the ONNX version of LivePortrait (https://github.com/KlingAIResearch/LivePortrait) in Chrome with WebGPU. It took 30 seconds to generate a single frame.…