Skip to content

Latest

r/LocalLLaMA Communities

I want people here to open your eyes and note how Google did not sign the anti- open-source-model coalition

They are not the champion of open source like the Google shills here want you to believe. Their Gemma models are garbage in all quantization and…

r/LocalLLaMA Communities

CachyLLama’s: llama.cpp fork with persistent KV cache that makes long local-agent sessions much less painful

I’m not affiliated with this project, but I’ve been running it recently and I’m surprised it hasn’t received more attention here: https://github.com/fewtarius/CachyLLama CachyLLama is a fork…

r/LocalLLaMA Communities

llama.cpp slower on P-Cores than on E-Cores with MoE Model and GPU+CPU offloading?

I am currently experimenting with my setup: RTX 5090 + Intel 270K Plus CPU (8 Performance Cores + 16 Efficiency Cores) + 128 GB DDR5-6000 RAM,…

r/LocalLLaMA Communities

Is corruption the lobbying against Open weights?

Like, reading things like Anthropic "donated" to some people with the condition of lobbying against Chinese LLMs.. it's that right? It feels nothing like freedom but…

r/LocalLLaMA Communities

[Paper] Statistically-Lossless Quantization of Large Language Models

Model quantization has become essential for efficient large language model deployment, yet existing approaches involve clear trade-offs: methods such as GPTQ and AWQ achieve practical compression…

r/LocalLLaMA Communities

Spent two weeks on a kernel that benchmarked 29x faster. End to end it’s maybe 6-10%, and it’s not even wired in yet.

I've been building a C99 inference engine from scratch (no Python, no BLAS, just gcc and make) that runs BitNet's ternary models on CPU. A few…

r/LocalLLaMA Communities

Agentic Kernel Optimization, visualized.

A swarm of GPT 5.6 Sol agents spent over 40 hours optimizing a Kimi K3-like model from 65 to 406 tok/s. This animation follows their collaboration…

r/LocalLLaMA Communities

Why won’t he sign the letter then?

submitted by /u/_Sneaky_Bastard_ [link] [comments]

r/LocalLLaMA Communities

It appears that the anti opensource AI lobby is far outgunned already

The earlier post on this subreddit by 20+ companies signing the petition including Microsoft, Meta, Nvidia, YC (https://www.microsoft.com/en-us/corporate-responsibility/topics/open-weight/) etc plus this https://xcancel.com/elonmusk/status/2080672505660834163 And the entire LLM…

r/LocalLLaMA Communities

Honest take on Laguna S2.1 and its uses (from actual use)

So I've taken some time to actually test laguna on a few of my own projects. I wanted to share as I feel most peoples comments…