Skip to content

Latest

r/LocalLLaMA Communities

Built Kivarro, an all-in-one local inference workbench. Looking for brutal feedback from people who actually run models locally.

I’ve been building Kivarro, a desktop app for local model inference. The idea is simple: Most local LLM tools solve one slice of the workflow. One…

r/LocalLLaMA Communities

How a 128gb ddr5 ram + 16gb vram, would work for a Moe model like Qwen 3.5 122b?

Who has results for this? submitted by /u/Stock-Union6934 [link] [comments]

r/LocalLLaMA Communities

[RELEASE] Supra-Router-51M – a tiny prompt routing model/orchestrator

Hey r/LocalLLaMA ! SupraLabs is back with a new model. Supra-Router-51M. Basically, this is a model which is made for routing requests to smaller or bigger…

r/LocalLLaMA Communities

I would like to have a strong local ollama with tooling alike GitHub’s Copilot that works through terminal..

I’ve already been experimenting with some larger models on my Apple M3 Ultra 96GB (can fit multiples up to around 65 GB total) and I’ve noticed…

r/LocalLLaMA Communities

I built Kivarro, an open-source Rust/Tauri local LLM inference workbench. Looking for r/LocalLLaMA testers.

Hey r/LocalLLaMA, I’m the creator of Kivarro. It’s an open-source local model inference workbench built with Rust/Tauri, focused on running GGUF models locally with llama.cpp, profile…

r/LocalLLaMA Communities

Qwen 3.6 27B – VLLM Performance Benchmark Results (BF16, FP8, NVFP4)

Sharing some testing of Qwen 3.6 27B using VLLM across the popular quants on my development system. I used llama benchy to generate the results, then…

r/LocalLLaMA Communities

is LM Link just too uncooked/experimental?

Newbie here. Setup LM Studio with Gemma and QWen, setup a local SearXNG container, verified that it works in the browser, linked LM Studio and the…

r/LocalLLaMA Communities

Using llama.cpp with pi

A lot of people have their own setups using pi with llama.cpp. I have my own so thought I'll share. This is the extension I and…

r/LocalLLaMA Communities

longcat 2.0 (1.6T, ~48B active) weights are now open under MIT license

From: elie on 𝕏: https://x.com/eliebakouch/status/2073690402503487902 ModelScope on 𝕏: https://x.com/ModelScope2022/status/2073710226365165679 Technical blog post (June, 30): https://longcat.chat/blog/longcat-2.0/ submitted by /u/Nunki08 [link] [comments]

r/LocalLLaMA Communities

Considering Buying Another RTX 3090 – Benefits?

Currently using dual RTX 3090s, and am happy with it. But never satisfied lol :) I know I've basically maxed out my single stream TPS. (140+…