My GX10 died
Everything ran fine, I was using UD 3.6 Q6 for 35 and 27B, each 4 concurrent requests at 200K context. I had Dify and Mastra to…
Latest
psa: can we talk about the number one slop producer & enemy to local/OSS models – “Pliny the Liberator”
i’m actually unsure how Pliny the walking hallucination has grown to as big & credible (amongst uneducated individuals) as he is. my working theory is that…
Python Toolkit: a GUI to manage python, venv, packages, reqs, AI interfaces and more…
submitted by /u/Kinda-Brazy [link] [comments]
hwatu: a verification browser for local coding agents. Headless WebKit, DOM eval, pixel-diff with real match %, no Chromium (MIT, Rust)
submitted by /u/hongnoul [link] [comments]
Who ONLY use local models?
Please be honest. I would love to hear about guys really dedicated to local AI and who really reject subscriptions (especially to openai and anthropic). What…
ModelExpress: Distributing Model Artifacts at the Speed of Light – NVIDIA Technical Blog
We cut DeepSeek-V4 Pro startup from 8 minutes to under 2 minutes by moving weights over the fastest path to GPU memory with GPU-to-GPU RDMA. This…
Setup for always on assistant
I have a dual 3090 rig that I use as coding assistant and while it works, it draws a lot of electricity. Now I want to…
Any idea about these jailbreaks?
Do you guys have any idea what these jailbreaks are? I searched, but I couldn't find any. submitted by /u/Suhan_XD [link] [comments]
PSA: DO NOT use Intel consumer platforms for multi-GPU setups
Since a lot more people are trying to build their own multi-GPU machines, I thought I should help to prevent a common mistake people make with…
DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report)
Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference. The goal…