STT That Can Challenge Dragon Professional on Windows
Are there any local LLM based speech to text that can challenge Dragon NaturallySpeaking or Dragon Professional? Notably with regards to being able to change/delete words…
Latest
ContextForge: local SDK for long term memory that actually holds up over long runs
I’ve been working on long term memory for LLMs for a while and kept hitting the same issue. Everything works for demos, then falls apart. So…
Findings from troubleshooting p2p on 4×5060 ti bifurcation.
I dumped the last week deep diving this and I’m I’ve been using Linux for 14 years and am a cloud systems engineer with a focus…
How to distill my own models?
I've been using cloud provided models for agentic theorem proving a lot, and cost is becoming an issue for me. I have funding for hardware cost…
Took the plunge! (Minisforum MS-S1 Max)
With Apple prices entering the stratosphere, the recent Fable gov't rug pull, and the inevitable closed-model price increases, I decided to pick up a (lightly) used…
Can Qwen3.6-35B-A3B on an RTX 3060 Replace Google Vision for Receipt-to-JSON Extraction?
I tried replacing Google Vision in my receipt pipeline with a local Qwen model. I had an old LINE message bot where I could send a…
Upgraded my budget build to multi-GPU for inference
I added: 1x RTX 3090 - 610 USD 1x Arc A770 - 222 USD 1x PCIe x1 to 4x USB 3.0 PCIe riser New cpu cooler…
My config for daily beta llaama.cpp vulcan on 7900xtx/ubuntu. 262k inf, qwen3.6 35b a3b iq4_xs. sits about 22k MiB. crazy fast token generation and compacting. Twice as fast as optimized rocm 7.14 and lower memory usage/footprint.
#!/usr/bin/env bash # llama-server (Vulkan) — Qwen3.6-35B-A3B IQ4_XS. Set paths in CONFIG, then run. # Needs a Vulkan-enabled llama.cpp build and a Vulkan driver (vulkaninfo --summary).…
Nemotron-3-Super-120B-A12B (hybrid Mamba+MoE) holds perfect needle retrieval to 504K tokens on 4×3090
TLDR: The Mamba/SSM layers keep a constant-size recurrent state instead of a growing KV cache, so context is nearly free. Full needle retrieval at half a…
vulkan: make TP viable by pwilkin · Pull Request #25051 · ggml-org/llama.cpp
The legend Piotr has taken a pass at making Vulkan Tensor Parallel somewhat usable, really looking forward to seeing this evolve submitted by /u/TKGaming_11 [link] [comments]