Benchmarks Benchmarked on gfx1151 (RDNA3.5/Strix Halo, 40 CUs) against latest master (2f6c815). These two models cover both the FA paths. Qwen3.5-27B Q4_K_M — head_dim=256 (TILE path),…
Hi. I don't have a GPU. So my biggest "local LLM" experience has been running ~26B models with single-digits tps values. However, the "serving economy" of…
I have this feeling for sometime. Also noticed few similar tweets online before. submitted by /u/pmttyji [link] [comments]
submitted by /u/johnnyApplePRNG [link] [comments]
Collection: https://huggingface.co/collections/tencent/hy3 From elie on 𝕏: https://x.com/eliebakouch/status/2074011171661701466 edit: To clarify: this is the non-preview version of Hy3 and they changed their license from the community one…
I've posted up earlier versions of this project before, promising a GitHub link, but never got around to pushing the code from my local system. Sorry…
## Why local is non-negotiable I'm a criminal defense lawyer and the documents I'd feed this contain **client-confidential case material**. Sending real case content to a…
I've been trying to keep as much of my AI workflow local as possible. One thing I couldn't find was a good way to work with…
*for a specific model, in the R language. veeery impractical. 60 seconds per token, yum 😋 not vibecoded, I just wanted to learn how the architecture…
submitted by /u/PetersOdyssey [link] [comments]