Looking for best current solutions for combining cloud models and local models seamlessly inside a harness' orchestration submitted by /u/tat_tvam_asshole [link] [comments]
Link to the article: KV Cache Quantization Benchmarks: KVarN, Precision Tail KLD benchmarks with BeeLlama.cpp v0.4.0, fork of llama.cpp with more KV cache quantization options. Models:…
I'm the author, so discount the enthusiasm accordingly. This is an unaffiliated community port, not endorsed by the vLLM project, which it uses to verify its…
Reading through this subreddit and many more I keep running into what I am calling "AI click bait". Either projects that seems interesting in the description/title…
(Disclaimer: I am a noob and don’t know what I am doing) Gemma 4 31b unsloth/gemma-4-31B-it-qat-GGUF I took the f16 MTP draft model and quantised it…
Following up on yesterday's post about running everyone's faves on 2 x 16gb cards while maximizing performance and KV. Previous post data used abandoned Cu130 VLLM…
NVIDIA Nemotron Parse 2.0 transforms document images into structured, machine-readable representations with text, layout classes, bounding boxes, and reading-order information. Given a Red, Green, Blue (RGB)…
In a previous post, I compared MinerU, Granite-Docling, and PaddleOCR-VL. Many commentors suggested I added their favorite parsers. So I did. And also added some new…
*Not affiliated with the project, just sharing new model I found submitted by /u/AaronFeng47 [link] [comments]
I ran 32 local models head to head on one fact-extraction corpus, 1,001 notes, paired bootstrap on every adjacent pair. Several weeks of compute time, all…