r/LocalLLaMA July 19, 2026 · Communities

Did someone used Nvidia’s Tensor-RT LLM or other streaming frameworks like DeepSpeed to run full precision models > than the VRAM+RAM?

I wan to do full precision inference with un-quantized, unmodified MoE models like Hy3 on a machine where the VRAM and system RAM together can't hold the full weights of the model. I have my own vibe-coded stuff but I was wondering if any of these pseudo-unified memory and streaming frameworks have been run with actual