VibeVoice 1.5B Running Locally…On an iPhone! Only ~2.2 GB of Memory and Up to 1.28× Real-Time Speed
I speed up the generation part of the demo in case you get bored 😄 I also tested another long-form generation, and the VRAM usage looks…
Latest
Thinking of buying more DRAM right now…
So I'm looking at https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF and I realize my 128GB of DRAM just isn't cutting it for this (incredibly powerful) model. If only I had another…
PSA Update CUDA from 13.2 to 13.3 to solve DeepSeek V4 Flash 0731 Looping Problem!
So one of yall mentioned that cuda 13.1 or 13.2 is broken for unsloth so I looked in to it, and they were right. I had…
Maple-Preview: 20B-A1B ternary-weight reasoning open-weight LLM
submitted by /u/cafedude [link] [comments]
DeepSeek-V4-Flash on SM89 4x48gb 4090s with DSpark
https://github.com/yhfgyyf/vllm-deepseek-v4-sm89 I couldn't believe that someone actually got vLLM working with this particular set of GPUs, but here it is. The video is from right after…
DeepSeek-v4-Flash-Mini 54GB GGUF running at ~20.5 t/s
Took the REAP adaptation of DeepSeek-V4-Flash (0xSero/DeepSeek-V4-Flash-0731-REAP) along with antirez/deepseek-v4-gguf as inspiration, and decided to see how aggressive we could get with standard quant tricks to…
White House AI Guidelines Exempt U.S. Open Models From Government Review
submitted by /u/realmvp77 [link] [comments]
GPT-OSS has turned one year old today!
It is one of the best local models ever released, in both 20B and 120B versions. I always come back to it, especially the 120B version.…
A 2.6B model with tool calling and 128K context now runs at 30 tok/s on a phone
Liquid AI released LFM2.5-2.6B today, and this might be more relevant to local AI than another massive model most people cannot run. The model is only…
Qwen 3.8 Max improves over Qwen 3.7 Max on the Debate Benchmark: 1462 → 1588. But the average cost per debate increased by 45%.
More info: https://github.com/lechmazur/debate https://preview.redd.it/5rr60v33bfhh1.png?width=3600&format=png&auto=webp&s=bbe9ff2e3c2e599e887a4ec0da7bcc26f71ad4fb This benchmark measures how well LLMs hold an argument under adversarial, multi-turn opposition across a wide range of topics. It rewards broad…