ExLlamaV3 v1.0.0 – Major Performance Upgrades
After over a year in development, ExLlamaV3 has had its first production release. Turboderp has been pulling 10 hour days with Fable to bring us this…
Latest
r/DestroyMyGame destroyed me to the void for using AI. I used Qwen 3.6 27B Q8 with MTP for about 20% of this single HTML file physics shooter game. I remember last year being blown away by GLM 4.5 Air being able to write a somewhat coherent HTML webpage.
Frontier models are just so good though. Fable 5... Gemini 3.1 Pro for design critique and brainstorming. Grok for verification passes. Antigravity with Gemini 3.5 Flash…
How to run Prism Bonsai 27B
According to their GitHub, the file needed is the -Q2_0.gguf version, which requires compiling their version of llama.cpp. Easiest way to download is to use huggingface-cli…
If you had a 384GB (4x Blackwell), what model would you put on it and why?
Hey guys, Company I work for is actually very interested in spending the money to host our own local model for the team. We expect probably…
No GPU? Ran a 350M LLM fully in the browser — pure WASM, 4-bit, zero server
Most "run a model in your browser" demos still lean on a server or WebGPU. This one doesn't: it's a pure WebAssembly inference engine for LiquidAI's…
Is anyone having any luck with the Ternary Bonsai 27B DFlash?
Title says it, is anyone having any luck with the Ternary Bonsai 27B DFlash? I have been playing around with it but have seen no speed…
Some of y’all wonder why anyone would self host AI. Would you accept the opinion of the CEO of Microsoft?
https://techcrunch.com/2026/07/13/satya-nadella-has-issued-a-shocking-warning-to-companies-using-ai/ Venture capitalists have been warning for awhile that OpenAI and Anthropic are getting access to sensitive business information. The risk is the model makers can…
How i got Bonsai-Ternary-27B to run at 120k context <10gb vram.
Hey guys, so its quite a read, its long. If you are bored just read the TLDR and see if its worth it for you: TLDR;…
Qwen 3.5 122B Heretic ROCmFP4 iMatrix
My first time doing anything like this, and I built it because I wanted it. If anyone wants the non Heretic I'll mosey that out as…
[audio.cpp] 10 hours of audio generated in 3 minutes on RTX 5090 (demo included)! C++/GGML based Supertonic 3, MOSS-TTS, IndexTTS2, and Irodori-TTS released
audio.cpp again. Hopefully you are not sick of it yet :) Release 0.3 adds five new models: Supertonic 3, MOSS-TTS-Local, MOSS-TTS-Nano, IndexTTS2, and Irodori-TTS. The highlight…