r/LocalLLaMA
· Communities
I tried fine-tuning a ternary model, Bonsai 8b, on metal
We've heard the hype around binary and ternary models, and it seems recently, their performance is starting to catch up and be competitive for every-day use across a range of devices, although there's still more to wish for... What these models are A new family of true sub-2-bit LLMs is out: PrismML's Bonsai line. Ever