r/LocalLLaMA
· Communities
Session-Adaptive Orthogonal Distillation (SAOD)? Technology compresses 744B (1.5TB) to under 100GB?
Tweet : https://xcancel.com/jun_song/status/2079914426334167258#m Looks like 8GB VRAM could do more like even run 70-100B MOE models possibly. Sorry about the clickbait title, I want more eyes on this..... zzz submitted by /u/pmttyji [link] [comments]