r/LocalLLaMA
· Communities
PSA: Upscaling Gemma 4 requires a proportional layer_scalar adjustment
A lot of people seem to be confused or mystified about this so figured I'd spell it out. I played around with RYS and realized that it broke Gemma 4 models. Turns out there's a `layer_scalar` value that is applied at each layer. If you don't adjust that so that the resulting model gets "the same amount", you break it.