LessWrong AI
· Communities
Side-Effects of Length Penalty in RL
TL;DR(This is a write-up of results obtained by Luc Feron and me from the sprint period of doing Neel Nanda’s MATS Stream Feb 23rd - Mar 6th).Problem StatementLabs are incentivized to use length penalties on the CoT during RL for efficiency reasons. A natural worry is that this causes negative side effects, especially