LessWrong AI
· Communities
We need to RL less
Recent AI models are really reward hack-y. This is bad. It's the primary way in which current models are misaligned/dangerous/uncontrollable. The hypothesis put forth in this article is that they're like this because we're RL-ing them too hard. We're applying so much optimization pressure on programming and other capab