LessWrong AI
· Communities
Measuring Reward-Seeking via Contrastive Belief Updates
Machine learning models can produce the right outputs for the wrong reasons. Famous examples include a reinforcement learning agent that, rewarded for collecting a coin always placed at the right end of the level, learns to run rightward rather than to seek the coin itself (Langosco et al., 2022; Shah et al., 2022). An