arXiv cs.LG
· Papers
Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winner
arXiv:2604.18239v4 Announce Type: replace Abstract: Preference optimization is widely used to align large language models (LLMs) with human preferences. However, many margin-based methods also suppress the chosen response when they try to suppress the rejected one, and there is no general way to prevent this across dif