Token-Level Entropy Reveals Demographic Disparities in Large Language Models
arXiv:2501.19337v5 Announce Type: replace-cross Abstract: A name alone measurably reshapes a language model's next-token distribution before a single token is sampled. We measure full-vocabulary Shannon entropy…