Improving the Distributional Alignment of LLMs using Supervision
Gauri Kambhatla University of Texas at Austin, Sanjana Gautam, Angela Zhang, Alexander Liu University of Texas at Austin, Ravi Srinivasan University of Texas at Austin, Junyi Jessy Li University of Texas at Austin, Matthew Lease Amazon and University of Texas at Austin
原文摘要
The ability to accurately align LLMs with diverse population groups on subjective questions would have great value. In this work, we show that adding simple supervision can more consistently improve the alignment of LLM-generated distributions with diverse population groups, as measured across three datasets spanning public health, public opinion, and values and beliefs. Beyond evaluating average alignment, we also report how alignment varies across specific groups. Our broad findings provide insights into the distributional alignment of LLM generations with diverse populations. By conducting evaluation over many LLMs and prompting strategies, we provide a benchmark to stimulate future research.
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表