← 研踪 PaperTrailarXiv:2607.22305 · v2
A Roadmap to Impactful Pluralistic Alignment Research
首次提交 2026-07-24 21:46
版本更新 2026-11-04 02:03
雷达观测 2026-11-05 11:11
摘要
Much research on responsible AI aims to change the behavior of AI systems that billions of people use. Pluralistic value alignment---the goal of building AI systems that represent and serve diverse human values and perspectives---has emerged as one such active research agenda. Yet, in an audit of the major frontier labs and open-weight developers, we find no public evidence that it has shaped how their models are trained or evaluated. Pluralistic alignment requires adoption to achieve its main goals, so we identify three barriers and lay out a roadmap of research to address each: 1. The justifications for pluralistic alignment so far are mostly normative or theoretical. Studies are needed that measure the benefits and drawbacks of pluralistic AI for users and society. 2. The pluralistic alignment research community has not settled when pluralistic behavior is warranted or what it should look like in practice. Research is needed to specify this behavior concretely enough for developers to operationalize. 3. Existing pluralism evaluations and methods are not ready for deployment. Evaluations and methods are needed that report their effects on other model properties and meet the requirements of production systems. These barriers are not unique to pluralistic alignment, so our roadmap applies to other areas of responsible AI research with similar theories of change. As people hand more decisions to AI systems and AI agents act on their behalf, we hope our roadmap helps this research reach the deployed models making those decisions.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-04 02:03 · 最新观测版本
A Roadmap to Impactful Pluralistic Alignment Research
Much research on responsible AI aims to change the behavior of AI systems that billions of people use. Pluralistic value alignment---the goal of building AI systems that represent and serve diverse human values and perspectives---has emerged as one such active research agenda. Yet, in an audit of the major frontier labs and open-weight developers, we find no public evidence that it has shaped how their models are trained or evaluated. Pluralistic alignment requires adoption to achieve its main goals, so we identify three barriers and lay out a roadmap of research to address each: 1. The justifications for pluralistic alignment so far are mostly normative or theoretical. Studies are needed that measure the benefits and drawbacks of pluralistic AI for users and society. 2. The pluralistic alignment research community has not settled when pluralistic behavior is warranted or what it should look like in practice. Research is needed to specify this behavior concretely enough for developers to operationalize. 3. Existing pluralism evaluations and methods are not ready for deployment. Evaluations and methods are needed that report their effects on other model properties and meet the requirements of production systems. These barriers are not unique to pluralistic alignment, so our roadmap applies to other areas of responsible AI research with similar theories of change. As people hand more decisions to AI systems and AI agents act on their behalf, we hope our roadmap helps this research reach the deployed models making those decisions.
查看此版本解读 →查看此版本原文 ↗