面向标注任务中人口统计分歧建模的可扩展多任务学习框架
A Scalable Multi Task Learning Framework for Modeling Demographic Disagreements in Annotation Tasks
Soumyajit Gupta, Maria De-Arteaga, Matthew Lease
AI 速读 基于摘要
提出 Stakeholder-Aware Joint Multi-Task Learning(SAJ-MTL)框架,联合建模标注者群体之间与群体内部的人口统计分歧,并支持按任务定制群体权重。
cs.CL · cs.LG
查看原文摘要
Annotators from different demographic backgrounds often disagree in systematic ways due to varying lived experiences, cultural contexts, and their relationship to the content. When such disagreement occurs, one group's perspective may sometimes warrant prioritization because they hold an epistemically advantaged standpoint or bear greater stake in the outcome. For example, when assessing toxicity of posts on social media, it may be important to prioritize a targeted group's viewpoint. We propose a Stakeholder-Aware Joint Multi-Task Learning (SAJ-MTL) framework to jointly model demographic disagreement both across- and within- annotator groups. The model uses conditional gradient routing to handle sparsity induced by group-conditional labeling, and parameterization supports customized weighting of groups in different task settings. SAJ-MTL improves group-level performance in toxicity detection through better context-aware understanding of demographic-sensitive content. Only annotator group-membership information is required, which is more practical than existing methods requiring more privacy-invasive individual-level annotator details. Our scalable architecture is also computationally efficient to support large-scale deployment.