超越第三人称审计:面向用户中心的LLM偏见研究的情境化交互审计
Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research
Andrés Abeliuk, Cinthia Sanchez Macias, Valentina Alarcón, Álvaro Madariaga, Claudia Lopez
AI 速读 基于摘要
提出情境化交互审计(SIA)框架,用于统一和比较以用户为中心的第一人称LLM偏见研究。
cs.CY · cs.CL
查看原文摘要
Research on bias in large language models (LLMs) has predominantly focused on third-person audits, which study how models represent or evaluate demographic groups as external subjects. However, this paradigm overlooks a structural blind spot because the user is absent from the audit. In practice, LLMs are used in open-ended, personal interactions, during which the model implicitly represents the user and adjusts its responses accordingly. When identical requests yield different responses depending on who is asking, bias manifests not in how the model describes others but in how it treats its interlocutor. Empirical first-person bias studies are multiplying, but they use different cues, outcomes, and validation choices, which makes their results hard to compare. We propose Situated Interaction Auditing (SIA), a shared diagnostic framework for this literature. It introduces a typology of user profile signals---names and surface markers, writing style and dialect, and stated identity---together with design principles and outcome measures that let studies be compared and accumulated. We instantiate the framework in a case study of Chilean first and last names across nine task domains and three models, and we close with guidance for practitioners and open questions for researchers.