Reducing Confirmation Bias on Controversial Issues through LLM-Generated Multi-Persona Debates
Li Shi, Houjiang Liu, Yian Wong, Utkarsh Mujumdar, Dan Zhang, Jacek Gwizdka, Matthew Lease
原文摘要
Multi-persona debate systems powered by large language models (LLMs) show promise in reducing confirmation bias; however, empirical evidence remains limited. Through a within-subjects experiment, we compared an LLM-generated multi-persona debate system with an opposing-stances retrieval system. We evaluated system effectiveness in mitigating confirmation bias over 312 study trials across 8 controversial topics, using behavioral measures (changes in eye-fixation patterns) and attitudinal measures (shifts between pre/post-intervention beliefs), with topic familiarity and cognitive tendencies as moderators. Our results show that: (i) participants allocated more than half of their visual attention to attitude-inconsistent content in both systems, with no significant between-system differences; (ii) the debate system showed a suggestive but statistically non-significant buffering pattern among participants with stronger cognitive tendencies, and topic familiarity did not significantly moderate its effect on visual attention; and (iii) reading effort and perceived argument quality influenced participants' engagement with attitude-inconsistent content. These findings suggest that multi-persona debate systems can support engagement with diverse viewpoints, while their comparative advantage over opposing-stances retrieval was not established; individual cognitive tendencies remained an important predictor of visual attention. We provide further implications for designing multi-persona debates to better mitigate confirmation bias, focusing on balancing facts and persuasion and on deciding when to explore diverse viewpoints versus offer concrete solutions.
cs.HC · cs.CY · cs.IR