VizCopilot: Fostering Appropriate Reliance on Enterprise Chatbots with Context Visualization
Sam Yu-Te Lee, Jingya Chen, Albert Calzaretto, Richard Lee, Samir Passi, Alice Ferng, Mihaela Vorvoreanu
原文摘要
Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers. However, when the retrieved context misaligns with user intentions, these chatbots often produce ``irrelevantly right'' responses that raise the risk of overreliance. To foster appropriate reliance, we need a different approach than asking users to verify chatbot outputs. We posit that human oversight and modification of the context retrieval process can improve the quality of chatbots' outputs and users' awareness, resulting in more appropriate reliance and higher accountability. In this work, we explore this approach through a Research-through-Design study using VizCopilot, a chatbot extended with corpus visualization techniques. Through a within-subject study comparing VizCopilot and a simplified pure-text chatbot, we evaluate the role of visualization in enterprise chatbots and surface future design opportunities for fostering appropriate reliance through context engineering. Our findings show that visualization not only helps users detect and correct misaligned context, but also enhances user agency and system transparency. At the same time, the study reveals limitations regarding verification support and trust in AI summaries. Based on these findings, we discuss how visualization can foster appropriate reliance, the associated design trade-offs, and our vision towards transparent and sustainable human-AI interaction.
cs.HC