信号、证据与语境:真实世界大模型对话中的用户审视行为
Signals, Evidence, and Context: User Scrutiny in Real-World LLM Conversations
Ahana Biswas, Yu-Ru Lin
AI 速读 基于摘要
基于 ShareChat 与 WildChat 的 489,283 组助手—用户回复对,分析回答中的归因语言、不确定性等线索与用户接受、核查、质疑行为的关系。
cs.AI
查看原文摘要
Large language models (LLMs) often present synthesized answers without making their supporting evidence easy to inspect. We study how features of these answers relate to users' subsequent accepting, checking, and challenging in 489,283 assistant-user response pairs from ShareChat and WildChat. Broad attribution language appears in 31.9% of qualifying responses, compared with explicit URLs in 1.33%, revealing a gap between claims of grounding and preserved inspectable support. Attribution, uncertainty, and other response cues generally accompany both checking and challenging; only uncertainty shows a small statistically distinguishable pooled association favoring checking. Within Technical Help, uncertainty favors checking relative to challenging; refusals and reasoning traces favor challenging in Seeking Information. Scrutiny is concentrated in some conversations, while its apparent turn-to-turn persistence becomes much smaller when conversation-level differences and response ordering are examined separately. These results suggest that supporting verification requires attention to what evidence can be inspected, which question a user is trying to resolve, and the history of the interaction.