← 研踪 PaperTrailarXiv:2611.22431 · v1
Controlling Proactive Multimodal AI: Separating Perception from Participation
首次提交 2026-11-17 17:06
版本更新 2026-11-17 17:06
雷达观测 2026-11-18 19:42
摘要
Proactive multimodal AI can watch a user's screen, listen to speech, track interaction, and decide whether to enter unfolding work. Little is known about how people want to control what such AI may perceive and how far it may participate. Informed by a 14-person Wizard-of-Oz study, we built SpotMe, a design probe that senses screen, speech, and interaction; chooses Silence, Ask, or Assist; and lets users separately configure what it may perceive, how far it may participate, and what types of support it may provide. In a 16-person study on a civic dashboard, participants first experienced proactive help, then inspected decisions to act or remain silent, and then configured SpotMe's perception and participation. We found that participants could keep all perception channels available while lowering participation to Ask or Silence; that exposing suppressed decisions made silence interpretable and contestable; and that seeing what SpotMe used led some to change how they clicked, highlighted, and spoke. We contribute an empirical account of inspecting and controlling proactive multimodal AI by decoupling perception from participation.
作者备注:30 pages, 4 figures, 2 tables
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-17 17:06 · 最新观测版本
Controlling Proactive Multimodal AI: Separating Perception from Participation
Proactive multimodal AI can watch a user's screen, listen to speech, track interaction, and decide whether to enter unfolding work. Little is known about how people want to control what such AI may perceive and how far it may participate. Informed by a 14-person Wizard-of-Oz study, we built SpotMe, a design probe that senses screen, speech, and interaction; chooses Silence, Ask, or Assist; and lets users separately configure what it may perceive, how far it may participate, and what types of support it may provide. In a 16-person study on a civic dashboard, participants first experienced proactive help, then inspected decisions to act or remain silent, and then configured SpotMe's perception and participation. We found that participants could keep all perception channels available while lowering participation to Ask or Silence; that exposing suppressed decisions made silence interpretable and contestable; and that seeing what SpotMe used led some to change how they clicked, highlighted, and spoke. We contribute an empirical account of inspecting and controlling proactive multimodal AI by decoupling perception from participation.
查看此版本解读 →查看此版本原文 ↗