跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.19484 · v1

Leveraging Dual-Process Theory for Multimodal Multiparty Turn-Taking Prediction

首次提交 2026-11-15 00:07
版本更新 2026-11-15 00:07
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

In multiparty conversations, a pause does not always mean a speaker has finished, and another voice does not necessarily indicate a change in who holds the conversational floor. Reliable turn-taking therefore requires recognising not only whether a speaker is ready to yield the floor, but also when a possible transition deserves attention. We propose a dual-process-inspired framework that separates continuous candidate discovery from selective verification. A fast, recall-oriented stage identifies plausible transition points. In contrast, a deliberative stage combines acoustic, linguistic, speaker-related, and predicted future-activity evidence, together with multimodal cues where available, to reject false candidates and refine transition timing. We evaluate the framework through three distinct tasks: SHIFT/HOLD classification at supplied decision points, verification of automatically generated candidates, and continuous discovery and localisation of floor transfers. On VoxConverse, we derive action-aware turn-taking events from speaker-diarization annotations to examine unconstrained conversations with varying numbers of speakers. Multi-TPC provides complementary evaluation of controlled triadic interactions using speech, gaze, motion, and prosodic information. The findings reveal that identifying plausible transitions is considerably easier than reliably confirming and localising genuine floor transfers, while the contribution of individual evidence sources varies across conversational settings. These results highlight the importance of treating multiparty turn-taking as a continuous discovery-and-verification problem rather than classification at predefined boundaries.

作者备注:13 pages, 5 figures. Preprint submitted to IEEE Transactions on Audio, Speech and Language Processing

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-15 00:07 · 最新观测版本

Leveraging Dual-Process Theory for Multimodal Multiparty Turn-Taking Prediction

In multiparty conversations, a pause does not always mean a speaker has finished, and another voice does not necessarily indicate a change in who holds the conversational floor. Reliable turn-taking therefore requires recognising not only whether a speaker is ready to yield the floor, but also when a possible transition deserves attention. We propose a dual-process-inspired framework that separates continuous candidate discovery from selective verification. A fast, recall-oriented stage identifies plausible transition points. In contrast, a deliberative stage combines acoustic, linguistic, speaker-related, and predicted future-activity evidence, together with multimodal cues where available, to reject false candidates and refine transition timing. We evaluate the framework through three distinct tasks: SHIFT/HOLD classification at supplied decision points, verification of automatically generated candidates, and continuous discovery and localisation of floor transfers. On VoxConverse, we derive action-aware turn-taking events from speaker-diarization annotations to examine unconstrained conversations with varying numbers of speakers. Multi-TPC provides complementary evaluation of controlled triadic interactions using speech, gaze, motion, and prosodic information. The findings reveal that identifying plausible transitions is considerably easier than reliably confirming and localising genuine floor transfers, while the contribution of individual evidence sources varies across conversational settings. These results highlight the importance of treating multiparty turn-taking as a continuous discovery-and-verification problem rather than classification at predefined boundaries.

查看此版本解读 →查看此版本原文 ↗