← 研踪 PaperTrailarXiv:2601.02629 · v3
Self-Learning Surprise Detection in Spatial Audio for Viewport-Aware 360-Video Streaming
首次提交 2026-01-06 09:01
版本更新 2026-10-29 03:36
雷达观测 2026-10-30 10:22
摘要
Adaptive streaming of 360-degree video relies on viewport prediction to allocate bandwidth efficiently. Current approaches predominantly use visual saliency or historical gaze patterns, neglecting the role of spatial audio in guiding user attention. This paper presents a self-learning framework for detecting "surprising" auditory events, moments that deviate from learned temporal expectations, and demonstrates their utility for viewport prediction. The proposed architecture combines SE(3)-equivariant graph neural networks with recurrent temporal modeling, trained via a dual self-supervised objective. A key feature is the natural modeling of temporal attention decay: surprise is high at event onset but diminishes as the listener adapts. Experiments on the AVTrack360 dataset show that integrating audio surprise with visual cues reduces bitrate waste by up to 18% compared to visual-only methods.
作者备注：Accepted at IEEE ICME 2026. 6 pages, 5 figures, 2 tables. Accepted manuscript; replaces the v2 withdrawal notice
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-10-29 03:36 · 最新观测版本
Self-Learning Surprise Detection in Spatial Audio for Viewport-Aware 360-Video Streaming
Adaptive streaming of 360-degree video relies on viewport prediction to allocate bandwidth efficiently. Current approaches predominantly use visual saliency or historical gaze patterns, neglecting the role of spatial audio in guiding user attention. This paper presents a self-learning framework for detecting "surprising" auditory events, moments that deviate from learned temporal expectations, and demonstrates their utility for viewport prediction. The proposed architecture combines SE(3)-equivariant graph neural networks with recurrent temporal modeling, trained via a dual self-supervised objective. A key feature is the natural modeling of temporal attention decay: surprise is high at event onset but diminishes as the listener adapts. Experiments on the AVTrack360 dataset show that integrating audio surprise with visual cues reduces bitrate waste by up to 18% compared to visual-only methods.
查看此版本解读 →查看此版本原文 ↗