arXiv:2607.10130 · v2
TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues
摘要
Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze-aligned textual cues. We design a transformer-based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in-/out-of-frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross-dataset generalisation without extra fine-tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation.
作者备注：Accepted at IJCAI-ECAI 2026 (IJCAI-26, Main Track), pp. 1586-1594. 9 pages, 5 figures. All contents are available at: https://github.com/idremo/TextGaze-IJCAI2026
期刊信息：In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-26), pp. 1586-1594, 2026