← 研踪 PaperTrailarXiv:2602.00123 · v2
Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models
首次提交 2026-01-28 01:56
版本更新 2026-11-07 20:26
雷达观测 2026-11-10 11:14
摘要
Vision-language models (VLMs) show promise for large-scale affect inference from images, but their alignment with human affective ratings remains unclear. We benchmarked 16 proprietary and open-source VLMs on three validated affective image datasets: the International Affective Picture System, the Nencki Affective Picture System, and the Library of AI-Generated Affective Images. In a zero-shot setting, models performed two tasks: (i) top-emotion classification and (ii) prediction of human ratings on 1--7/9 Likert scales for discrete emotions and affective dimensions. We also evaluated rater-conditioned prompting on the LAI-GAI dataset using de-identified participant metadata. Best models showed strong discrete emotion classification (60%--80% on six-emotion tasks; 60%--75% on a 12-category task), with anger and surprise predicted least accurately. For continuous ratings, models aligned moderately to strongly with human judgments ($r > 0.75$) but showed consistent biases, especially weaker performance on arousal and overestimation of response intensity. Rater-conditioned prompting had only small, inconsistent effects. Overall, VLMs captured broad affective patterns but sometimes lacked the nuance, although newer model generations performed better.
作者备注：Accepted at ACII 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-07 20:26 · 最新观测版本
Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models
Vision-language models (VLMs) show promise for large-scale affect inference from images, but their alignment with human affective ratings remains unclear. We benchmarked 16 proprietary and open-source VLMs on three validated affective image datasets: the International Affective Picture System, the Nencki Affective Picture System, and the Library of AI-Generated Affective Images. In a zero-shot setting, models performed two tasks: (i) top-emotion classification and (ii) prediction of human ratings on 1--7/9 Likert scales for discrete emotions and affective dimensions. We also evaluated rater-conditioned prompting on the LAI-GAI dataset using de-identified participant metadata. Best models showed strong discrete emotion classification (60%--80% on six-emotion tasks; 60%--75% on a 12-category task), with anger and surprise predicted least accurately. For continuous ratings, models aligned moderately to strongly with human judgments ($r > 0.75$) but showed consistent biases, especially weaker performance on arousal and overestimation of response intensity. Rater-conditioned prompting had only small, inconsistent effects. Overall, VLMs captured broad affective patterns but sometimes lacked the nuance, although newer model generations performed better.
查看此版本解读 →查看此版本原文 ↗