← 研踪 PaperTrailarXiv:2611.15047 · v1
Saying No by Asking Yes: Affirmative Decomposition and Evidence Reflection for Frozen CLIP
首次提交 2026-11-11 13:16
版本更新 2026-11-11 13:16
雷达观测 2026-11-13 11:11
摘要
Frozen vision-language models often match concepts mentioned in a caption without respecting their negation: a query "a kitchen with no toaster" favors an image containing a toaster. Yet these models are substantially stronger at matching affirmative descriptions. We exploit this asymmetry with Negation-Aware Evidence Scoring (NAES), which reformulates negated descriptions into affirmative queries and combines their evidence by caption structure. A lightweight text decomposer, NegSplit, identifies this structure by extracting negated spans and the remaining affirmative context. For captions with an affirmative remainder, NAES matches that content and applies a presence-based correction for the denial. When no affirmative content remains, it queries the denied object's presence and reflects the evidence around a reference to score the negated description. With LAION-2B CLIP, NAES achieves 82.4% accuracy on COCO-MCQ and 93.0% on VOC2007-MCQ, surpassing KTN by 2.9 and 0.9 percentage points; with OpenAI CLIP, it exceeds PeakPatch accuracy by 4.4 and 26.0 points, respectively. NAES also improves negated-description retrieval on COCO and MSR-VTT, with gains extending to SigLIP, SigLIP 2, and larger CLIP encoders. The strongest multiple-choice results use competing descriptions to center and align evidence; retrieval improvements persist without this conditioning. These findings show that explicitly separating affirmative matching from denied-object evidence can substantially improve negation-aware ranking.
作者备注:27 pages, 6 figures
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-11 13:16 · 最新观测版本
Saying No by Asking Yes: Affirmative Decomposition and Evidence Reflection for Frozen CLIP
Frozen vision-language models often match concepts mentioned in a caption without respecting their negation: a query "a kitchen with no toaster" favors an image containing a toaster. Yet these models are substantially stronger at matching affirmative descriptions. We exploit this asymmetry with Negation-Aware Evidence Scoring (NAES), which reformulates negated descriptions into affirmative queries and combines their evidence by caption structure. A lightweight text decomposer, NegSplit, identifies this structure by extracting negated spans and the remaining affirmative context. For captions with an affirmative remainder, NAES matches that content and applies a presence-based correction for the denial. When no affirmative content remains, it queries the denied object's presence and reflects the evidence around a reference to score the negated description. With LAION-2B CLIP, NAES achieves 82.4% accuracy on COCO-MCQ and 93.0% on VOC2007-MCQ, surpassing KTN by 2.9 and 0.9 percentage points; with OpenAI CLIP, it exceeds PeakPatch accuracy by 4.4 and 26.0 points, respectively. NAES also improves negated-description retrieval on COCO and MSR-VTT, with gains extending to SigLIP, SigLIP 2, and larger CLIP encoders. The strongest multiple-choice results use competing descriptions to center and align evidence; retrieval improvements persist without this conditioning. These findings show that explicitly separating affirmative matching from denied-object evidence can substantially improve negation-aware ranking.
查看此版本解读 →查看此版本原文 ↗