ViSAR: Training-Free Adaptive-$k$ Retrieval for Document Visual Question Answering
Adrien Mialland, Marc Plantevit, Julien Gallois, Céline Robardet
原文摘要
Document Visual Question Answering (DocVQA) often leverages Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large Vision-Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k$ number of pages regardless of query complexity, which increases LVLM latency and may degrade answer accuracy when fewer pages would be enough. We introduce ViSAR (Visual Semantic Activation Retrieval), a training-free, budget-aware adaptive-$k$ retrieval method for late-interaction visual document retrieval. ViSAR operates directly in the embedding space to construct a query-conditioned page-level similarity matrix that highlights query-relevant semantics and dynamically determines the number of pages to retrieve, within a given page budget. We evaluate our method across multiple datasets, encoders, and LVLMs. ViSAR retrieves compact, query-adapted page sets that reduce end-to-end RAG latency by up to 58.9\%, with no statistically significant accuracy loss and significant gains in several settings, compared with fixed top-$k$ and adaptive retrieval heuristics. Furthermore, we show that the similarity matrix structure correlates with answer accuracy, suggesting future directions for retrieval quality-aware document understanding.
cs.IR · cs.AI · cs.CL · cs.CV