arXiv:2606.10177 · v2

VArify: A Visual Analytics System for Verifying Knowledge Enhanced Large Language Model Responses in Food Science

首次提交 2026-06-09 05:11

版本更新 2026-11-12 12:03

雷达观测 2026-11-13 14:16

摘要

Graph Retrieval-Augmented Generation (GraphRAG) grounds Large Language Model (LLM) responses in structured, domain-specific knowledge. However, verification remains difficult because errors can arise at multiple layers: an answer may not be faithful to its retrieved context, the retrieval pipeline may select irrelevant evidence, and the knowledge graph may omit relevant facts or contain incorrect triplets. We present VArify, a visual analytics system for food-science researchers to inspect and repair these layers in a GraphRAG workflow. A formative study with five participants using a conventional chat-and-table interface revealed behaviors reflecting answer anchoring, over-trust, and difficulty tracing multi-hop relationships. In response, VArify combines an evidence-first answer reveal with coordinated chat and hierarchical graph views, provenance inspection, and subquery editing. It connects the generated answer, the retrieval state, and the provenance-linked graph so that researchers can trace discrepancies across them and repair the retrieved context. From a qualitative system-use study on six food science researchers, we characterize how they scrutinize GraphRAG evidence with progressive visual inspection. Our study also reveals an important verification boundary: information absent from both retrieval and a user's knowledge remains unverifiable. We discuss the broader role of visualization in supporting oversight of LLM responses and outline future research directions.