arXiv:2609.05880 · v2
Grounded and Faithful P&ID Reasoning: Constraining Vision-Language Models with Recovered Evidence Graphs
摘要
Piping and Instrumentation Diagrams (P&IDs) are the authoritative maps of process plants: isolation, maintenance, and HAZOP decisions depend on what connects to what. Vision-language models describe these sheets fluently, yet they often invent or miss process connections---and an invented or missed link can reverse an isolation or reachability call, so a plant decision cannot trust a fluent answer that was never checked against the linework. We instead recover an explicit graph of the drawing---its symbols, the process connections between them, and the tags that name them---and then require the model to answer only by querying that graph through seven read-only operators, so a topology claim is returned only when it cites the query results that support it. On TopoPID-VQA, a new suite of 3000 topology questions over these sheets, Graph-Grounded Harness (Ours) raises exact match accuracy from 36.7--41.3% under image-only prompting to 74.3--76.0% for Qwen3-VL-4B, Qwen3-VL-8B, and Gemma-4-E4B. It does so on an imperfect substrate: on Digitize-PID dataset the recovered graph scores F1 0.742 on exact process connections, and 0.801 once symbols and tags are pooled in. The residual errors track that gap---grounding pays off where the recovered graph is right, and perception error still breaks topology questions where it is not.
作者备注：Accepted to the NeurIPS 2026 VLM4RWD Workshop. Changes in v2: (1) added answer-contract ablation results (Table 2), as suggested by reviewers; (2) added fine-tuned VLM results; (3) added the harness system prompts (Appendix A.9) and appendix sections describing the answer-contract ablation (A.10) and the fine-tuning experiments (A.11)