Xiaoguang Chang, Tuo Chen, Chengze Jiang, Teng Wang, Changyin Sun
Scene Graph Generation (SGG) produces structured scene representations, yet its training-time backdoor risk remains unexplored. Unlike classification, SGG exposes a pair-level attack surface that enables selective corruption of relations involving a designated object category. However, because SGG largely relies on localized evidence, globally placed triggers may face difficulty in manipulating such predictions. Recent SGG architectures increasingly incorporate global context to improve scene modeling, which may in turn facilitate such attacks. In this paper, we systematically study this backdoor risk and investigate the role of the global context pathway in backdoor behavior. Concretely, we first instantiate this attack with an effective method, and propose a three-axis evaluation protocol to characterize backdoor behavior. Experiments on Visual Genome with representative open-vocabulary and closed-set SGG models demonstrate the effectiveness of the attack. Mechanistic analyses reveal that the explicit global context pathway serves as a major carrier of the backdoor effect across the evaluated SGG models. In particular, replacing the poisoned global context representation with its clean counterpart during inference sharply suppresses the attack, yielding 100% and 97% reductions in attack success for the OvSGTR family and RelationFormer, respectively. Moreover, training-time interventions in the OvSGTR family show that suppressing trigger signals in this pathway either reduces attack success or substantially degrades utility. Together, these findings suggest that explicit global context designs may pose a backdoor risk in SGG architectures.
Linzheng Wang, Jason Chen, Nicolas Tricard, Zituo Chen, Xingsen Guo, Sili Deng
Understanding physical systems often requires recovering their full spatial state from measurements available only at sparse locations. This reconstruction becomes challenging when physical boundaries are unknown, high-resolution training fields are unavailable, or important quantities cannot be measured directly. Sparse observations may also be consistent with multiple plausible states, leading to significant reconstruction uncertainty. Here we introduce Direct Measurement-to-Field Generation (\DMF{}), a functional rectified flow framework that learns how sparse measurements drive physical-field generation end to end. Instead of relying on grid-based rasterization or runtime guidance, \DMF{} jointly learns a sensor--global--sensor representation with the generative transport, dynamically fusing local grounding with global context to evaluate physical fields at arbitrary query coordinates. Across benchmarks involving unobserved boundaries, missing high-resolution training fields, and unmeasured physical quantities, \DMF{} outperforms alternative baselines by significantly reducing reconstruction errors and preserving the fidelity of physical properties. Furthermore, its stochastic ensemble generation reliably quantifies uncertainty while improving continuous ranked probability scores. These results extend sparse reconstruction from fixed-setting point estimation toward measurement-conditioned generation of physical fields under incomplete and changing information.