平滑内点、 erratic 外点:用于双视图对应学习的频率感知 Mamba
Smooth Inliers, Erratic Outliers: Frequency-Aware Mamba for Two-View Correspondence Learning
Zhihua Wang, Yifeng Gao, Yunhao Feng, Rui Pan, Zhouyuan Ma, Baiheng Zhao, Zixing Chen, Yanping Li 等 10 位作者
AI 速读 基于摘要
提出 SFMambaNet,通过聚类扫描和可学习频域门控,利用频率特性区分正确与错误匹配,提升双视图对应学习性能。
cs.CV · cs.AI
查看原文摘要
When shown matches between two photographs of the same cathedral, people can tell at a glance which ones are wrong: a window matched to its look-alike jumps against its neighbors, while correct matches shift together. Two-view correspondence learning, also called correspondence pruning, asks a network to make this judgment for thousands of candidate matches, most of which are wrong. Mamba, a fast sequence model, now drives strong pruners, but it reads the matches as a plain sequence, and whether its scan can capture such agreement is unclear. We propose SFMambaNet, which scans the matches cluster by cluster, so that correct matches vary slowly along the sequence while wrong ones stay erratic, and filters the scanned features with learnable frequency-domain gates between its forward and backward passes. On camera pose estimation, outlier removal and homography estimation, we find that SFMambaNet outperforms spatial-domain Mamba baselines with about two million parameters. Contrary to the usual expectation that such gains come from extra capacity, we find that a parameter-free low-pass filter beats point-wise layers as large as the gates. The learned gates turn low-pass on their own and make the scan far less sensitive to how the matches are shuffled. Together, these results show that frequency is a natural language for describing agreement between correspondences, with implications for other sequence models that must read unordered sets, such as point clouds and graphs.