← 研踪 PaperTrailCVF · CVPR 2026:cvf:bbae92be1441ce37fe7bc099f2ffd7db · 记录 #1
Iris: Integrating Language into Diffusion-based Monocular Depth Estimation
来源日期（仅到年份） 2026
来源记录日期 2026
雷达观测 2026-11-05 03:27
摘要
Conventional monocular depth estimators suffer from visual ambiguities and nuisances. We demonstrate that language can improve the fidelity of estimates by providing additional information through text as a condition, thereby reducing the solution space for depth estimates. This conditional distribution is learned during text-to-image pre-training of diffusion models. To generate images under various viewpoints and layouts that reflect textual descriptions, the model implicitly encodes object shapes and their spatial relationships, comprising the 3-dimensional (3D) scene structure. We investigate the benefits of integrating text descriptions into the training and inference of the diffusion-based monocular depth estimators (MDE). We experiment with three different diffusion-based MDEs, namely Marigold, Lotus, and E2E-FT, and their variants. By training on HyperSim and Virtual KITTI, and evaluating on NYUv2, KITTI, ETH3D, ScanNet, and DIODE, we find that our strategy of integrating text into MDEs improves the overall accuracy of depth estimates, especially in small areas. The effect is also targeted in that improvements are more pronounced in specific regions described in the text. Naturally, this lends to iterative refinement of the depth estimate by providing additional descriptions of the 3D scene. Separately, we find that incorporating text can accelerate convergence of training and the inference diffusion trajectory.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026 · 最新观测版本
Iris: Integrating Language into Diffusion-based Monocular Depth Estimation
Conventional monocular depth estimators suffer from visual ambiguities and nuisances. We demonstrate that language can improve the fidelity of estimates by providing additional information through text as a condition, thereby reducing the solution space for depth estimates. This conditional distribution is learned during text-to-image pre-training of diffusion models. To generate images under various viewpoints and layouts that reflect textual descriptions, the model implicitly encodes object shapes and their spatial relationships, comprising the 3-dimensional (3D) scene structure. We investigate the benefits of integrating text descriptions into the training and inference of the diffusion-based monocular depth estimators (MDE). We experiment with three different diffusion-based MDEs, namely Marigold, Lotus, and E2E-FT, and their variants. By training on HyperSim and Virtual KITTI, and evaluating on NYUv2, KITTI, ETH3D, ScanNet, and DIODE, we find that our strategy of integrating text into MDEs improves the overall accuracy of depth estimates, especially in small areas. The effect is also targeted in that improvements are more pronounced in specific regions described in the text. Naturally, this lends to iterative refinement of the depth estimate by providing additional descriptions of the 3D scene. Separately, we find that incorporating text can accelerate convergence of training and the inference diffusion trajectory.
查看此版本解读 →查看此版本原文 ↗