跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2603.04836 · v3

Beyond Text: Text-Image Fusion for Scalable Two-Tower Retrieval

首次提交 2026-03-05 13:43
版本更新 2026-11-16 12:22
雷达观测 2026-11-17 17:12
arXiv ↗PDF ↗

摘要

E-commerce customers make purchase decisions by jointly considering product text and visual information, yet most industrial retrieval systems rely almost exclusively on textual signals. Deployed at scale on CPU-based nearest-neighbor infrastructure, these systems leave substantial visual evidence untapped. In this work, we study unified text-image fusion for two-tower retrieval models at a large-scale e-commerce platform, training on 20M query-item pairs. We demonstrate that domain-specific fine-tuning and modality-specific query alignment between user queries and both product text and image modalities are crucial for effective multimodal retrieval. Building on these insights, we propose a mixture-of-modality-experts fusion architecture with a bilinear interaction network to capture cross-modal complementary information. Our model achieves up to 4.86% improvement in nDCG@1 for desirability and 2.36% for relevance over a strong text-only baseline, while preserving the pre-computable item embedding design required for production deployment.

作者备注:Accepted as a regular paper to the Industry & Government Program of the 2026 IEEE International Conference on Big Data (IEEE BigData 2026), Phoenix, AZ, December 14-17, 2026. 9 pages, 6 figures, 5 tables. v2: accepted version (revised title and text). (c) 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v3,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v3 · 2026-11-16 12:22 · 最新观测版本

Beyond Text: Text-Image Fusion for Scalable Two-Tower Retrieval

E-commerce customers make purchase decisions by jointly considering product text and visual information, yet most industrial retrieval systems rely almost exclusively on textual signals. Deployed at scale on CPU-based nearest-neighbor infrastructure, these systems leave substantial visual evidence untapped. In this work, we study unified text-image fusion for two-tower retrieval models at a large-scale e-commerce platform, training on 20M query-item pairs. We demonstrate that domain-specific fine-tuning and modality-specific query alignment between user queries and both product text and image modalities are crucial for effective multimodal retrieval. Building on these insights, we propose a mixture-of-modality-experts fusion architecture with a bilinear interaction network to capture cross-modal complementary information. Our model achieves up to 4.86% improvement in nDCG@1 for desirability and 2.36% for relevance over a strong text-only baseline, while preserving the pre-computable item embedding design required for production deployment.

查看此版本解读 →查看此版本原文 ↗