Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval
Yukuan Zhang, Jiarui Zhao, Shangqing Nie, Jin Kuang, Dong Liu, Shengsheng Wang
原文摘要
Cross-modal hashing (CMH) uses compact binary codes for efficient image-text retrieval. Most methods learn image-text relationships in a continuous space and binarize the representation only at the end. Semantic learning and bit decisions thus remain largely separate. This work addresses two key questions concerning hash-bit generation: \textbf{(1) How to determine hash bits?} We leverage the discrete nature of spiking activity and determine each bit through competition between positive and negative spikes accumulated over time, rather than through a single firing event or subsequent thresholding of continuous representations. \textbf{(2) How to generate such hash bits?} We propose an unsupervised spiking framework for cross-modal hashing, in which image and text jointly contribute to hash-bit generation through temporal interaction. An activation constraint further prevents unsupported decisions caused by silent channel pairs. Extensive experiments on three benchmarks demonstrate the effectiveness of our approach and support temporal spike competition as a design principle for hash-code generation.
cs.IR · cs.MM