← 研踪 PaperTrailarXiv:2610.27508 · v2
KatoANN: Enabling Near-Exact Vector Search at the Speed of Lossy Scans with CPU Matrix Engines
首次提交 2026-10-20 10:22
版本更新 2026-11-13 13:44
雷达观测 2026-11-17 04:57
摘要
Emerging CPU matrix engines, such as Intel AMX, enable high throughput by batching multiple queries. However, they support only low-precision formats, which can significantly degrade recall (i.e., search quality). We present KatoANN, the first approximate nearest neighbor (ANN) search system that achieves both high throughput and high recall simultaneously on CPU matrix engines. Specifically, KatoANN represents each vector using a 16-bit code, splits it into high- and low-byte planes, and computes near-exact distances using 8-bit matrix multiplications that the matrix engine supports. To reduce extra multiplication overhead, it ranks all candidates by their high bytes alone and refines only a few survivors with the full distance. KatoANN also reduces memory footprint since it does not maintain a full-precision copy of the vectors. We implement a KatoANN prototype on Intel AMX. Our experimental results show that KatoANN outperforms existing partition-based systems across the high-recall range.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-13 13:44 · 最新观测版本
KatoANN: Enabling Near-Exact Vector Search at the Speed of Lossy Scans with CPU Matrix Engines
Emerging CPU matrix engines, such as Intel AMX, enable high throughput by batching multiple queries. However, they support only low-precision formats, which can significantly degrade recall (i.e., search quality). We present KatoANN, the first approximate nearest neighbor (ANN) search system that achieves both high throughput and high recall simultaneously on CPU matrix engines. Specifically, KatoANN represents each vector using a 16-bit code, splits it into high- and low-byte planes, and computes near-exact distances using 8-bit matrix multiplications that the matrix engine supports. To reduce extra multiplication overhead, it ranks all candidates by their high bytes alone and refines only a few survivors with the full distance. KatoANN also reduces memory footprint since it does not maintain a full-precision copy of the vectors. We implement a KatoANN prototype on Intel AMX. Our experimental results show that KatoANN outperforms existing partition-based systems across the high-recall range.
查看此版本解读 →查看此版本原文 ↗