← 研踪 PaperTrailarXiv:2601.16907 · v2
What Does a Cosine of 0.8 Mean? Monotone Calibration of Embedding Similarity
首次提交 2026-01-24 01:14
版本更新 2026-11-16 18:08
雷达观测 2026-11-17 17:11
摘要
Text embedding models turn sentences into vectors, and the cosine between two vectors is the usual measure of their similarity. Search, semantic caching and deduplication often accept a pair when this cosine reaches a fixed threshold. When people judge one pair of sentences more similar than another, the cosine usually agrees, but its value means different things for different models, because each model packs its scores into its own narrow range. The same threshold can therefore keep most pairs with one model and few with another. We propose to calibrate the score instead of changing the model. An increasing function fitted on sentence pairs rated by people translates the cosine of each model into the human rating scale and never reverses the order of two pairs. We fit three such functions for six models, ranging from a simple S-shaped curve to a free step function. On new pairs, calibrated scores agree with human ratings and the models agree with one another. One threshold selects pairs of similar quality whatever the model. The scale obtained belongs to the human ratings used to fit it, so its values shift on two datasets rated differently while the models stay comparable.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-16 18:08 · 最新观测版本
What Does a Cosine of 0.8 Mean? Monotone Calibration of Embedding Similarity
Text embedding models turn sentences into vectors, and the cosine between two vectors is the usual measure of their similarity. Search, semantic caching and deduplication often accept a pair when this cosine reaches a fixed threshold. When people judge one pair of sentences more similar than another, the cosine usually agrees, but its value means different things for different models, because each model packs its scores into its own narrow range. The same threshold can therefore keep most pairs with one model and few with another. We propose to calibrate the score instead of changing the model. An increasing function fitted on sentence pairs rated by people translates the cosine of each model into the human rating scale and never reverses the order of two pairs. We fit three such functions for six models, ranging from a simple S-shaped curve to a free step function. On new pairs, calibrated scores agree with human ratings and the models agree with one another. One threshold selects pairs of similar quality whatever the model. The scale obtained belongs to the human ratings used to fit it, so its values shift on two datasets rated differently while the models stay comparable.
查看此版本解读 →查看此版本原文 ↗