跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2609.09953 · v2

Improving Cross-Lingual Token Representations by Adding a Pinch of SALT

通过加入少量 SALT 改进跨语言词元表示

首次提交 2026-09-09 17:43
版本更新 2026-11-19 01:40
雷达观测 2026-11-19 15:04
arXiv ↗PDF ↗

摘要

Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.

AI 论文解读

基于摘要 · 非全文审阅

作者提出轻量级后训练方法 SALT,向已有的跨语言句子编码器注入片段级监督,以改善词元表示并兼顾句子级表现。

正在读取账号与研究记录…

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-19 01:40 · 最新观测版本

Improving Cross-Lingual Token Representations by Adding a Pinch of SALT

Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.

查看此版本解读 →查看此版本原文 ↗