← 研踪 PaperTrailarXiv:2611.15971 · v1
Better Boundary Prediction Does Not Guarantee Better Structural Representation
首次提交 2026-11-12 04:15
版本更新 2026-11-12 04:15
雷达观测 2026-11-13 14:16
摘要
Punctuation marks often mark phrase boundaries, carrying linguistic information. Therefore, punctuation marks' abundance and feasibility as proxies of linguistic structure has led to suggestions that using punctuation marks to optimise on phrasal structure may help language models (LMs) improve their representations of argument, semantic, and syntactic structures. In this paper, we test the above claim in English across three primary LM architectures: decoder-only, encoder-only, and encoder-decoder LMs, by complementing traditional token prediction with punctuation restoration (PR). Our evidence, collected at the <1B parameter scale, suggests that optimising for boundary prediction does not necessarily guarantee higher-quality structural representation in decoder-only, encoder-decoder, and encoder-only LMs, failing to improve performance on downstream tasks that require understanding structure in linguistic input.
作者备注：Presented at LP4FM @ NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-12 04:15 · 最新观测版本
Better Boundary Prediction Does Not Guarantee Better Structural Representation
Punctuation marks often mark phrase boundaries, carrying linguistic information. Therefore, punctuation marks' abundance and feasibility as proxies of linguistic structure has led to suggestions that using punctuation marks to optimise on phrasal structure may help language models (LMs) improve their representations of argument, semantic, and syntactic structures. In this paper, we test the above claim in English across three primary LM architectures: decoder-only, encoder-only, and encoder-decoder LMs, by complementing traditional token prediction with punctuation restoration (PR). Our evidence, collected at the <1B parameter scale, suggests that optimising for boundary prediction does not necessarily guarantee higher-quality structural representation in decoder-only, encoder-decoder, and encoder-only LMs, failing to improve performance on downstream tasks that require understanding structure in linguistic input.
查看此版本解读 →查看此版本原文 ↗