The Linguistic Structure of Dependency-Conditioned Embeddings
Bach Phan-Tat, Tung Kieu, Kris Heylen, Dirk Geeraerts, Stefano De Pascale, Dirk Speelman
原文摘要
Dependency-based distributional models capture syntactically informed lexical similarities, but the geometric organisation of dependency-conditioned lexical representations remains underexplored. We investigate the global and local geometry of lemma-dependency relation embeddings using four representations trained on WikiText-103: two count-based PPMI configurations and two Skip-Gram with Negative Sampling (SGNS) configurations. At the global level, we analyse dependency-relation centroids, control for lexical identity through within-lemma centring, and examine their alignment with part-of-speech (POS) distributions. At the local level, we evaluate cross-relation geometric consistency, cross-method agreement, lexical similarity, and polysemous usages. Results show that global geometry is structured strongly by context POS across all representations. Locally, broader lexical similarity structures are partially preserved across dependency relations, while immediate neighbourhoods vary substantially, highlighting different usages. PPMI-based representations exhibit greater cross-relation consistency than SGNS-based representations, with cross-method agreement varying by relation. These findings reveal systematic linguistic organisation at multiple geometric scales and its sensitivity to representation choices.
cs.CL