← 研踪 PaperTrailarXiv:2607.27002 · v2
IMFuse: Instance-Aware Multi-Layer Fusion for LLM-Enhanced Sequential Recommendation
首次提交 2026-07-29 22:58
版本更新 2026-11-04 18:38
雷达观测 2026-11-05 11:10
摘要
Leveraging large language models (LLMs) to enrich item embeddings with textual semantics has become a prevailing paradigm for enhancing sequential recommendation (SR). However, existing methods typically rely solely on the final-layer LLM embeddings, leaving the semantics encoded in other layers largely unexploited. Through an empirical analysis, we uncover three key findings: 1) item embeddings from deeper LLM layers suffer from increasingly severe spectral collapse, which reduces their information richness and hinders the optimization; 2) intermediate LLM layers encode complementary item semantics beyond the final layer; 3) different items exhibit heterogeneous layer-wise semantic patterns. These findings suggest that the final layer alone is insufficient and that multi-layer LLM semantics should be exploited adaptively for each item. To this end, we propose IMFuse, a novel Instance-Aware MultiLayer Fusion strategy for LLM-enhanced SR. IMFuse adaptively aggregates embeddings from all LLM layers with learnable layerwise fusion weights, and further customizes the fusion for each item via an instance-aware Mixture-of-Experts (MoE) module. As a plugand-play strategy, IMFuse can be seamlessly integrated into various SR backbones and LLM-based enhancement methods. Extensive experiments on four real-world datasets demonstrate that IMFuse consistently improves state-of-the-art methods, achieving an average relative improvement of 6.72% with limited computational overhead.
作者备注：13 pages, 5 figures
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-04 18:38 · 最新观测版本
IMFuse: Instance-Aware Multi-Layer Fusion for LLM-Enhanced Sequential Recommendation
Leveraging large language models (LLMs) to enrich item embeddings with textual semantics has become a prevailing paradigm for enhancing sequential recommendation (SR). However, existing methods typically rely solely on the final-layer LLM embeddings, leaving the semantics encoded in other layers largely unexploited. Through an empirical analysis, we uncover three key findings: 1) item embeddings from deeper LLM layers suffer from increasingly severe spectral collapse, which reduces their information richness and hinders the optimization; 2) intermediate LLM layers encode complementary item semantics beyond the final layer; 3) different items exhibit heterogeneous layer-wise semantic patterns. These findings suggest that the final layer alone is insufficient and that multi-layer LLM semantics should be exploited adaptively for each item. To this end, we propose IMFuse, a novel Instance-Aware MultiLayer Fusion strategy for LLM-enhanced SR. IMFuse adaptively aggregates embeddings from all LLM layers with learnable layerwise fusion weights, and further customizes the fusion for each item via an instance-aware Mixture-of-Experts (MoE) module. As a plugand-play strategy, IMFuse can be seamlessly integrated into various SR backbones and LLM-based enhancement methods. Extensive experiments on four real-world datasets demonstrate that IMFuse consistently improves state-of-the-art methods, achieving an average relative improvement of 6.72% with limited computational overhead.
查看此版本解读 →查看此版本原文 ↗