Junyoung Kim, Woojoo Kim, Jaehyung Lim, Dongha Kim, Wonbin Kweon, Hwanjo Yu
Sequential Recommendation (SR) learns sequence-level Collaborative Filtering (CF) from interactions, but ID-based models struggle with sparse data and cold-start items. In contrast, item titles and images can reveal preference continuity, such as a recurring color or style. Yet existing approaches miss visual continuity: small encoders take a single item as input, and Large Language Models (LLMs) read only titles. We therefore shift our focus to Large Vision-Language Models (VLMs), which can read a user's entire image sequence, not just its titles. However, VLMs inherently suffer from modality collapse: they over-rely on text and, as a result, overlook visual continuity for recommendation. We find that standard contrastive Supervised Fine-Tuning (SFT), essential for adapting VLMs to SR, further amplifies this collapse: updates follow the text gradient, visual discrimination weakens, and adding image embeddings to the text embeddings consistently lowers accuracy. We propose VLM2Rec with two objectives: Weak-modality Penalized Contrastive Learning strengthens each user's weaker modality, and Cross-modal Relational Topology Regularization aligns each user's relative similarity structure over the same items across the two modality spaces. Its variant, $\text{VLM2Rec}_{\text{plug}}$, reuses the learned item embeddings in SR backbones and thus serves at the latency of conventional SR models. Experiments show that VLM2Rec mitigates the collapse and consistently improves accuracy and robustness.
Jaehyung Lim, Woojoo Kim, Junyoung Kim, Dongha Kim, Wonbin Kweon, Hwanjo Yu
LLM-powered recommender agents use reasoning and memory for personalization, but centralized access to user information raises privacy concerns, while isolated agents cannot learn from others' experiences. We formulate Federated Recommender Agents, a setting where local agents collaborate through natural-language knowledge while keeping interaction histories and personal memories local. We propose FedRecAgent, which enables this collaboration without updating LLM parameters. Its central challenge is determining which reasoning to integrate, since similar behavioral contexts can lead to different choices. FedRecAgent represents local experiences as Condition-Transition-Reasoning (C-T-R) units and hierarchically integrates reasoning by condition and transition. Within this structure, it consolidates related explanations and discriminates reasoning across alternative transitions. Each client maintains semantic and decision memories to capture preferences and conditional choice tendencies, guiding the retrieval and selection of shared reasoning for recommendation. Experiments across three domains under dense and sparse settings demonstrate improved recommendation performance over federated and LLM-based baselines, while ablations support the contributions of personalized memories and structured reasoning integration.