Why Freeze the Encoder? A Controlled Study of End-to-End Sequence-Graph Learning for User Behavior
Yuen Chen, Yulun Wu, Samuel Sharpe, Igor Melnyk, Nam H. Nguyen, Furong Huang, C. Bayan Bruss, Rizal Fathony
原文摘要
Modeling user behavior couples two distinct signals: a per-user sequence of events and a persistent user-to-user relation graph. Standard pipelines preserve these native formats by combining them in two stages: a sequence encoder is pretrained, its per-user embeddings are frozen, and a graph model is trained on top to allow for cheap serving and cross-task reuse. We ask a simple question: does training the sequence and graph models end-to-end help, and if so, when? To isolate the impact of the training strategy, we hold a standard sequence-encoder-to-GNN pipeline fixed and vary only the optimization method; on Brightkite, a matched control that starts both pipelines from the same pretrained encoder and changes only whether it is updated attributes a 10.6-point MRR gain to end-to-end updating. Across relationship prediction and fraud detection on five real-world datasets, we find that the benefit of end-to-end training is conditional: it improves clearly over frozen two-stage pipelines on relationship prediction, where the relation graph appears to carry much of the signal, whereas gains shrink where the predictive signal is largely node-local (fraud detection), a regime in which a text-fed relational graph transformer achieves competitive results. Furthermore, recasting the data as a temporal graph yields inconsistent results compared to this native sequence-graph formulation. Ultimately, this work provides a controlled empirical account of when jointly training the two structures helps.
cs.LG · cs.AI