TaoMate-R1:面向流式多参考视频生成的时间角色专门化
TaoMate-R1: Temporal Role Specialization for Streaming Multi-Reference Video Generation
Qijun Gan, Chenwei Zhang, Xinyang Li, Meiguang Jin, Junfeng Ma
AI 速读 基于摘要
作者提出按“场景建立”与“场景延续”两种时间角色分配低秩适配器的流式多参考视频生成器,并在自建多参考案例上报告了边界一致性与时序相似度方面的改进。
cs.CV
查看原文摘要
Streaming multi-reference video generation establishes a scene from specified people and objects, then continues it under changing instructions. We introduce TaoMate-R1, a streaming multi-reference generator that organizes adaptation parameters through temporal role specialization. It assigns separate low-rank adapters to scene establishment and continuation on a frozen backbone. Each adapter performs both denoising and context writing for its role; Self Gradient Forcing jointly trains the roles through the history they construct. This allocation contrasts with operation-based specialization, which separates denoising from context writing across all chunks. A fixed-replay gradient analysis characterizes the update directions admitted by the two sharing schemes. On 32 multi-reference cases, we compare the schemes at the same two-adapter budget. Across ten approximately two-minute streams with changing instructions and shared teacher-audio conditioning, TaoMate-R1 reduces the mean boundary luminance jump from 1.96 to 0.613, with higher boundary feature similarity. It also achieves higher mean CLIP and DINO temporal similarity across short interactions, long streams, and public reference compositions. These results support grouping adaptation parameters by scene establishment and continuation for streaming multi-reference generation. Our code and model are released at https://github.com/TaoLiveAIGC/TaoMate-H3.