← 研踪 PaperTrailarXiv:2607.29180 · v2
MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation
首次提交 2026-07-31 17:01
版本更新 2026-11-15 13:41
雷达观测 2026-11-17 19:15
摘要
Which latent space should a text-to-motion generator model? We show that a self-supervised motion representation, once compressed, is a better generative space than latents learned by reconstruction. Our approach, MoRAE, is simple: we pre-train a Motion-JEPA encoder that predicts hidden joint trajectories from the rest of the body, and compress its frozen features into a compact latent for a standard flow transformer. Two observations drive this design. First, prediction yields features that capture action semantics and cross-joint structure better than reconstruction. Second, these features are ill-conditioned, and a standard flow transformer generates poorly in them directly. Compressing them into a compact latent makes generation reliable and places cross-joint structure on the directions the flow models most accurately. With a 58M DiT and the standard CLIP text condition, MoRAE achieves state-of-the-art FID on HumanML3D and KIT-ML while maintaining retrieval-based text consistency and diversity close to real motion.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-15 13:41 · 最新观测版本
MoRAE: Flow-Friendly Self-Supervised Latents for Text-to-Motion Generation
Which latent space should a text-to-motion generator model? We show that a self-supervised motion representation, once compressed, is a better generative space than latents learned by reconstruction. Our approach, MoRAE, is simple: we pre-train a Motion-JEPA encoder that predicts hidden joint trajectories from the rest of the body, and compress its frozen features into a compact latent for a standard flow transformer. Two observations drive this design. First, prediction yields features that capture action semantics and cross-joint structure better than reconstruction. Second, these features are ill-conditioned, and a standard flow transformer generates poorly in them directly. Compressing them into a compact latent makes generation reliable and places cross-joint structure on the directions the flow models most accurately. With a 58M DiT and the standard CLIP text condition, MoRAE achieves state-of-the-art FID on HumanML3D and KIT-ML while maintaining retrieval-based text consistency and diversity close to real motion.
查看此版本解读 →查看此版本原文 ↗