← 研踪 PaperTrailarXiv:2503.13344 · v3
DeepSTEP: Simultaneous Tracking and Estimation of Pose for Animals and Humans
首次提交 2025-03-18 00:22
版本更新 2026-11-05 03:41
雷达观测 2026-11-06 11:06
摘要
Maintaining a consistent identity across video frames while estimating pose is crucial for downstream tasks such as behavior analysis and action recognition, yet existing pipelines largely decouple tracking and pose estimation. Top-down methods rely on per-frame detections followed by identity association, leading to increased computational cost and error propagation, whereas bottom-up approaches often struggle to preserve identity reliably in multi-instance settings. To address these limitations, we introduce DeepSTEP (STEP), a Transformer-based framework for joint tracking and pose estimation across diverse animal species and humans. STEP unifies localization, identity maintenance, and keypoint estimation into a single model, eliminating the need for per-frame detections or post hoc matching. Our approach leverages discriminative model weight prediction to propagate target identity across frames while predicting anatomically meaningful keypoints. Discriminative model prediction typically requires predefined target states. In contrast, STEP assumes only an initial bounding box in the first frame, removing the need for explicit keypoint initialization. This is enabled by two components: Gaussian Map Soft Prediction (GMSP) and Offset Map Regression Adapter (OMRA), which allow direct keypoint prediction and simplify inference. Starting from the initial bounding box, STEP tracks the subject and estimates its pose in subsequent frames. We train and evaluate STEP on datasets spanning multiple species, demonstrating strong performance and greater efficiency than existing methods. By jointly modeling tracking and pose, STEP provides an effective and scalable solution.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-05 03:41 · 最新观测版本
DeepSTEP: Simultaneous Tracking and Estimation of Pose for Animals and Humans
Maintaining a consistent identity across video frames while estimating pose is crucial for downstream tasks such as behavior analysis and action recognition, yet existing pipelines largely decouple tracking and pose estimation. Top-down methods rely on per-frame detections followed by identity association, leading to increased computational cost and error propagation, whereas bottom-up approaches often struggle to preserve identity reliably in multi-instance settings. To address these limitations, we introduce DeepSTEP (STEP), a Transformer-based framework for joint tracking and pose estimation across diverse animal species and humans. STEP unifies localization, identity maintenance, and keypoint estimation into a single model, eliminating the need for per-frame detections or post hoc matching. Our approach leverages discriminative model weight prediction to propagate target identity across frames while predicting anatomically meaningful keypoints. Discriminative model prediction typically requires predefined target states. In contrast, STEP assumes only an initial bounding box in the first frame, removing the need for explicit keypoint initialization. This is enabled by two components: Gaussian Map Soft Prediction (GMSP) and Offset Map Regression Adapter (OMRA), which allow direct keypoint prediction and simplify inference. Starting from the initial bounding box, STEP tracks the subject and estimates its pose in subsequent frames. We train and evaluate STEP on datasets spanning multiple species, demonstrating strong performance and greater efficiency than existing methods. By jointly modeling tracking and pose, STEP provides an effective and scalable solution.
查看此版本解读 →查看此版本原文 ↗