arXiv:2505.04978 · v2
A Tactile-Informed Bi-Level Framework with Complementary Motion-Contact Planning and Tracking for Robust Model-Based In-Hand Manipulation
摘要
In dexterous in-hand manipulation, learning-based methods achieve strong task performance with increasing generalization to unseen objects. A complementary advantage of model-based methods, central to this work, is online adaptation to new task specifications without policy retraining, given suitable system models. However, due to the complexity of physical contacts and limited sensing modalities, existing model-based methods still struggle with modeling errors, even when using advanced contact models or optimization schemes, which limit their real-world applications. To advance the effectiveness and robustness of model-based contact-rich in-hand manipulation, this paper proposes a novel bi-level framework with tactile incorporation to mitigate these limitations. Specifically, at the high level, reference finger motions and contacts are jointly generated via contact-implicit model predictive control. The high-level module enables online reactive planning and disturbance recovery. At the low level, these references are concurrently tracked through a hand force-motion model and tactile feedback. The low-level module compensates for modeling errors and enhances manipulation robustness. The complementarity between the two levels is manifested in two dimensions: 1) subsystem decomposition, with a slow optimization-based planning layer and a fast tactile-reactive control layer; and 2) functional decomposition, combining task-level optimality with contact-level stability. Extensive experiments demonstrate that our approach outperforms existing model-based methods in accuracy and robustness, while supporting online replanning during execution. Our method successfully completes a diverse set of challenging tasks in real-world environments, even under appreciable external disturbances and modeling uncertainties.
作者备注：Accepted for publication in The International Journal of Robotics Research (IJRR). Project website: https://director-of-g.github.io/in_hand_manipulation_2/