← 研踪 PaperTrailarXiv:2603.03280 · v2
How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference
首次提交 2026-03-04 02:59
版本更新 2026-11-02 14:47
雷达观测 2026-11-04 23:42
摘要
Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain difficult for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human-evaluated task quality. Using only 50~200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.
作者备注：Project page can be found at https://toruowo.github.io/peel
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-02 14:47 · 最新观测版本
How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference
Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain difficult for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human-evaluated task quality. Using only 50~200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.
查看此版本解读 →查看此版本原文 ↗