跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.12804 · v1

Benchmarking Real-World Robotic Manipulation through Stochastic Challenges

通过随机挑战对真实世界机器人操作进行基准测试

首次提交 2026-11-10 02:10
版本更新 2026-11-10 02:10
雷达观测 2026-11-10 15:22
arXiv ↗PDF ↗

摘要

A challenging bottleneck in manipulation research is reproducibility and comparability of methods in real-world environments. Since manipulation is an interactive task, it cannot be evaluated on a static dataset of observations. Benchmarking in simulation is a common remedy, but cannot cover the whole complexity of real-world tests. This paper proposes a new paradigm of stochastic challenges. Like the well-known YCB objects, it is based on reproducible real-world scenes and therefore allows system-level innovation and comparability between different platforms. Due to their stochastic nature, our challenges however, mimic the rigor and surprise of hidden test sets. To this end, we show how different robots in different labs can be compared based on their success rate over different random task samples, if these samples come from the same distribution. We propose stochastic challenges as a general paradigm for robot benchmarking and implement this paradigm in a novel benchmark for mobile manipulation, on which we compare two robots tested in two different labs.

作者备注:Accepted at the 10th Conference on Robot Learning (CoRL 2026). 15 pages, 3 figures

AI 论文解读

基于摘要 · 非全文审阅

论文提出“随机挑战”这一真实世界机器人操作基准测试新范式,并以移动操作任务为例,在不同实验室的两台机器人上进行比较。

正在读取账号与研究记录…

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-10 02:10 · 最新观测版本

Benchmarking Real-World Robotic Manipulation through Stochastic Challenges

A challenging bottleneck in manipulation research is reproducibility and comparability of methods in real-world environments. Since manipulation is an interactive task, it cannot be evaluated on a static dataset of observations. Benchmarking in simulation is a common remedy, but cannot cover the whole complexity of real-world tests. This paper proposes a new paradigm of stochastic challenges. Like the well-known YCB objects, it is based on reproducible real-world scenes and therefore allows system-level innovation and comparability between different platforms. Due to their stochastic nature, our challenges however, mimic the rigor and surprise of hidden test sets. To this end, we show how different robots in different labs can be compared based on their success rate over different random task samples, if these samples come from the same distribution. We propose stochastic challenges as a general paradigm for robot benchmarking and implement this paradigm in a novel benchmark for mobile manipulation, on which we compare two robots tested in two different labs.

查看此版本解读 →查看此版本原文 ↗