EmbodiedLab: Beyond Manipulation, Benchmarking Embodied Agents That Act to Know
Hongcheng Gao, Zelin Zheng, Alex Zhou, Jay Zhu, Shijia Ge, Yazhe Wang, Xiping Li, Jianshu Zeng 等 21 位作者
原文摘要
Physical action not only changes the state of an environment but is also the way to know its hidden properties: what cannot be seen must be inferred from the feedback to the change. As embodied agents advance from executing procedures to pursuing goals, success depends on properties neither given nor visible, so they must act to know. Existing benchmarks separate changing from knowing: manipulation benchmarks give the target state, so nothing has to be found out, whereas exploration and discovery benchmarks let the agent find out by looking or querying, so nothing physical has to be changed. Quantitative science experiments offer a natural testbed: the properties sought are new in every instance, appear only in instrument readings under conditions the agent sets up, and can be checked exactly. We introduce EmbodiedLab, a simulated laboratory of 35 such experiments. Given only a goal, the agent must decide what to measure, operate the apparatus with a robot arm, read instruments through cameras and judge when its readings suffice. Its 30 determination tasks specify what to find out, whereas its 5 optimization tasks ask only for the best result the apparatus allows. With native control, the strongest of seven frontier agents succeeds on 41.4% of the tasks. Ablations reveal two bottlenecks: contact-rich manipulation, where ground-truth depth and motion planning change the success rate by at most 10 points, whereas executing every operation for the agent on five of the tasks raises it from at most 20% to at least 70%, so what an agent can know is bounded by how reliably it acts; and closed-loop experimental rigor, where with manipulation supplied, agents fail to gather sufficient readings or read instruments accurately, and most optimization runs stop at one design. Beyond evaluation, EmbodiedLab offers a resamplable environment with verifiable rewards for training embodied agents.
cs.RO