跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文，不再错过。

覆盖计算机全学科，以 AI 为快捷入口。关注方向、作者和关键词，把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+1已有论文修订
+101新增 AI 导读

arXiv 最近完成扫描：2026-11-08 23:55 · 全站最近入库：2026-11-08 06:02各信源进度 →

统计口径与数据新鲜度

统计 2026-11-08 00:05 — 2026-11-09 00:05（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-08 23:55最近发现新增版本：2026-11-08 06:02库内论文最新更新时间：2026-11-06 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

17,100 已收录论文14,661 AI 相关4,057 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 879 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-08 23:54（北京时间）
版本更新 · v4arXiv:2511.20348

Material-informed Gaussian Splatting for 3D World Reconstruction in a Digital Twin

Andy Huynh, João Malheiro Silva, Holger Caesar, Tong Duy Son

原文摘要

3D reconstruction for Digital Twins often relies on LiDAR-based methods, which provide accurate geometry but lack the semantics and textures naturally captured by cameras. Traditional LiDAR-camera fusion approaches require complex calibration and still struggle with certain materials like glass, which are visible in images but poorly represented in point clouds. We propose a camera-only pipeline that reconstructs scenes using 3D Gaussian Splatting from multi-view images, extracts semantic material masks via vision models, converts Gaussian representations to mesh surfaces with projected material labels, and assigns physics-based material properties for accurate sensor simulation in modern graphics engines and simulators. This approach combines photorealistic reconstruction with physics-based material assignment, providing sensor simulation fidelity comparable to LiDAR-camera fusion while eliminating hardware complexity and calibration requirements. We validate our camera-only method using an internal dataset from an instrumented test vehicle, leveraging LiDAR as ground truth for reflectivity validation alongside image similarity metrics.

cs.CV · cs.RO

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2602.23934

Learning to Build: Autonomous Robotic Assembly of Stable Structures Without Predefined Plans

Jingwen Wang, Johannes Kirschner, Paul Rolland, Luis Salamanca, Stefana Parascho

原文摘要

This paper presents a novel autonomous robotic assembly framework for constructing stable structures without relying on predefined architectural blueprints. Instead of following fixed plans, construction tasks are defined through targets and obstacles, allowing the system to adapt more flexibly to environmental uncertainty and variations during the building process. A reinforcement learning (RL) policy, trained using deep Q-learning with successor features, serves as the decision-making component. As a proof of concept, we evaluate the approach on a benchmark of 15 2D robotic assembly tasks of discrete block construction. Experiments using a real-world closed-loop robotic setup demonstrate the feasibility of the method and its ability to handle construction noise. The results suggest that our framework offers a promising direction for more adaptable and robust robotic construction in real-world environments.

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02777

Beyond Policy Patches: Distilling RL Specialists into Vision-Language-Action Policies

Zhennan Jiang, Tingying Xie, Shilong Ji, Shu'ang Yu, Tianxing Zhou, Yuhui Chen, Jiahuan Gao, Kaizhi Xie 等 14 位作者

原文摘要

Vision-language-action (VLA) policies can handle noncritical stages of long-horizon manipulation yet fail during short precision-critical phases. Lightweight reinforcement learning (RL) specialists can address these failures without directly applying RL to the full VLA, but retain auxiliary controllers and switching mechanisms at deployment. We study how to transfer their capabilities into a single flow-matching VLA. Direct imitation of specialist rollouts can reproduce action jitter and leave the VLA unprepared for states encountered during its own execution. We introduce REPAIR, a value-guided distillation framework that combines offline initialization with on-policy refinement. At each critical-phase observation, REPAIR queries the corresponding specialist for multiple action candidates and weights them using its critic to construct a supervision distribution. A shared weighted flow-matching objective trains the VLA first on observations from specialist-assisted rollouts, then on observations collected through its own execution. Specialists provide supervision without taking control during on-policy collection and are removed at deployment. Across six real-robot tasks with 30 evaluation trials per task, REPAIR achieves 100% success on all six tasks, with lower mean end-effector jerk than specialist-assisted execution on every task.

cs.RO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02699

Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up

Joyjeet Singh

原文摘要

We reimplement the dynamics model of the Robotic World Model (RWM) and test the uncertainty estimate that its follow-up, RWM-U, uses to penalize reward in offline policy optimization. Our implementation matches the released model's gradients. On held-out data, training on the model's own rollouts beats one-step training at the preregistered long horizon for all ten episodes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss on sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02693

Source-Resolved Collective Cognition in Robotics: Mathematical Foundations for Multimodal and Collective Estimation, and Their Reach across the Cognition Loop

Juergen Graf

原文摘要

The Generalized Singular Value Decomposition (GSVD) is a central tool for comparing two linear operators that act on the same unknown. Once the output metrics are fixed, it answers the basis-independent question "along which state directions does the first operator dominate the second, and by how much?". This foundational volume builds the GSVD and its surrounding estimation theory from the ground up. Part I re-derives every piece of linear algebra and statistical estimation machinery needed later -- inner products, the four subspaces, orthogonal matrices, the spectral theorem, quadratic forms, the ordinary SVD, Lie-group tangent perturbations, Jacobians, covariance, whitening, Fisher information, CCA, mutual information, discrimination criteria, and covariance intersection -- with self-contained derivations and worked numerical examples. Part II states the GSVD in both a transparent and a dimensionally general form, then proves it by two complementary routes. Part III works one small matrix pair through five different lenses, each computed by hand and verified numerically. Part IV separates information attribution, subspace geometry, statistical dependence, and decision-theoretic discrimination for collective systems. Part V evaluates the role and limits of the GSVD across the cognition loop. Part VI covers numerical practice, including a tested reference implementation. Every number printed in this document was verified in double precision. Where a statement is an assumption rather than a theorem, it is labelled as such.

cs.RO · cs.MA

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02691

Closing the Dynamics Gap through Self-Directed Robotic Interaction

Xilun Zhang, Liam Achenbach, Luis Pabon, Matthew Foutter, Marco Pavone

原文摘要

Agentic Real2Sim2Real systems can reconstruct a scene in simulation from camera observations, but contact-rich dynamics such as friction and restitution cannot be observed without interaction. We call this the dynamics gap and present CUE, an agentic system that closes it by interacting with the environment itself. Given only a high-level instruction and no resets, the agent builds its own perception, identifies physical parameters, and targets interactions where its simulation is least accurate. On billiards, CUE reduces held-out ball-position error by 72% relative to the strongest baseline across 25 hidden-physics conditions in simulation, and we further deploy it on a real pool table. More details can be found on the project website: https://cue-dynamics.github.io/

cs.RO · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02677

Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction

Kaihong Ouyang, Zihao Su, Chengxuan Zheng, Weiyi Lai, Dong Zhou

原文摘要

Long-horizon robotic assembly requires reliable perception and action across grasping, alignment, and insertion. Vision captures global object pose and scene context, where as fingertip touch provides local contact cues that are especially useful as contact states change. We present a visuotactile diffusion policy that combines four-step histories from two camera views and two fingertip tactile images to condition action-sequence generation. A recurrent temporal representation integrates recent observations from both modalities. To encourage it to capture how contact evolves under action, an auxiliary objective predicts fingertip-image changes two steps ahead from the temporal representation and demonstrated actions. This objective is used only during training and adds no inference-time computation. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of $59.0\% \pm 2.6\%$, improving on the visual-only baseline ($42.7\% \pm 1.5\%$) by 16.3 percentage points.

cs.RO

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2604.05484

CoEnv: Code-Driven Exploration in Real-to-Sim Digital Twins for Multi-Arm Manipulation

Li Kang, Yutao Fan, Rui Li, Heng Zhou, Yiran Qin, Zhemeng Zhang, Songtao Huang, Xiufeng Song 等 14 位作者

原文摘要

General-purpose coding agents can now write, debug, and refine entire robot programs, but only if they have somewhere to practice. Physical hardware is a poor practice ground: exploratory rollouts are slow to reset and, when several arms share a workspace, unsafe. Generic simulation benchmarks are safe but do not match the geometry, assets, or viewpoints of the workspace where the robot must actually work. We present CoEx (Code-driven Exploration), which lets the agent practice in an executable digital twin of its own deployment workspace. The twin is built by CoEnv, our real-to-sim framework, which reconstructs the physical scene from multi-view RGBD into a physics simulation. Inside it, a centralized code agent explores at the program level, writing a complete multi-arm program, running it in the twin, and refining it against grounded verification (privileged state checks, checkpoint predicates on risky steps, and adaptive viewpoints against inter-arm occlusion). Verified programs transfer to hardware through collision-checked deployment. On five real-world multi-arm tasks with up to three heterogeneous robots, CoEx reaches 70% average success, against 50% for the same system without the code agent (where a VLM must choose every primitive step by step), outperforms two prior VLM-planning frameworks, and produces verified episodes at 3.8-11.7x the ablated rate. Because every verified rollout is also a demonstration, exploration leaves reusable experience behind: a $π_{0.5}$ policy fine-tuned purely on 200 agent-generated rollouts reaches 100% and 70% real-robot success on two tasks, with no human demonstrations.

cs.RO · cs.CV

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02660

HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation

Gyeonghun Kim, Donghoon Kim, Minsoo Choi, Suyun Lee

原文摘要

Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-relevant history in explicit language through factorized event detection and event-triggered memory updates. We evaluate HeLM across 10 non-Markovian real-world tasks with 56 variants, including five newly formulated inter-episode tasks that require persistent context across task boundaries. HeLM improves intra-episode success by 12.8 percentage points over the strongest baseline and achieves 78% subtask success on compositional generalization tasks, while using up to 4.5x less memory and running 2.5x faster than the keyframe baseline at long horizons.

cs.RO

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2607.15880

Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation

Zhenduo Shang, Xiyao Liu, Bohan Li, Xudong Wang, Teng Ren, Lianqing Liu, Zhi Han

原文摘要

Imitation Learning aims to learn skills from extensive observations and demonstrations for robots, so it suffers from data scarcity and environment generalization. The existing methods predominantly focus on imitation from in-domain tasks and consequently struggle with generalization to unseen tasks. To bridge this generalization gap, we propose the \textbf{D}ynamics-\textbf{A}ware \textbf{M}eta-\textbf{I}mitation (DAMI) framework. By integrating meta-learning to construct a shared skill space, DAMI equips agents for rapid adaptation to novel tasks. We introduce the Visual-Motor Trajectory (VMT) module to capture complex spatio-temporal dynamics within the task latent space. Furthermore, we propose the Unpaired Unified Task (U2T) block to fuse unstructured multimodal observations. To coordinate these representations, we integrate a Task-Conditioned Feature Modulation (TCFM) mechanism customized for modulating low-level 3D features. By capturing intrinsic dynamics from a random complete reference demonstration, our framework learns the underlying task logic rather than memorizing static cues, ensuring effective generalization. Extensive experiments in both simulation and real-world settings demonstrate that our approach outperforms state-of-the-art baselines regarding direct inference on seen tasks and adaptation to unseen tasks via few-shot fine-tuning.

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2601.12428

ReWorld: Multi-Dimensional Reward Modeling for Embodied Video Generation

Baorui Peng, Wenyao Zhang, Liang Xu, Bohan Li, Hu Zhu, Zekun Qi, Hongsi Liu, Jiazhao Zhang 等 10 位作者

原文摘要

Current embodied video generation models primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. In this paper, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the video generation with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K) video preference dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical alignment algorithm that post-trains flow-based video generators using this reward through a computationally efficient PPO-style algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods.

cs.RO · cs.CV

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2609.11308

2AM: Grounding Agent-Side Memory as Guidance for Steerable Action Models in Long-Horizon Manipulation

Yutong Hu, Fengjiao Chen, Xuezhi Cao, Renaud Detry

原文摘要

Long-horizon robot manipulation requires memory, but not necessarily inside the action policy. To address such tasks, current agentic systems often combine VLAs with planners and geometric tools, sometimes using additional depth or calibrated geometry. These systems confound attribution: gains may come from richer observations or alternative motor tools, while failures may stem from either the policy or an under-specified language interface. We isolate this question through a deliberately constrained design: less tool breadth, but greater interface bandwidth. 2AM makes a multimodal Agent the sole holder of task memory and a single RGB-based, episodically stateless Action Model the sole executor of task-relevant motion. The Agent compiles interaction history into subtask language and optional 2D grasp, place, and move hints that bind its physical intention at different time scales. To teach this steerability to the VLA, we augment demonstrations with structured hint labels and train under condition dropout, spatial noise, and temporal jitter to tolerate imperfect Agent outputs. On LIBERO-Mem, without depth, online geometry, or planner-based object motion, 2AM reaches 76.3% average completion, a 61.5-point improvement over the strongest reported baseline of 14.8%, together with 63.0% relaxed and 11.8% strict success. These results show that task memory can remain Agent-side. They further show that Action Model capability depends not only on what the policy has learned, but on how precisely the Agent can steer it. Project website: agent-mem-action-model.github.io

cs.RO · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02626

Neural Integral Control Barrier Functions for Agile Collision Avoidance

Dasari Raj Vamsi, Vrushabh Zinage, Efstathios Bakolas, Rohan Chandra

原文摘要

Control barrier functions (CBFs) are a standard way to keep a planner inside a safe set, but they assume control affine dynamics, a barrier function given in closed form, and a known nominal controller. Integral control barrier functions (ICBFs) remove the first assumption and absorb input constraints, yet they still assume a given nominal controller and a barrier function. In this work, we present an approach to learn all three from data, using neural ordinary differential equations (ODEs) for the dynamics, imitation for the nominal controller, and supervised regression for the barrier function. The learned dynamics, nominal controller, and the barrier function are used to build the ICBF filter constraint analytically. Furthermore, the barrier is trained on viability rather than geometry, meaning that a state counts as safe only if some emergency maneuver can still avoid the constraint under the actuator limits, a correction that geometric labels miss entirely and that grows with speed. Actuator limits then enter the online program as hard bounds while the learned barrier rows may be relaxed, which keeps the program solvable at every state and the actuator limits exact, and we bound how far the input it returns can drift from the one the true models would give. On agile racing with a 6-D dynamic bicycle, the filter removes all 12 unsafe episodes from 200 held out layouts at an unchanged success rate and above 100 Hz on one CPU core, holds the input constraint on every episode of a benchmark where a jointly trained baseline holds it on 53.38%, and keeps that advantage under large errors in the plant friction.

cs.RO · cs.LG · eess.SY

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2605.24044

MERIS: Adaptive Real-Time Scheduling of Refined Inference DAGs in Dynamic Robotic Environments

Zexin Li, Tao Ren, Johnathan Liu, Xiaoxi He, Cong Liu

原文摘要

Robots deployed in dynamic environments execute changing directed-acyclic-graph (DAG) workloads under limited compute and timing budgets. This article presents MERIS, a framework for model scheduling under real-time constraints that combines proportional intermediate deadlines, MIMONet-aware graph refinement, and event-driven synchronization for multi-task inference on resource-constrained robotic platforms. We distinguish three representations: the application DAG, each MIMONet's internal encoder/decoder graph, and the refined scheduling DAG, and make the scope of the timing argument explicit. If the profiled execution bounds, release assumptions, precedence barriers, and per-level capacity premises hold, satisfaction of the assigned sub-deadlines composes to the end-to-end deadline. For non-partitionable tasks, we define a profile-bounded admission rule; before interpreting that rule as empirical timing evidence, we require direct per-task completion checks. We report run-level QoE alongside the offered-node deadline missing rate, making explicit that run completion and successful service of every node are different outcomes. We propose a comprehensive minibenchmark for evaluating the performance of schedulers under various DNN-driven autonomous driving scenarios. Extensive experiments on Nano Orin, AGX Orin, and M3 Max demonstrate the effectiveness of our proposed method.

cs.RO · cs.SE · eess.SY

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2603.03280

How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference

Toru Lin, Shuying Deng, Zhao-Heng Yin, Pieter Abbeel, Jitendra Malik

原文摘要

Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain difficult for autonomous robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human-evaluated task quality. Using only 50~200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.

cs.RO · cs.AI · cs.CV · cs.LG · eess.SY

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02613

MAPO: Mixture of Experts with Alternating Policy Optimization for Universal Humanoid Motion Tracking

Seonghyeon Lim, Hyeonwoo Lee, Mincheol Kim, Kimin Lee, Hyun Myung

原文摘要

Universal humanoid motion tracking is essential for enabling humanoid robots to perform diverse tasks in human environments. To capture the wide diversity of human motions, mixture-of-experts (MoE) policies have been widely adopted, leveraging dynamic routing to enhance policy expressiveness. However, we identify that divergence between old and updated policies can be misestimated during policy-gradient updates of MoE policies, due to their dynamic routing. Because most policy-gradient algorithms regulate policy updates based on the divergence, this misestimation can lead to either overly conservative or excessively large updates, which can yield a suboptimal policy. In this paper, we propose Mixture of Experts with Alternating Policy Optimization (MAPO), which alternately optimizes the router and the experts to avoid the divergence misestimation. MAPO consistently tracks 182 motions that PPO with MoE fails to track, whereas PPO with MoE tracks only one motion that MAPO fails to track. MAPO also attains the highest tracking success rate among the compared methods and the lowest tracking error across most error metrics, demonstrating its superior performance. Videos can be found on the project website: https://mapo-corl2026.github.io/MAPO/

cs.RO

查看论文
arXiv ↗PDF ↗
版本更新 · v4arXiv:2602.02220

LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation

Bo Miao, Weijia Liu, Jun Luo, Lachlan Shinnick, Jian Liu, Thomas Hamilton-Smith, Yuhe Yang, Zijie Wu 等 11 位作者

原文摘要

Language-conditioned goal navigation requires embodied agents to locate user-specified targets without step-by-step guidance. However, existing benchmarks largely focus on category-level goals or rely on instance descriptions generated by vision-language models, which often contain ambiguities and semantic errors, limiting systematic and reliable evaluation. We introduce HieraNav, an open-vocabulary navigation task with goals at four hierarchical semantic levels: scene, room, region, and instance. We present Language as a Map (LangMap), the first goal navigation benchmark to enrich real-world indoor 3D scans with human-verified semantic annotations supporting tasks across all four goal levels. Built on HM3D using a contrastive annotation protocol that compares same-scene regions and instances, LangMap provides region labels and discriminative region and instance descriptions covering 414 object categories and contains over 18K tasks. Each target has concise and detailed descriptions, enabling evaluation across instruction styles. Automated and human evaluations validate our annotation quality: our descriptions improve text-to-view matching accuracy over GOAT-Bench's by 23 points on all shared annotated instances, and an independent human audit yields 92.5% unique-and-correct matches. We also propose PlaNaVid, an RGB-only baseline that combines Bounded Diverse Memory with high-level planning to prime a reactive policy for multi-goal navigation, achieving top-tier success rates without depth, 3D scene representations, or object masks. Further analyses reveal that exploration and hierarchical disambiguation failures become more prominent at finer goal levels, while long-tail categories, small objects, distant targets, timely stopping, and multi-goal completion remain challenging. Benchmark and code: https://bo-miao.github.io/LangMap

cs.CV · cs.RO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02600

W2A: Distilling World Priors into Action Models for Real-Time Robotic Control

Xiaopeng Zhang, Yongjin Mu, Yueyang Weng, Chengzhi Zhao, Yanjie Li

原文摘要

World Action Models (WAMs) have recently shown great promise in robotic control, benefiting from the future imagination capability of large-scale pretrained video models. However, most existing WAMs require explicitly imagining future states at test time, incurring substantial computational overhead and latency that make them unsuitable for real-time robotic control. In this paper, we propose W2A, a two-stage distillation framework for real-time robotic manipulation that distills intermediate denoising representations from a video diffusion model into an action model during training, while entirely discarding the video model at test time. We hypothesize that although video latent representations are high-dimensional, a compressed low-dimensional subspace is sufficient to provide effective supervision for action generation. This makes it possible to transfer such information effectively into an action model. Extensive experiments in both simulation and real-world settings demonstrate competitive success rates while significantly improving inference efficiency. On LIBERO, W2A achieves a 95.9% average success rate. More importantly, W2A achieves about 21x faster inference than the representative WAM baseline.

cs.RO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02564

Central pivot point control of a body-attached spring-mass runner: periodic gaits, stabilization, template embedding and approximate analytic return maps

İzel Sever, O. Kaan Karagöz, M. Mert Ankaralı

原文摘要

The spring-loaded inverted pendulum (SLIP) describes the center-of-mass (COM) motion of runners but has no trunk. In trunk models, the hip torque couples posture and COM motion and destroys the central-force structure on which SLIP analysis relies. As a result, trunk models have been studied almost only numerically. We show that this structure can be restored. The central pivot point (CPP), a hip-torque law that directs the ground reaction force through the COM, decouples pitch exactly and reduces stance to a planar central-force problem. At upright posture, time-reversal symmetry produces one-parameter families of periodic gaits. A stance-phase pitch law and a stride-level regulator stabilize all computed gaits, also with bounded hip torque and friction. Under CPP, any radial force law in virtual-leg coordinates can be embedded independently of posture. For a linear radial template, the stance radius and the constant-momentum sweep have exact closed forms. With a new gravity correction, the resulting analytic return map, to our knowledge the first for a trunk model, predicts liftoff velocity and apex position with mean errors of 1.6% and 0.9%. The natural gaits of the embedded runner are symmetric and need no force step at touchdown. A near-reversibility argument explains their unit multiplier, and a scalar equation predicts them within 0.8%. A deadbeat stride controller based on the analytic map tracks changes of speed, height and posture. CPP thus makes trunk models accessible to the analytic tools developed for SLIP.

cs.RO · eess.SY · math.DS · physics.bio-ph

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2607.15659

Harnessing Plastic Deformation for Continuous Passive Stability in Soft Grippers

Junlong Xiao, Yaoqiang Pan, Xuan Zhang, Michael Yu Wang, Chao Chen

原文摘要

Soft robots offer compliance and adaptability for interacting with complex environments, but their low structural stiffness can compromise stability under dynamic disturbances. Here, we present a continuously stable structure through plastic deformation (CSSPD), a hybrid metal-elastomer architecture that enables passive shape retention without continuous energy input. By deliberately exploiting plastic deformation in a kirigami-patterned metal layer, CSSPD establishes new equilibrium configurations while increasing structural stiffness and resistance to dynamic disturbances. A theoretical model is developed to characterize the deformation and stability of the hybrid structure and guide the geometric design of the metal layer. We further introduce a bioinspired paw pad that provides top-side support for enhanced grasp stability and serves as an integrated sensing interface for rapid object acquisition. A telescopic soft pressing mechanism (TSPM) mechanically couples paw-pad motion with tendon actuation. Experiments demonstrate passive holding forces of up to 16 N without continuous energy input, comparable to pneumatic actuation at 0.3 MPa, and stable grasping under pulsed accelerations of up to 400 m/s^2 when combined with pneumatic actuation. Energy-free perching on tree branches further demonstrates the potential of the proposed composite architecture for stable and energy-efficient manipulation.

cs.RO

查看论文
arXiv ↗PDF ↗
版本更新 · v3arXiv:2610.31028

Multimodal Progress-Residual Fusion for Anticipating Navigation Retreat in Robotic Flexible Endoscopy

Chi Kit Ng, Tianchun Wu, Jiachen Sun, Yuming Sun, Jianru Zhang, Hongliang Ren

原文摘要

Flexible robotic endoscopes provide access through tortuous anatomy, where friction, deformation, and wall interaction separate commanded insertion from useful distal progress. Operators interpret this mismatch and retreat to recover a navigable configuration. Recognizing the approach to that decision requires relating motor intent to distal response and the local visual scene. We introduce Multimodal Progress-Residual Fusion (MPRF), a causal model that anticipates operator retreat from commands, electromagnetic (EM) tracking, and endoscopic video. A nominal-progress model predicts the response expected in normal or successful navigation; a temporal predictor combines its EM residual with all three observed modalities. In a 150-episode colon-phantom collection, MPRF achieves 0.580 +/- 0.007 AUPRC across three seeds, compared with 0.511 for the strongest reduced-modality control. Complete multimodal histories provide the principal ranking advantage, while the residual adds an explicit intent-response representation with a small, uncertain increment over raw fusion. Analysis of frozen alarms relates warning timing to the behavioral label and explains the increase in event recall under a longer matching window. These results establish a basis for behavioral assistance during operator-controlled navigation.

cs.RO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02555

Sequential Recovery Updates for Learned Controllers under Successful-Trajectory Preservation

Zongtan Li, Tianming Liu, Ye Zhao

原文摘要

A competent learned controller already recovers from many disturbances. Improving it means learning a correction, and learning when not to apply it. Freezing a skill's parameters does not settle the second half. In a controller that routes among frozen skills, a new entry rule can redirect the states an old skill succeeded on. We therefore constrain entry instead of parameters. Each update searches for a feedback skill at a measured failure. It fits an entry rule that reads observations and rejects every state on an accumulated successful trajectory. The skill is admitted only if the composed controller reproduces the new recovery on its own and replays every protected trajectory exactly. A finite-trajectory condition states when this preserves the previous controller. One mechanism runs on four streams: two motions on each of two humanoid systems. Two updates per stream add 62 recoveries over 192 withheld perturbations, and lose none of the base controller's successes. We then price that guarantee. Removing the constraint recovers 33 more conditions but stops eight protected ones from completing at all, and excluding only what admission proves necessary recovers none of that cost.

cs.RO · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02546

Energy-Based Policy Orchestration

Weirui Ye, Yen-Jen Wang, Elvis Hsieh, Zhiyang Dou, Shibo Zhao, Pieter Abbeel, Yilun Du, Leslie Pack Kaelbling 等 9 位作者

原文摘要

Hierarchical control scales robot behavior by separating task-level planning from low-level control. In dynamic, high-dimensional tasks, however, a fixed planner-to-policy interface can fail when contacts, balance, or disturbances evolve faster than high-level policy updates. Execution therefore requires choosing not only which low-level policy to run, but also which policy-specific goal should be given to it. We formulate this problem as policy orchestration: runtime selection of a policy-goal pair that is likely to remain feasible while tracking the planner target. We develop EPO, an energy-based policy orchestrator for modular goal-conditioned policy families. For each policy, EPO learns a composite energy over low-level goals, where lower energy corresponds to higher feasibility and more accurate target realization. At runtime, EPO refines each policy's goal and executes the lowest-energy policy-goal pair. Because energies are policy-specific, new policies can be incorporated by learning their own energy models, without retraining the existing policies or a centralized scheduler. In simulation, EPO improves the empirical precision-robustness trade-off by refining policy-specific goals before selectively switching among policies based on energy scores. We further demonstrate sim-to-real deployment of the same frozen policies and energy models on real humanoid loco-manipulation tasks. Project website: https://yewr.github.io/epo

cs.RO · cs.LG

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02537

Interactive 3D Cutaway Visualization for Subsurface Target Localization in Robot-Assisted Surgery

Yuxin Chen, Wenxuan Wang, Wanwen Chen, Qi Zeng, Septimiu E. Salcudean

原文摘要

Purpose: Blended augmented reality (AR) overlays can provide ambiguous depth cues in which subsurface anatomy floats above the tissue surface. We investigate whether an interactive 3D cutaway visualization can improve robot-assisted target localization. Methods: We propose Virtual Box, an interactive 3D cutaway interface that allows users to independently manipulate the viewing perspective to inspect subsurface targets. In a controlled, counterbalanced within-subject tissue phantom study, 15 engineering students used Virtual Box and a depth-enhanced blended AR overlay to position a da Vinci instrument at virtual targets in a water tank. We used registered 3D transrectal ultrasound to measure 3D localization error and depth error. We also collected completion time and subjective ratings. Results: Virtual Box reduced mean 3D error from 12.92 to 4.96 mm (61.6% reduction) and depth error from 9.48 to 2.07 mm (78.2% reduction; both p < 0.001). Spatial understanding, depth perception, and confidence ratings were higher (all unadjusted p < 0.001), but completion time increased from 21.64 to 32.58 s (p = 0.008). Overall workload did not differ significantly. Conclusion: Virtual Box improved localization accuracy and perceived spatial understanding in this controlled task, with additional completion time, supporting further evaluation for surgical navigation.

cs.RO

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2610.19431

$S^3$-Nav: Structured Spatial-Semantic Decision Making for Language-Guided Robot Navigation

Chaemin Lee, Hae Min Cho

原文摘要

Language-guided robot navigation under partial observability requires robots to reason over spatial-semantic evidence accumulated across observations, particularly when goals involve spatial relations between objects. This paper presents $S^3$-Nav, a structured spatial-semantic decision-making framework that integrates online mapping, persistent object--relation memory, and goal-conditioned navigation. Using RGB-D observations and SLAM, $S^3$-Nav maintains a persistent object--relation memory containing object records and confirmed relation records alongside an evolving navigation context containing exploration frontiers, without requiring a pre-existing map. A goal-conditioned evidence interface selects task-relevant evidence from this structured state and constructs a capacity-controlled decision view as part of the high-level policy input for exploration, target approach, and stopping decisions. The persistent memory remains available for subsequent reasoning and verification even when some of its records are excluded from the decision view. To ensure grounded execution, provisional policy decisions are independently validated against persistent object--relation memory and current geometric evidence before physical execution or task termination. Evaluations across multiple environments and object- and relation-goal tasks show improved closed-loop navigation reliability and reduced language-model token consumption with capacity-controlled evidence exposure under the evaluated inference configuration.

cs.RO

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02525

Touching Projected Agents through the DAT Screen -System Design and Questions of Interaction Boundaries-

Takaya Tamamoto, Tomoko Yonezawa

原文摘要

Touching a virtual agent involves more than detecting a finger on a display: the contact surface can deform and resist the user's push, and the agent can respond to that action. We describe the Desktop Amid Touch (DAT) screen, which combines spatial projection onto a semi-transparent, deformable mesh with touch sensing and motorized control of resistance to indentation. An initial prototype detected a single touch and estimated its position and depth from tension changes around a flexible projection screen. A subsequent prototype added motorized tension control to vary the screen's resistance to indentation. We describe the tension-controlled DAT implementation and its demonstrations, distinguishing what has been observed from what remains an open design question. These capabilities raise a design question: should touchable virtual agents respond as though they had preferences or boundaries, and under what conditions should such behavior be implemented on systems like DAT. The present paper describes a system and frames this question; it does not establish how users interpret an agent's refusal or consent.

cs.HC · cs.AR · cs.RO · eess.SY

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02517

A Study on the Effects of Finger Movements on a Humanoid Hugging Robot

Daigo Suhara, Tomoko Yonezawa

原文摘要

This study aimed to examine the impact of finger movements in hugging robots on users by experimentally investigating how the presence and patterns of such movements affect users psychologically. The results suggest that the presence and patterns of finger movements influence the sensations users seek from the robot; specifically, repetitive movements and asynchronous movements across multiple fingers tended to enhance these sensations. Additionally, simultaneous movement of all fingers and gentle hugging tended to evoke a greater sense of care. Conversely, no differences were observed regarding resistance to physical contact initiated by the robot.

cs.RO · cs.AR · cs.HC

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02510

Neuro Inspired Robotics: What Robot Learning Can Borrow from the Sensorimotor Brain

Ruihan Zhang, Davy Deng

原文摘要

Robot policies still fail at everyday manipulation: an unfamiliar task, an object never held before, a scene that changes mid-motion, a skill that should improve with use. They fail despite pretraining on over a million trajectories, pooled from dozens of robots at great expense, while a child acquires all of these abilities within a few years of learning from a single body. This survey argues that the gap is one of design, and proposes designs for the next generation of robot models, drawn from how the nervous system solves the same problems. From the body we take compliance: the policy commands stiffness alongside position, so the controller yields where the task allows and presses where it must, which greatly lowers the demand for precise position output. From the visual system, where a fast pathway for action runs in parallel with the slow pathway for recognition, we take a dedicated fast vision system for motor behavior: it precomputes the quantities the policy needs and drives a reactive policy at camera rate. From the cerebellum we take a predictor beside the policy that forecasts the consequence of each command and recalibrates from its own error. From motor cortex we take a single model that composes action units from primitives, verifies each at a sensory boundary, practices the weakest unit with a hypothesis in mind, and reasons each attempt closer to the goal. Proven primitives are then consolidated from code into weights, so they run fluently and respond quickly the next time. Together these describe a robot that acts on its own initiative, the basis of deployment and recursive self-improvement, and we offer each proposal as a hypothesis to be tested.

cs.RO · cs.AI

查看论文
arXiv ↗PDF ↗
新论文 · v1arXiv:2611.02497

RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild

Tan Sang Nguyen, Yi Ru Wang, Shailesh Shailesh, Gang Yang, Jiamo Hu, Jianuo Jiang, Haoquan Fang, Zhongzheng Ren 等 27 位作者

原文摘要

General-purpose robot policies must work across robots, tasks, and environments, yet the variation that makes physical trials informative also makes them difficult to reproduce. We introduce RoboColosseum, a distributed evaluation framework that combines unified policy serving and standardized protocols across a network of locally operated robots in diverse environments. Our initial study evaluates five vision-language-action (VLA) policies and three frontier AI agents across four embodiments (SO-101, Franka, Bimanual-YAM, and G1 Humanoid). The Open Track compares models without task-specific adaptation in blinded, pairwise trials under matched local conditions. The Fine-tune Track measures how well VLA policies adapt to 20 tasks across all four embodiments when each policy receives the same 100 demonstrations per task. Across 5,699 pairwise Open Track comparisons at 124 sites, frontier agents rival specialized VLA policies, while all systems struggle to complete open-world tasks reliably. After fine-tuning, GR00T N1.7 leads on SO-101 and pi0.5 leads on Franka and Bimanual-YAM. Policy rankings show weak agreement across embodiment-specific task suites (Kendall W = 0.24). Open Track rankings poorly predict relative performance after fine-tuning, with substantial rank reversals on all three embodiments shared between tracks. Measuring partial progress reveals differences that task success alone misses. On the G1 Humanoid, no policy completes any of the five tasks, yet average progress ranges from 13.3% to 23.2% across policies. Open Track rankings stabilize as trials accumulate, and active matchmaking achieves comparable ranking agreement with substantially fewer physical trials than uniform sampling. These results show how distributed physical evaluation can reveal where robot policies generalize, how well they adapt to new tasks, and how far they progress on tasks they fail to complete.

cs.RO · cs.AI

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2606.22174

OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

Yingdong Hu, Haodong Zhu, Boyuan Zheng, Yihang Hu, Tong Zhang, Zunhao Chen, Junming Zhao, Ruiqian Nai 等 9 位作者

原文摘要

Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and $Ψ_0$) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].

cs.RO

查看论文
arXiv ↗PDF ↗