跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+549新增论文
+443其中 AI 相关
+62已有论文修订
+95新增 AI 导读

arXiv 最近完成扫描:2026-11-21 12:19 · 全站最近入库:2026-11-21 12:19各信源进度 →

统计口径与数据新鲜度

统计 2026-11-20 19:34 — 2026-11-21 19:34(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-21 12:19最近发现新增版本:2026-11-21 12:19库内论文最新更新时间:2026-11-20 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

29,158 已收录论文24,181 AI 相关8,992 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 2,061 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-21 12:17(北京时间)
版本更新 · v2arXiv:2603.04757

Gait Generation Balancing Joint Load and Mobility for Legged Modular Robots with Easily Detachable Joints

Kennosuke Chihara, Takuya Kiyokawa, Tetsuya Shitara, Kensuke Harada

原文摘要

Modular robots can adapt their morphology to diverse tasks, but locomotion can impose large loads on detachable joints. This paper presents an NSGA-III-based multi-objective gait optimization framework with three objectives: a locomotion speed, a static stability, and a normalized joint-load metric based on joint reaction forces. The framework yields nondominated gait candidates that reveal the trade-offs among these objectives. In a flat-terrain comparison using the 6-legged tetrapod gait, adding the joint-load objective reduced the mean maximum joint reaction force by 11.5% and increased the mean minimum stability by 41.2%, while decreasing the mean forward speed by 10.5%. Simulations under flat, slope, and step conditions showed that locomotion feasibility depended on the combination of robot configuration, gait, and terrain. In physical experiments, the 4-legged trot completed locomotion on flat terrain and a 10 degrees slope, whereas the 6-legged tetrapod and tripod completed locomotion on flat terrain. These results demonstrate the feasibility of joint-load-aware gait generation based on explicit trade-offs among the three evaluated quantities

cs.RO

新论文 · v1arXiv:2611.20465

ReRead: Wrong Predictions, Useful Representations - Correcting Frozen Robot Policies by Re-reading Their Own Representations

Xu Su, Shunlong Xiao, Qintao Yi, Mingkuan Zhao, Xinmiao Yu, Yizhe Li, Zihan Wang, Bokui Chen

原文摘要

When a pretrained robot policy fails on challenging tasks, common remedies include retraining the backbone, adding an external visual encoder, or introducing a memory module. We argue that wrong predictions do not imply wrong representations: a policy can retain the required information yet fail to read it out as the correct action. To test this view, we propose ReRead: with the backbone frozen, a corrector re-reads the current and two previous caches to predict an action residual. The corrector sees no images, is trained solely on offline demonstrations from all 50 RoboTwin 2.0 tasks, and adds only 4.17% latency. Ablations and interventions show that the correction relies on the backbone's own representations: none of four external visual encoders adds further gain, and swapping in past caches from other trajectories sharply increases action error. On the 11 hardest RoboTwin 2.0 tasks, ReRead improves the success rates of LingBot-VLA, $π_{0.5}$, and Fast-WAM by 23.45, 13.09, and 8.00 percentage points, respectively. In paired evaluation on all 50 tasks, it raises Fast-WAM's average success rate from 90.73% to 92.66%. On a Flexiv dual-arm robot, it raises success on a phase-ambiguous task from 0% to 36%.

cs.RO · cs.AI

版本更新 · v3arXiv:2610.24122

HAVANA: A Hierarchical Video-Action Model for Vision-Language Navigation

Dongzhihan Wang, Yi Du, Chen Wang, Xiuxian Li, Liang Xu

原文摘要

Vision-Language Navigation (VLN) requires long-horizon instruction-consistent planning and short-horizon path planning grounded in the currently observed geometry. Anticipating how future observations evolve along the instructed route provides predictive guidance beyond the current field of view, motivating video-action models (VAMs) that learn representations of future visual states. However, many existing VAMs couple future visual prediction with action generation during inference, despite their distinct temporal requirements: instruction-conditioned route guidance can be reused across multiple control steps, whereas local trajectories must continuously adapt to fresh observations. To bridge this gap, we introduce HAVANA, a dual-system VAM that decouples predictive reasoning from reactive local planning. A slow system produces a compact language-conditioned latent plan from observation history, while a fast system combines the latest plan with the current observation to generate local trajectories. Their asynchronous execution enables rapid adaptation without repeated future prediction, and no future video is generated during inference. Experiments on OmniNavBench demonstrate state-of-the-art navigation performance over prior baselines, while zero-shot real-world deployments further show robust generalization to unseen environments.

cs.CV · cs.RO

新论文 · v1arXiv:2611.20460

Stochastic Self-Distillation for Flow-Based Vision-Language-Action Models

Jingru Guo, Alessandra Russo, Joey Bose, Christos Ziakas

原文摘要

Flow-based vision-language-action (VLA) models achieve strong performance in robotics through imitation learning, yet fine-tuning them with on-policy reinforcement learning (RL) is limited. In group-based RL methods like Group Relative Policy Optimization (GRPO), every rollout step requires multi-step ODE integration, and the sampler's limited stochasticity leads to rollout groups with low variance. We propose VLA-SSD, which addresses both high inference latency and limited stochasticity. In particular, our method self-distills a pretrained flow-matching action head into a stochastic one-step policy by conditioning a single-jump generator on a Brownian path. We derive a conditional action density via our stochastic one-step policy, enabling efficient post-training via likelihood-ratio RL algorithms. We show that distilling VLA-SSD from the $π_0$ VLA reduces the action head's forward-pass time by a factor of 7.7, at the cost of a 5.5% relative decrease in the success rate on the LIBERO benchmark. However, by leveraging the increased sample diversity of VLA-SSD during group generation, post-training a $π_{0.5}$ VLA using GRPO yields a 20.6% relative improvement.

cs.RO

新论文 · v1arXiv:2611.20452

XMimic and XSoma: Learning Humanoid Motion Tracking and Locomotion in Under a Minute

Nico Bohlinger, Jan Peters

原文摘要

We present XMimic and XSoma, which learn humanoid motion tracking and locomotion on rough terrain in under a minute of training. Both follow SuperMimic, which learns a world model of the robot and optimizes the policy with analytic gradients through short rollouts imagined in this model. Existing implementations of this method do not make use of larger GPUs, as every training iteration is a long chain of small GPU operations that run one after another. XMimic and XSoma therefore shorten this chain and run the steps of every iteration at the same time on up to four GPUs. We compile the complete training loop with JAX, including the MuJoCo Warp simulator. For locomotion, XSoma leaves only the dynamics of the robot to the world model and computes everything else exactly, including the terrain height under the feet. It learns with a horizon of one second and without a critic. On four H100 GPUs, XMimic learns to track all 40 LAFAN1 clips with the Unitree G1 in 0.86 minutes, 3.7 times faster than BlinkMimic on one GPU. XSoma reaches the tracking target that Holosoma's tests set for FastSAC in 0.77 minutes, 12.3 times faster than FastSAC on one GPU, while needing only an eighth of its environment steps.

cs.RO · cs.LG

版本更新 · v4arXiv:2609.36413

One from Infinity: Actualizing Futures from Pretrained World Models into Robot Actions

Bang Du, Yichen Xie, Shuqi Zhao, Yuxin Chen, Menglin Wu, Masayoshi Tomizuka

原文摘要

A pretrained video world model admits many plausible futures for a scene, but a robot must realize the exact task-conditioned one. To turn world models into executable robot policies, existing methods fine-tune the heavy world model backbone using large-scale robot data and computational resources. Challenging this status quo, we argue that the expensive part has already been paid in the world model pretraining since the representation space of a video world model encoder lays out the diverse potential futures. In this case, what remains is to determine the future that accomplishes the task and to read out the actions that realize it. We formalize this task as actualization, which learns a task-conditioned realization on top of a prior supplied by a frozen world model encoder. This can be solved by a tiny actualizer model. We implement RoboActualizer with as few as 60M parameters on top of a frozen world model encoder. The actualizer is composed of two lightweight DiT experts that jointly predict future latents and actions by flow matching. The model can be trained entirely on a single GPU with 32 GB peak memory. With up to 100x fewer trainable parameters than existing WAMs and VLAs, RoboActualizer reaches great performance on simulation benchmarks including LIBERO, LIBERO-Plus, RoboTwin 2.0 and five tasks on two real-world platforms, with a low latency of 39 ms that allows real-time control.

cs.RO

新论文 · v1arXiv:2611.20402

Making Koopman Control Guarantees Hold on Robots

Abulikemu Abuduweili, Changliu Liu

原文摘要

A Koopman controller can stabilize its learned latent-space linear model yet fail to stabilize the nonlinear plant. The learned model propagates predictions in latent space, while the deployed controller repeatedly re-encodes the measured physical state. We show that nominal closed-loop stability transfers to the nonlinear plant when the learned and true closed-loop dynamics agree to first order on the embedding's tangent space at the target. We derive a robustness margin for approximate matching and show that a stable open-loop latent model cannot exactly match an unstable equilibrium. We introduce Tangent-Invariant Koopman (TIK), which estimates a local plant linearization from offline transitions and enforces first-order matching to this estimate. TIK recovers the fitted LQR gain at the target, while a bilinear predictor captures state-dependent actuation during large maneuvers. Across four simulated unstable benchmarks, all 40 Standard Koopman controllers have stable latent closed loops, but their deployed nonlinear systems are locally unstable. In contrast, all 40 TIK controllers locally stabilize their respective nonlinear plants. An offline stability test using empirical curvature estimates identifies stable TIK controllers on two cart-pole systems, but is inconclusive under high noise. Under explicit model-error and curvature bounds, an invariant terminal set guarantees constraint satisfaction after entry under local feedback. TIK achieves 88\% constrained cart-pole swing-up success with a 0.25 ms planning budget and controls simulated seven-joint arms within 1-5 ms per step.

cs.RO · cs.AI · eess.SY

版本更新 · v2arXiv:2603.06531

Spatial Calibration of Diffuse LiDARs

Nikhil Behari, Luke Apostolides, Ramesh Raskar

原文摘要

Diffuse direct time-of-flight LiDARs report per-pixel depth histograms formed by aggregating photon returns over a wide instantaneous field of view, violating the single-ray assumption behind standard LiDAR-RGB calibration. We present a simple spatial calibration procedure that estimates, for each diffuse LiDAR pixel, its footprint (effective support region) and relative spatial sensitivity in a co-located RGB image plane. Using a scanned retroreflective patch with background subtraction, we recover per-pixel response maps that provide an explicit LiDAR-to-RGB correspondence for cross-modal alignment and fusion. We demonstrate the method on the ams OSRAM TMF8828.

cs.CV · cs.RO

版本更新 · v5arXiv:2607.02764

MorphQuad: Morphable Quadrotor for Superhuman Maneuverability, Manipulation, and Resiliency

Jose Diaz Peon Gonzalez Pacheco, Jiawei Xu, Andrew Zhao, Hongyu Zhou, Atharva Navsalkar, Andrew Scheffer, Amrith Malli Reddi, Sashreek Shankar 等 10 位作者

原文摘要

Infrastructure maintenance, contact-based inspection, and emergency response can benefit from aerial vehicles that act as a flying human hand with extreme maneuverability, manipulation, and resiliency (MMR): maneuverability to fly in arbitrary orientations; manipulation to point sensors, turn valves, and press tools at arbitrary orientations; resiliency to maintain motion accuracy and force application despite disturbances from arbitrary directions, such as wind, ground effects, and friction. Realizing MMR requires more than omnidirectional flight; it also requires (I) vectoring of maximum thrust in any direction, (II) almost-everywhere stability, accounting for kinematic singularities, control singularities, and inter-rotor downwash interference, and (III) miniaturizable structural designs. Existing aerial vehicles satisfy a subset of I--III. We present MorphQuad, a morphable quadrotor that delivers MMR by realizing I--III on a quadrotor structure with independently articulated rotor systems through novel control and thrust allocation algorithms. On hardware, MorphQuad preserves a planar quadrotor structure, and independently articulates each of the four rotor systems, eliminating control singularities and enabling all rotors to align and vector the platform's maximum thrust in any direction; on control and thrust allocation, MorphQuad enjoys a closed-form null-space correction for gimbal lock and downwash interference that maintains almost-everywhere stability via coupling with a generalized geometric controller. With fully-onboard computation, localization, and trajectory execution, MorphQuad demonstrates continuous multi-revolution rotation while translating or hovering (maneuverability); valve turning, perching, and object pressing and pushing with human-wrist strength (manipulation); and push-pull recovery and wind rejection, even when directed to a single rotor (resiliency).

cs.RO

版本更新 · v4arXiv:2607.27924

ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow

Dongxiu Liu, Haoyi Niu, Peng Cheng, Yuan Gao, Xirui Kang, Sangli Teng, Koushil Sreenath, Xianyuan Zhan

原文摘要

In the physical world we inhabit, space and time are fundamentally continuous. However, existing machine learning paradigms for world modeling are largely confined to discrete-time prediction, thereby exhibiting significant inefficiency in capturing the dynamics of physical world. We introduce Physical-Time Flow (PT-Flow), a novel approach that learns a continuous latent velocity field operating in physical time. Crucially, the underlying dynamics of sequential data are parameterized by an ordinary differential equation (ODE) embedded in a well-structured representation space. Under this paradigm, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space. Building upon PT-Flow, we construct ODEWorld, a continuous-time latent world model that is both efficient and versatile. By extracting time-variant features and enforcing ODE properties on both the dynamical representation space and the latent velocity field, ODEWorld effectively addresses the long-standing representation collapse issue in latent world model literature. This also enables high-quality image reconstruction even after long-horizon prediction. Moreover, its continuous nature allows for arbitrary temporal resolution and even backward prediction, which is impossible for most discrete-time models. Lastly, ODEWorld can provide rich planning-oriented information to facilitate downstream policy learning. Comprehensive experiments demonstrate that ODEWorld successfully reconciles planning-conducive dynamics abstraction with visual realism, excelling in both video generation and robotic control. Project page: https://dstate.github.io/odeworld_website/.

cs.LG · cs.CV · cs.RO

新论文 · v1arXiv:2611.20324

High-Speed Forward and Reverse Tractor-Trailer Driving with Learned Vehicle Dynamics

Aaron Chen, David Shen, Taekyung Kim, Kaleb Ben Naveed, Dimitra Panagou

原文摘要

Tractor-trailer systems are difficult to control, especially unstable during reverse motion, and their dynamics depend on difficult-to-measure factors, including road friction and suspension coefficients. We introduce a learned dynamics model for tractor-trailer control and compare it with a reduced-order dynamics model with Fiala tire forces as an analytic baseline. Specifically, we compare a model predictive path integral (MPPI) controller that uses the learned dynamics model to an MPPI controller that uses the analytic model, as well as to three additional baseline controllers. In the main closed-loop tracking experiments across four simulated environments, MPPI with learned dynamics achieves mean speeds closer to the target magnitudes and lower root-mean-square (RMS) hitch angles than MPPI with the analytic model at every tested setpoint, with higher completion rates in seven of eight reverse configurations.

cs.RO · eess.SY

新论文 · v1arXiv:2611.20320

Course-Referenced Lateral Control of Trail-Following Convoys: String Stability and Adaptive Feedforward under Load Uncertainty

Nitesh Kumar, Sixu Li, Swaroop Darbha

原文摘要

Vehicles in a convoy can steer along the GNSS trails of the vehicles ahead when lane markings are unavailable, as on snow covered roads. Since each vehicle's tracking error enters the reference of the next, lateral errors can grow along the convoy. This paper presents a course referenced formulation of trail following in arc length, in which each follower takes the direction of its fitted trail as its heading reference. Error propagation then reduces to a single transfer function. We derive a condition under which combining the trails of the leader and the predecessor keeps the errors bounded for any convoy length, and show that following the predecessor alone lets the errors accumulate without bound, even without amplification, on roads that end turned or shifted sideways. Passenger and cargo loads and tire stiffness vary in practice, and we show that they enter the curvature feedforward through a single coefficient, whose mismatch leaves steady offsets on curves. We propose an adaptive law that estimates this coefficient on curves and removes the offsets. Simulations of an eleven vehicle convoy confirm the analysis.

eess.SY · cs.RO

新论文 · v1arXiv:2611.20314

NF-VLA: A Normalizing-Flow Baseline for Vision-Language-Action Learning

Ruichen Deng, Qinyu Zhao, Yong-Hyun Park, Mutian Tong, Liang Zheng, Jiatao Gu

原文摘要

Autoregressive normalizing flows (NFs) provide a tractable density for learning and sampling continuous robot actions. We show that a single maximum-likelihood objective can train vision--language--action (VLA) policies competitive with leading methods on manipulation benchmarks. We present NF-VLA, which combines conditional autoregressive normalizing flows with score-based action refinement. Its deep--shallow architecture concentrates visual, language, and proprioceptive prefix conditioning in a high-capacity deep block, while unconditioned shallow blocks provide a shared invertible action transformation. The vision--language backbone and flow are jointly trained by exact maximum likelihood on Gaussian-smoothed action chunks. The same learned density supplies a single score-based correction at inference, refining generated actions to account for training-time smoothing without a separately trained denoiser. Trained on LIBERO from a pretrained vision--language backbone without robot pretraining, NF-VLA achieves 98.35\% success across the four standard suites and 72.6\% on LIBERO-Plus using the same checkpoint without further fine-tuning. Ablations show that deep-only prefix conditioning and score-based refinement both improve closed-loop success. Together, these results establish carefully structured autoregressive flows as a competitive approach to VLA control, with one learned density supporting policy training, action generation, and refinement.

cs.RO · cs.AI

新论文 · v1arXiv:2611.20306

Ontology-Compiled Control Barrier Functions for Heterogeneous Robot Teams

Peng Wu, Seyede Fatemeh Ghoreishi, Nathan Bastian, Mohsen Imani, Tian Lan, Mahdi Imani

原文摘要

Safety filters for heterogeneous robot teams fail because of missing constraints, overestimated braking capabilities and conflicts between individually satisfiable constraints. We present OntoCBF, which compiles a domain ontology with a class taxonomy, typed relations, safety axioms and a dynamics profile per robot class into a control barrier function filter with one quadratic program per agent. Description-logic entailment supplies the constraints of any team online, including those that follow from chains of relations and from classes defined by what a robot carries, so the existing safety axioms cover new robot classes, payload classes and combinations. We prove that a subclass may reuse the braking capability of its parent whenever it can produce every velocity of the parent, a property separate from subsumption. For box-bounded double integrators this property is also necessary, and every loss of safety then begins with a conflict. Once three robots interact, the safe set of pairwise barriers contains states from which a collision is unavoidable, and such states persist in larger teams under any fixed capability reduction. In a simulated warehouse where humans ignore the robots, a filter written for the design classes violates safety in 89-100 % of the episodes with unseen classes. In all scenarios, including one with 32 robots, OntoCBF violates safety in 20-70 % of the episodes, each time after a conflict, with a median deepest intrusion of 10-17 % of the keep-out distance. A multi-agent PPO policy trained with the filter in the loop lowers the rate of infeasible quadratic programs, nearly all of them conflicts, by 18-41 %.

cs.RO · cs.LG

新论文 · v1arXiv:2611.20244

Right Goal, Wrong Body: Re-grounding Frozen VLAs under Actuation Shift

Umer Siddique, Peilang Li, Yongcan Cao

原文摘要

Vision-language-action (VLA) models are capable of producing desired robotic behaviors in ideal settings. However, they often fail when the robot's actuation changes at deployment. For example, remounting, mechanical wear, or command latency changes can lead to failed tasks and unsafe behaviors. To address the inevitable actuation shift, we propose ReGround, an inference-time method that keeps the VLA frozen and re-grounds its intended motion. ReGround learns the deployed robot's dynamics from task-agnostic motor babbling, and then tracks the policy's intended motion while staying close to the policy's own plan. ReGround activates only upon detecting a shift, and needs no rewards, demonstrations, or retraining. We show via two VLAs, eight LIBERO tasks, and five shift settings that ReGround can restore near-oracle success, such as raising $π_{0.5}$'s success from 0.16 to 0.90 under 400 ms latency, while remaining safer than a linear-calibration baseline. Project website and open-source code: https://umersiddique94.github.io/ReGround/.

cs.RO · cs.AI · cs.LG

版本更新 · v4arXiv:2606.20197

Stable Transformer-Actor-Critic Model Predictive Control: A Contraction Analysis Approach

Antonio Marino, Valerio Modugno, Marco Cognetti

原文摘要

Hybrid optimization-learning frameworks, such as Actor-Critic Model Predictive Control (MPC), rely on dynamically adjusting cost parameters to manage complex control landscapes. In this paper, we introduce a certifiably robust Transformer-Actor-Critic MPC architecture that leverages history-aware sequence models to shape local optimization while maintaining strict physical safety guarantees. First, we mathematically constrain the Transformer sequence models to satisfy global incremental Input-to-State Stability ($δ$ISS). Second, we integrate these neural Lipschitz bounds with Riemannian contraction theory, deriving a novel coupled small-gain condition that enforces uniform closed-loop contraction stability across the interconnected system. Evaluated on quadrotor control and decentralized multi-agent navigation tasks subjected to severe observation noise and process disturbances, our closed-loop stable formulation successfully translates theoretical bounds into physical safety.

cs.RO

版本更新 · v2arXiv:2606.05015

Flying on Imagination: DreamerV3-Style World Models under Environment Distribution Shift for Quadrotor Navigation

Luca Zanatta, Grzegorz Malczyk, Kostas Alexis

原文摘要

DreamerV3-style reconstructive world models offer a route to vision-based navigation for aerial robots through learned latent dynamics. Yet their behavior under shifts in the training environment and their ability to support navigation using imagined observations, predictions generated by rolling the latent dynamics forward without new sensory input, remain insufficiently characterized. We study four training-environment regimes for depth-based quadrotor navigation, spanning a fixed layout, a finite set of layouts, low-discrepancy continuous obstacle sampling, and i.i.d. uniform sampling. We cross-evaluate reconstruction before and after Reinforcement Learning fine-tuning, separately evaluate the resulting policies in simulation, and transfer them to a real quadrotor without real-world fine-tuning. We demonstrate onboard flight driven by imagined state and depth: a 24.7m mission in which 66% of flight time and 76% of traveled distance occur during repeated 5s sensor-free imagination windows, with prediction quality degrading over each open-loop interval and recovering after sensory re-grounding. Additionally, in closed-loop hardware transfer, all policies succeed in a setting close to the simulated geometry, whereas the policy trained from a single fixed layout fails under a corridor-topology and obstacle-type shift, while the other three succeed.

cs.RO

新论文 · v1arXiv:2611.20202

DAPPT: Domain-Aware Policies from Procedural Trajectories for Sim-to-Real Dexterous Tool Use

Beining Han, Jia Deng

原文摘要

Sim-to-real RL has achieved impressive results in dexterous manipulation but remains limited to tasks with accurate rigid-body simulation. However, these methods cannot be applied to skills that involve deformable and particles, with which the simulation speed will become too slow for large-scale RL learning. We present DAPPT, a sim-to-real framework for end-to-end dexterous tool-use manipulation skills, e.g., sand scooping, play-dough rod-rolling, without simulating non-rigid materials. Our method learns a domain-aware RL policy by tracking procedural trajectories parameterized by domain parameters, and distills it into an end-to-end domain-aware sensorimotor policy. Furthermore, explicit domain represen-tation enables fast selection of domain parameters that can maximize downstream task performance. We evaluate DAPPT end-to-end on six dexterous manipulation tasks, including sand scooping, play-dough rod-rolling, sand scraping, mouse-wheel scrolling, dispenser pressing, and spray triggering. Our method achieves 1.5-3.8x in task metrics through the best domain parameter and transfers zero-shot to the real world.

cs.RO

版本更新 · v5arXiv:2610.18826

WorldOracle: Benchmarking Action-Conditioned World Models against Real-World Executions

Haoyi Duan, Liliang Chen, Nan Wang, Yang Zhang, Yuhang Lu, Xiaotong Chen, Wei-Cheng Tseng, Shenyuan Gao 等 16 位作者

原文摘要

We introduce WorldOracle, a benchmark specifically designed to evaluate action-conditioned world models (ACWMs) against paired real-world executions. Given the same initial observation and low-level action sequence, WorldOracle asks whether a predicted rollout reproduces what happens in the real world, including physical consequences and task-state transitions. We evaluate models along four complementary dimensions: physical state correctness, interaction spatial alignment, key interaction area fidelity, and multi-view action coherence. To support this evaluation, we curate real-world training and evaluation suites that cover different interaction dynamics and embodiments. The evaluation suite contains long, multi-segment episodes with both successful and failed executions, enabling evaluation across varied action outcomes and behavior distributions. Experiments on seven representative ACWMs reveal distinct model strengths and failure modes. WorldOracle provides a systematic foundation for measuring and advancing action-conditioned world modeling for manipulation. We release the datasets, evaluation code, and leaderboard: https://worldoracle-benchmark.github.io/.

cs.RO · cs.CV

新论文 · v1arXiv:2611.20185

RoboKit: Accelerating Parallel Differentiable Optimization for Robot Learning

Jianglong Ye, Xuanbin Peng, Conor Mc Gartoll, Changwei Jing, Lai Wei, Jiali Chen, Xueyan Zou, Xiaolong Wang

原文摘要

We present RoboKit, a high-performance, general-purpose toolkit for parallel differentiable optimization in robotics, built on three principles: (i) Performance and flexibility: GPU-parallel multi-seed solvers pair sampling with versatile optimizers, running 2-60$\times$ faster than task-specific baselines across seven tasks; (ii) Geometry, rendering and physics: hybrid and coarse-to-fine representations accelerate collision, differentiable rendering and simulation, optimizing 4096 grasps in 2.97 s; (iii) Integration with learning: costs are differentiable and simulation is parallel, so RoboKit both distills optimization into policies and trains RL; distilled policies run 9-240$\times$ faster at comparable accuracy, and RL reaches a GPU baseline's scores up to 2.1$\times$ sooner on the same simulator.

cs.RO · cs.AI · cs.LG

版本更新 · v2arXiv:2605.06481

OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation

Yushan Liu, Peibo Sun, Shoujie Li, Yifan Xie, Lingfeng Zhang, Xintao Chao, Shiyuan Dong, Fang Chen 等 10 位作者

原文摘要

World action models (WAMs) learn manipulation by predicting actions and future observations, yet their success drops sharply when the camera viewpoint, object layout or robot initial state changes. Such changes keep the instructed object the same but alter the appearance and pose features the policy observes, which in standard attention also form the keys that locate the object. We introduce OA-WAM, an object-addressable WAM that finds each object by an identity address fixed for the episode and reads its changing state. In every layer of a pretrained multimodal trunk, object-slot keys depend only on the address and frame position, and values carry the current state; the same addresses key the action pointer and index per-object forecasts, which can trigger re-grounding. Fine-tuned on standard LIBERO demonstrations, OA-WAM transfers zero-shot to LIBERO-Plus with 90.5% success, outperforming all ten published baselines. It also leads on LIBERO (99.0%) and, fine-tuned on BridgeData V2, on SimplerEnv WidowX (88.9%). On five real-robot tasks, it succeeds in 89 of 100 nominal trials, versus 73 for the strongest baseline. Ablations trace its robustness to addressing rather than to object slots: with standard attention, slots do no better than a slot-free policy, and addressing them by identity raises geometric success by 10.7 points. Forecasting the addressed objects is likewise a stronger training signal than forecasting image codes.

cs.RO

新论文 · v1arXiv:2611.20137

Cleanly-DINO: Efficiently Learning Visual Task Spaces for Robotics

Adrian Röfer, Melika Shekarriz, José Arce y de la Borbolla, Iman Nematollahi, Abhinav Valada

原文摘要

Learning manipulation tasks from few demonstrations requires abstractions that capture the changes relevant to task completion, such as object poses in rigid-body manipulation. However, tasks such as cleaning are defined by changes in visual state rather than object pose, making the choice of task representation less clear. In this work, we propose Cleanly-DINO to learn visual task spaces from few demonstrations. Given a mask of a task-relevant region and frozen ViT features, Cleanly-DINO fits a projection to isolate a task space and fit a progression metric to it. We train the model on diverse cleaning examples and evaluate its ability to localize incomplete task regions and transfer across visual domains. We also study its utility for robot learning by using it as an observation encoder and reward signal in a reinforcement learning setting. Finally, we deploy our fitted model within a real-world robotic wiping pipeline to evaluate its use in guiding task execution.

cs.RO · cs.LG

版本更新 · v2arXiv:2608.26739

Simulation-Based Evaluation of Residual DDPG-Augmented Computed Torque Control for a Cable-Driven Lower-Limb Rehabilitation Robot Model under Disturbances and Parametric Uncertainties

Mohammad-Hossein Fakouri, Ali Keymasi-Khalaji

原文摘要

Nominal computed torque control (CTC) relies on accurate inverse dynamics; parameter mismatch and disturbances can leave tracking error, while joint limits and unilateral cable forces constrain motion. This numerical study evaluates a bounded Deep Deterministic Policy Gradient (DDPG) residual correction around nominal CTC for a planar three-degree-of-freedom, three-cable lower-limb rehabilitation robot model. In four simulation cases, we examine tracking, joint limits, workspace, cable-Jacobian conditioning and inferred cable demands. In a representative combined uncertainty/disturbance case, Cartesian RMS error was 0.0221419 m with CTC and 0.0128462 m with residual DDPG-CTC (approximately 42% lower). On a different circular reference, the residual branch nevertheless increased joint-limit-violating channel samples from 44 to 101. Separate supplemental experiments analyze training variability and checkpoints, alternative compensators, reward/action sensitivity, independently varied parameters and disturbances, trajectory transfer, constrained cable-force allocation and force lag. These studies show both improvements and failures, including early terminations and force-capacity-dependent degradation. The verified original algebraic force-backprojection path and later bounded-force simulations are distinguished; numerical allocations do not establish physical pretension or motor performance. Simscape and MSC ADAMS checks are numerical cross-model consistency assessments. No hardware, hardware-in-the-loop or patient tests were performed, and no clinical efficacy or physical safety guarantee is claimed.

cs.RO · eess.SY

新论文 · v1arXiv:2611.20127

Aerial Person Detection under Edge Constraints: Held-Out Evaluation and Edge-Ground Verification

Raghu Venkat, Tricha Anjali

原文摘要

A search drone sees a missing person as a few pixels, carries a small computer, and shares a narrow radio link. We study person detection under those limits; the primary evaluations were declared before they ran, and later analyses are marked exploratory. In an exploratory comparison, holding out recording blocks inferred from thermal frame identifiers cut recall at a fixed false-alarm target from 32.7% to 12.3%. With nothing tuned on test blocks, a 46,000-parameter detector gated by a contrast cue finds 10.4% of people against 5.7% for the same network applied everywhere. A standard detector, YOLO11n, trained under the same folds finds 54.5% at a similar false-alarm rate for about eleven times the arithmetic, and on our 4K field footage at 117 m has a walking person among its top 16 detections in up to 92% of frames against 74% for our board. Run alone, YOLO11n's thresholds set at one location overshot their false-alarm target 1.7 to 3.4 times at new locations, and its best configuration at altitude cost 4 to 24 standard passes per frame. Re-scoring its 16 best crops (154 KB per frame) with an 842,000-parameter ground model, thresholds frozen on a validation location, raised recall from 44.9% to 53.7% against the same 16 crops with YOLO's own scores; in a post hoc paired analysis over 15 test locations the gain was +8.8 points [+2.4, +12.5]. Pooled false detections fell from 0.86 to 0.64 per frame, but that paired change was not distinguishable from zero. Coverage of a still person in earth tones was very low for both detectors in the clips we flew.

cs.CV · cs.RO

新论文 · v1arXiv:2611.20107

SAFER: Navigating the Exploration-Exploitation Dilemma in World-Model-Based Planning for Autonomous Driving

Yansong Qu, Zilin Huang, Zihao Sheng, Sikai Chen, Samuel Labi

原文摘要

An autonomous vehicle must foresee the consequences of its actions, especially in the rare, time-critical situations where crashes happen. A world model offers this foresight, but only within the coverage of its training data, which poses an exploration-exploitation dilemma. Within this coverage a planner can hardly improve on the data, and beyond it the model's errors mislead it. Logs of competent driving rarely contain the failures that would widen the coverage, and agents that learn from interaction gain them only by failing themselves, without being shown what to do instead. We propose SAFER, a framework that learns offline from failures that others make, judged by an expert, and keeps learning online from its own driving. Offline, partially trained agents and scripted, recoverable perturbations Stage failures in simulation, and an expert Annotates every visited state with its action and value, so that much of what the planner explores is covered before it drives. In planning, a chunked latent world model Foresees the consequences fast enough for real-time use, and a risk-aware sampling-based planner Evaluates candidate maneuvers by their return minus a bounded collision penalty. Online, a loop continuously Refines the model wherever this driving leaves the coverage of the offline data. In closed-loop evaluation against the compared methods, SAFER reaches the highest route completion and reward per step on the CARLA Leaderboard routes, 0.95 and 0.77 from a privileged bird's-eye view, and the highest episode reward in MetaDrive, 53.4 in regular and 49.0 in safety-critical traffic. Project page: https://ys-qu.github.io/safer-website.

cs.RO · cs.AI

新论文 · v1arXiv:2611.20100

EgoSwarm: Learning Decentralized Robot Swarm Intelligence from Human Swarm Demonstrations

Ziqiang Liu, Xitong Chen, Min Li, Yiqun Li, Zhuoping Yin

原文摘要

Advancing from individual embodied intelligence to swarm embodied intelligence is a key step toward scalable, general-purpose embodied intelligence. Complex tasks often require multiple agents to divide responsibilities, coordinate handovers, and manipulate shared objects collaboratively. However, existing robot learning methods rely primarily on individual demonstrations, which provide limited supervision for modeling dependencies among agent actions, while collecting multi-robot demonstrations at scale remains difficult. Human egocentric demonstrations offer a scalable alternative, yet distributed observations from multiple participants cannot directly supervise decentralized policies. We introduce EgoSwarm, the first robot learning framework for learning decentralized multi-robot collaboration from human swarm demonstrations. EgoSwarm collects more than 50 hours of collaborative experience using a robot-free multi-person egocentric system. We introduce the Swarm Interaction Packet (SIP) and Swarm Interaction Token (SIT) to recover cross-agent interaction relations from distributed observations and transform this experience into local supervision consistent with deployment-time information constraints. We further propose SwarmVLA, which transfers knowledge of swarm behavior to decentralized policies conditioned on local observations, local history, and limited neighbor messages. SwarmVLA achieves a mean success rate of 72.2% across three categories of multi-robot collaborative tasks. Project page:https://xitong-c.github.io/EgoSwarm_Page/.

cs.RO

新论文 · v1arXiv:2611.20098

A Geometric Observer for Visual-Inertial Odometry using Optical-Flow Constraints

Tarek Bouazza, Soulaimane Berkane, Tarek Hamel

原文摘要

This paper presents a geometric observer for monocular visual-inertial odometry driven by individual optical-flow constraints. For each tracked point, the bearing and its rotationally compensated derivative give a depth-free homogeneous constraint on the body-frame velocity. These constraints enter the innovation of an observer on the gravity-augmented group $\mathbb{SE}_3(3)$ through a Riccati gain, without estimating depths or reconstructing the direction of motion. The auxiliary gravity state makes the six-dimensional body-frame velocity-gravity error dynamics independent of the attitude correction, and their uniform observability is shown to be equivalent to persistent excitation of a matrix formed from the inertial bearings and their derivatives. Under this condition, the body-frame velocity and gravity errors converge globally and uniformly exponentially, and a gravity-based attitude correction renders the combined velocity, gravity, and reduced-attitude error almost globally asymptotically stable and locally uniformly exponentially stable. Simulations illustrate the excitation condition, recovery from initial errors, and sensitivity to perturbed measurements.

eess.SY · cs.RO

新论文 · v1arXiv:2611.20075

Prompt Re-Specification and Target-Mask Availability in Mask-Conditioned Visual Reinforcement Learning for Moving-Platform Landing

Ho Tim Tsui, Jongyun Kim

原文摘要

Visual reinforcement learning (RL) policies that land a drone on a moving platform can fail when the landing pad looks different from the one used in training. Using binary target masks as the visual input removes most pixel-level appearance information while preserving the spatial representation of the target, and lets a text prompt specify the target. In simulation, we test whether a frozen (fixed-weight) mask-based DrQ-v2 policy can handle a new pad when only the prompt of its fine-tuned SAM 3 detector changes. A banana-themed pad lowered landing success from 75.6% to 12.0%, and SAM 3 rarely detected it under the original prompt. A revised prompt, chosen after testing several candidates, restored success to 68.4% with no weight updates. In a separate experiment with a policy trained on ground-truth masks, filling in masks that were missing while the pad was visible helped landing far more than the tested mask-centroid and area corrections. Tracking the pad over time supplied masks more often and also kept producing them after the pad was confirmed absent from the rendered image, but it did not establish a reliable improvement in landing success. Whether a usable mask reaches the policy at all thus appears to be an important limiting factor for this approach.

cs.RO · cs.CV · cs.LG

版本更新 · v2arXiv:2611.17110

SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models

Jiachun Jin, Zetong Zhou, Haonan Wang, Zhijie Deng

原文摘要

Embodied foundation models hinge on diverse robot interaction data, which are costly to collect. Egocentric human videos offer a complementary source of experience, and latent action models (LAMs) provide a popular means of leveraging these videos for policy supervision. Compared with world-action models (WAMs), LAMs can offer greater training efficiency and potentially lower inference costs by avoiding explicit pixel prediction during policy learning and execution. However, most existing LAMs learn latent actions solely through video reconstruction, which can cause their representations to cluster by visual appearance rather than by the underlying actions. As a result, they struggle to align human actions with their robot counterparts. To bridge the gap, we introduce SALA, which emphasizes establishing the semantic correspondence for latent actions. Without requiring paired data, we use the language descriptions of unpaired human and robot clips to estimate semantic similarities and supervise cross-embodiment correspondence. Specifically, we translate these similarities into soft neighborhood structures that guide human latent actions toward semantically related robot latents, while robot action reconstruction grounds the latent space in control. The resulting shared latent action space provides a unified supervision signal for training an embodied foundation model on both human and robot videos. Notably, latent action supervision enables scaling along two axes: (1) pre-training on increasing amounts of general human video consistently reduces robot action prediction error on downstream tasks, and (2) post-training on increasing amounts of task-specific human video substantially improves real-world robot task success rates.

cs.RO · cs.AI · cs.LG

版本更新 · v2arXiv:2611.17160

XBot-Omni: An Omni-Modal Model for Embodied Understanding and Interaction

Kerui Gu, Xing Fu, Shuoyang Sun, Cheng Peng, Jingru Duan, Huayu Gu, Zhenyu Chen, Jiaxin Huang 等 13 位作者

原文摘要

Embodied interaction requires an agent to integrate multimodal perception with verbal and nonverbal behaviors to effectively communicate its intentions, including speech, body motion, and facial expressions. We present XBot-Omni, a Thinker--Talker--Actor architecture that extends the Omni paradigm to robotic modalities, including body motion and facial expressions, with ``X'' representing the extensibility to robotic-specific modalities. The Thinker processes text, images, audio, and video to formulate contextually appropriate responses, which are then realized by the Talker and Actor through speech and coordinated bodily expressions, respectively. In this report, we innovatively introduce the concept of ``Actor'' as a dedicated component for embodied agents: a coordinated, streaming module that leverages the native representations of the Thinker and Talker within an end-to-end pipeline, introducing only modest latency overhead. This unified architecture enables multimodal generation conditioned on omni-modal inputs directly to prevent information loss in staged designs. We comprehensively evaluate XBot-Omni across seven benchmarks, demonstrating strong multimodal understanding capabilities on both general-domain and robot-centric videos, as well as high-quality, natural, and well-coordinated motion generation. Furthermore, our results validate its capability to generate embodied multimodal responses directly from omni inputs, meeting the diverse requirements of embodied interaction. XBot-Omni enables robots not only to perceive the omni world but also to express themselves through speech and bodily behaviors, extending Omni models from conversational interaction to embodied communication in physical environments.

cs.RO