跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1680新增论文
+1333其中 AI 相关
+187已有论文修订
+59新增 AI 导读

arXiv 最近完成扫描:2026-11-18 04:28 · 全站最近入库:2026-11-17 23:35各信源进度 →

统计口径与数据新鲜度

统计 2026-11-17 06:11 — 2026-11-18 06:11(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-18 04:28最近发现新增版本:2026-11-17 23:35库内论文最新更新时间:2026-11-17 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

25,786 已收录论文21,491 AI 相关7,725 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 1,694 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-18 04:27(北京时间)
版本更新 · v2arXiv:2510.10731

Controllable Generative Trajectory Prediction via Weak Preference Alignment

Yongxi Cao, Julian F. Schumann, Jens Kober, Joni Pajarinen, Arkady Zgonnikov

原文摘要

Deep generative models such as conditional variational autoencoders (CVAEs) have shown great promise for predicting trajectories of surrounding agents in autonomous vehicle planning. State-of-the-art models have achieved remarkable accuracy in such prediction tasks. Besides accuracy, diversity is also crucial for safe planning because human behaviors are inherently uncertain and multimodal. However, existing methods generally lack a scheme to generate controllably diverse trajectories, which is arguably more useful than randomly diversified trajectories, to the end of safe planning. To address this, we propose PrefCVAE, an augmented CVAE framework that uses weakly labeled preference pairs to imbue latent variables with semantic attributes. Using average velocity as an example attribute, we demonstrate that PrefCVAE enables controllable, semantically meaningful predictions without degrading baseline accuracy. Our results show the effectiveness of preference supervision as a cost-effective way to enhance sampling-based generative models.

cs.RO · cs.LG

版本更新 · v2arXiv:2609.21178

OpenRoIS: A Community-Driven Open-Source Middleware Implementing the Robotic Interaction Service (RoIS) Framework for Physical Robots, Virtual Avatars, and Digital Services

Sebastian Carrera Villalobos, Christopher Nolan Arellano, Arne Hitzmann, Edilson Morais Brito, Akira Utsumi, Yukiko Horikawa, Takahiro Miyashita, Hiroshi Ishiguro 等 9 位作者

原文摘要

Service applications for human-robot interaction are commonly written against the hardware-specific interfaces of one platform, so a change of hardware forces a rewrite. The Robotic Interaction Service (RoIS) Framework 2.0, standardized by the Object Management Group (OMG), addresses this fragmentation with a platform-independent model in which Service Applications drive Human-Robot Interaction (HRI) Engines through standardized interfaces and symbolic messages. A specification alone, however, does not provide the maintained implementation, Software Development Kits (SDKs), and adapters that adoption requires. This paper presents OpenRoIS, a community-driven open-source middleware that implements RoIS 2.0, and takes the position that an openly developed, paradigm-neutral implementation is what carries the standard into practice. OpenRoIS contributes a recursive engine for both the main and the sub HRI Engines, a single protocol in which the RoIS method catalog over JSON-RPC 2.0 and WebSocket is the only interface at every hop, a single-source type pipeline for three languages, client SDKs for JavaScript, Python, and Unity, a component SDK, a browser inspector, and a Model Context Protocol (MCP) server for AI agents. These interfaces let a Service Application, scripted or an AI agent, address physical robots, virtual avatars, and digital services alike, each behind an adapter. In the functional validation, an AI agent and the browser inspector drove two physical robots from different vendors, a Reachy Mini and a Kachaka, behind one gateway, where a single command makes one robot greet a visitor while the other drives to the entrance. All of it is openly developed under the Apache-2.0 license at https://openrois.org/.

cs.RO · cs.HC

新论文 · v1arXiv:2611.19688

Physics-Grounded STL Monitoring from Image Observations

Xinpei Ni, Glen Chou, Samuel Coogan

原文摘要

Robotic manipulation missions must satisfy timed and conditional requirements that can be expressed in signal temporal logic (STL). While a practical means of monitoring these requirements is from camera images, this is challenging because STL robustness depends on physical margins, such as distances and clearances, that are not directly observed. Rather than learning a separate visual monitor for each formula, we learn a shared latent representation from multi-view images and decode physically meaningful margins for individual predicates. Co-training the encoder with predicate decoders increases the certification rate from 58.7% to 96.3% compared with a frozen encoder, despite nearly unchanged predicate sign accuracy. STL semantics then compose these margins into robustness estimates for any bounded formula over the trained predicates, without retraining. We further calibrate the decoding errors of selected STL clauses using conformal prediction, yielding two-sided intervals that certify both satisfaction and violation and propagate through interval STL to any formula composed from the clauses. In simulated kitchen missions (LIBERO), our monitor correctly predicts satisfaction or violation in 98.9% of 4662 evaluations across 18 unseen formulas. For missions lasting up to 17 min, calibrated bounds establish satisfaction or violation for 96.9% of missions without incorrect conclusions. A single calibration also provides simultaneous 91.0% coverage across all 255 conjunctions of eight clauses, allowing new formulas without recalibration.

cs.RO · eess.SY

新论文 · v1arXiv:2611.19684

Early Readability of Trajectory Quality in Diffusion Driving Planners

Xiang Zhang, Ziyang Leng, Yuxin Bao, Zhihao Zhao, Bolei Zhou, Xugui Zhou

原文摘要

Diffusion-based driving planners can generate multiple trajectories for the same scene, but these candidates vary in quality. Can their internal states reveal which candidate will be better before denoising is complete? We systematically study this question in three pretrained planners: Alpamayo, ReCogDrive-IL, and HDP, using linear probes to predict final trajectory quality from intermediate states and evaluating candidate ranking and selection on held-out scenes. Across all three planners, we observe that states from the first denoiser evaluation already support informative selection, while further denoising does not consistently improve selection quality. Further analyses show that readout depends on the representation and reader, and that selection can coexist with limited overall ranking accuracy. These findings motivate an early-selection method that selects after the first denoiser evaluation and completes only the chosen candidate. This reduces computation relative to fully denoising every candidate and improves open-loop trajectory quality over random selection across all three planners. In particular, it reduces Alpamayo's average displacement error by 17.8% and 13.4% in one- and four-camera configurations, respectively. Closed-loop evaluations also show higher mean driving scores and progress.

cs.RO

版本更新 · v3arXiv:2604.20744

AAC: Admissible-by-Architecture Differentiable Landmark Compression for ALT

An T. Le, Vien Ngo

原文摘要

We introduce the Architecturally Admissible Compressor (AAC), which adapts landmark-based A* heuristics to query demand under a fixed label-memory budget. AAC provides two continuous relaxations of ALT (A*, Landmarks, and Triangle inequality) selection. With exact landmark distances, every parameter value defines a lower bound on shortest-path distance. Both relaxations deploy ALT using only the selected labels. The row-stochastic relaxation takes convex combinations of landmark distances. We characterize the maximal admissibility-preserving linear classes and prove that the relaxed and discrete optima coincide. The fractional relaxation mixes subset heuristics and supports continuous greedy (AAC-CG) with exact gradients and an entropy-regularized oracle for differentiable selection. For expected heuristic value, we prove an approximation guarantee with finite-step, oracle-error, and finite-sample terms. The limiting $1-1/e$ factor is optimal for arbitrary supplied pools and query distributions unless P${=}$NP. At matched label memory, AAC-CG reduces A* expansions by $1.9$-$22.7$% relative to farthest-point ALT (FPS-ALT) in all $19$ tested combinations of graph and memory budget across six road networks and two synthetic graphs. Each combination uses one $500$-query test split; all methods share distance tables when distances are symmetric. Canonical greedy also expands fewer nodes than FPS-ALT in all $19$ graph-budget pairs. AAC-CG achieves $99.6$% of canonical greedy's mean heuristic value on test queries, averaged over these pairs. AAC makes admissibility independent of training success, while its analysis and experiments distinguish selection quality from search efficiency.

cs.AI · cs.LG · cs.RO

新论文 · v1arXiv:2611.19660

PSK: A Matrix-Free Single-Loop Method for K-Level Physics-Based Control

Zherong Pan

原文摘要

Physics-based control is central to robot learning, but differentiating through implicit physics steps typically requires Hessian or Karush--Kuhn--Tucker (KKT) linear solves; we ask whether gradient steps suffice. Each simulation step minimizes an energy under many contact and collision constraints. Without constraints, we view control over $K$ steps as $K$-level optimization with strongly convex lower levels, recast as one min--max problem. Our Penalized Single-loop $K$-level method (PSK) uses only gradients and a projection, updates all levels in parallel, and reaches an $ε$-stationary control in $O(ε^{-3})$ iterations, with a $K$-independent exponent but a $K$-dependent constant. Contact-rich constraints, however, typically violate a constraint qualification (CQ). We replace each constraint by a smooth wall that acts on a band of constant width. Under mild assumptions that need no CQ, every exact solution of the banded lower levels is then feasible and banded Fritz--John, and PSK keeps the $O(ε^{-3})$ complexity. Without a CQ that is uniform in the band width, however, a shrinking band loses the $K$-independent exponent: with width $ε$ and fixed $K\geq2$, deterministic zero-respecting methods need $Ω(ε^{-(K+3)/4})$ calls on a hard instance whose control dimension grows with $1/ε$. We prove our assumptions for smoothed kinematic and dynamic multi-robot navigation among segment obstacles. In navigation experiments, PSK, tuned per case, reaches every goal with sixteen levels and, at equal time, reaches a lower loss than tuned unrolling in 17 of 18 cases, while implicit differentiation is lower in 10.

cs.RO · math.NA

版本更新 · v2arXiv:2608.13438

ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models

Gehan Zheng, Matthew Johnson-Roberson, Weiming Zhi

原文摘要

Contact-rich manipulation failures are often detected only after the robot has committed to contact. This is especially limiting in wrist-camera setups: close gripper--object views help observe contact, but a poor approach may already push, miss, slip, or disturb the object before conventional detectors react. We introduce ContactGuard, a pre-contact execution monitor for chunked visuomotor policies. Given the policy's planned action chunk, ContactGuard predicts its short-horizon consequence in latent visual space and aborts if the predicted future latent indicates likely failure. Its latent world model is trained from unlabelled robot trajectories to predict compact multi-view visual embeddings under planned actions, avoiding pixel-level video prediction. A lightweight failure probe is then trained from a small labelled set of pre-contact clips. At deployment, ContactGuard anchors prediction before an imminent contact event, rolls the model forward under the policy's own actions, and verifies the predicted post-contact latent. Across real-world contact-rich manipulation tasks, ContactGuard consistently benefits from action-conditioned latent prediction over current-state and direct-prediction alternatives. Counterfactual action perturbations further show that its failure score responds to the specific action being proposed. We deploy the same frozen monitor online with the visuomotor policy, where it provides a pre-contact abort signal without modifying the underlying policy.

cs.RO · cs.AI · cs.CV

版本更新 · v3arXiv:2610.35165

How Tuning, Reference Preview, Actuator Modeling, and Fallback Policies Change Quadrotor Controller Comparisons: A Reproducible Simulation Study

Xuan-Quy Dao, Le-Minh Nguyen, Ngoc-Bich Le

原文摘要

Quadrotor controller comparisons commonly report a single tracking error, although the ranking also depends on tuning effort, reference information, actuator prediction, optimizer fallback, and command timing. We quantify these effects for cascade PID, hover LQR, boundary-layer sliding-mode control (BL-SMC), linear MPC with LQR fallback (LMPC-F), a preview-enabled LMPC-F, and a geometric controller on a common nonlinear plant with rotor gyroscopic coupling, motor lag, and saturation. A primary study (556 episodes at 50~Hz) combines matched twelve-candidate tuning, 400 paired evaluation episodes, and a 48-cell MPC factorial; a confirmation study (1462 episodes) repeats the evaluation on 50 fresh realizations with tuning budgets of 12, 24, and 48 candidates. Matched tuning changed the mean-RMSE ordering of the fixed baseline settings, mainly by moving BL-SMC from last to first. The ordering recurred on fresh realizations and did not change with 48 candidates, although BL-SMC's lead there depends on a wider search range for one of its gains. Reference preview reduced MPC tracking error and removed fallback activity on the figure-eight task. In single abrupt-step runs, completion depended jointly on fallback policy, predictor, and actuator authority. Replaying measured command-call delays terminated three of four nominal MPC configurations on a desktop and all four on a Raspberry Pi~5, whereas capping the solver at 100 iterations removed every deadline miss in these profiles. We propose a reporting checklist that adds tuning effort, failure rates, fallback activity, applied-input variation, and command timing to tracking accuracy.

cs.RO

新论文 · v1arXiv:2611.19621

Toward Planning Representations That Separate Task Structure from Visual Content

Aarav Srivastava, Vishwas Sathish, Rajesh P. N. Rao

原文摘要

Effective planning requires representations that preserve task-relevant geometry and object identity while ignoring visual variation that does not alter the task. Existing evaluations of visual generalization often change several factors simultaneously, making the source of a failure difficult to identify. We introduce a controlled evaluation of object-goal navigation (ObjectNav), the task of reaching an instance of a named object category, in procedurally generated ProcTHOR houses. Training and evaluation houses share floor plans, object locations, goals, and start states, while visual factors are changed individually or in combination. Across model-free reinforcement learning (RL), model-based RL, and agents using frozen pretrained visual representations, our results show that appearance changes impair navigation, but their effects differ sharply by factor. Surface material changes account for most of the average change in success, whereas clutter and lighting have little effect. The magnitude of a global image statistics shift does not reliably predict performance: changing only the goal object's appearance can substantially impair an end-to-end agent despite a small image-level change. We found that a model based agent that plans over frozen features from the pretrained DINOv2 image encoder achieves the highest success (0.83) under the full shift, but remains sensitive to surface appearance. Our experiments reveal an important requirement for planning representations: they must separate invariant spatial task structure from appearance.

cs.CV · cs.RO

版本更新 · v4arXiv:2610.39126

Actuator-Aware Observer Inputs for Cascade ADRC of Quadrotors Under Rotor Lag, Measurement Delay and Sensor Noise

Xuan-Quy Dao, Ngoc-Bich Le, Le-Minh Nguyen

原文摘要

In sampled cascade linear active disturbance rejection control (LADRC) of quadrotors, the extended-state observer is usually driven by the instantaneous command, although the delayed measurement reflects an earlier, rotor-filtered acceleration. This simulation and software-in-the-loop (SITL) study examines actuator-aware (AA) observer inputs: the altitude and attitude observers use the limited virtual input filtered by a rotor-lag model and aligned with the measurement delay, and the horizontal observers use the acceleration reconstructed from the measured attitude. Fixed gains are selected on the sampled model under sensitivity and step-response requirements and covariance-based ceilings on the noise-induced control increment, including the path from position noise to the inner loop. In 30 paired set-point trials, the two AA designs reach position root-mean-square errors (RMSEs) of 19.2 and 16.9~mm, versus 65.5~mm for a proportional-integral-derivative (PID) controller tuned under the same requirements, with lower rotor-command increments. Ablations attribute the improvement to the rotor-lag model and the measured-attitude input; delay alignment is negligible at the nominal 5-ms delay. PX4 SITL runs reproduce these results within 2~mm. Behind the PX4 state estimator, the white-noise ceilings no longer describe the loop. After redesign with an identified estimation-error model, one AA design reduces the set-point RMSE from 59.1 to 44.0~mm with a 15\% lower quantized rotor-command increment, whereas the half-gain variant is worse than the PID. With fixed gains, the AA designs tolerate less unmodelled rotor lag and delay than the PID. Hardware validation remains future work.

cs.RO

新论文 · v1arXiv:2611.19616

Towards Probabilistically Safe Reinforcement Learning: Theory and Deployment on Humanoid and Aerial Robots

Pol Mestres, Varun Gabbita, Jason H. K. Siu, Sergio A. Esteban, Hampus Serneke, Joel W. Burdick, Aaron D. Ames

原文摘要

Reinforcement learning (RL) enables robots to acquire complex skills, but standard RL formulations do not guarantee that learned policies satisfy safety constraints, which limits their use in safety-critical settings. Existing safe RL methods with provable guarantees typically support only restricted classes of constraints and require unbiased estimators of the objective and constraint functions, which prevents their direct integration with state-of-the-art algorithms such as proximal policy optimization (PPO). In this paper, we extend the recently introduced robust safe gradient flow-based reinforcement learning (RSGF-RL) framework to overcome these limitations. Our formulation accommodates a general class of trajectory-level safety constraints, including chance constraints such as collision avoidance with a prescribed probability, and our analysis allows for biased value and gradient estimators. We show that, with high probability, the constraint violation throughout training is bounded by the estimator bias plus a term that vanishes as the number of rollouts grows, and decays to this level if the initial policy is unsafe. We also establish the convergence properties of the iterates to approximate KKT points of the original safe RL problem. Finally, we show that RSGF-RL implemented with PPO-style estimates enables zero-shot sim-to-real deployment of whole-body control policies on a Unitree G1 humanoid and collision-free waypoint navigation on a Crazyflie drone.

cs.RO · eess.SY

版本更新 · v2arXiv:2610.41044

Consensus Within, Competition Between: Decentralized LLM Teams for Perimeter Defense

Khawaja Abaid Ullah, Ananya Acharya, Mohammad Javad Khojasteh

原文摘要

The use of large language models (LLMs) to coordinate multi-agent teams raises a fundamental architectural question: whether task allocation should be centralized in a single LLM controller or distributed across autonomous LLM agents, particularly when communication is unreliable. In a deliberately simplified perimeter-defense simulation against LLM-controlled intruders, we compared a central commander, an auction-based consensus team and independent agents under five network conditions. Breach rates were 15% for the consensus team, 32% for the commander and 29% for independent agents. Burst message loss, falsified reports and limited radio range raised the commander's breach rate by 10 to 18 percentage points (pp), whereas the consensus team's changed by at most 5. In the consensus team, giving a low-cost LLM more authority over the bid computation raised its breach rate by 38 pp. With birds and civilian drones in airspace, we added an image for each to the input text state. Only the near-frontier model treated them apart from intruders. Intruders disguised as existing taxidermy bird drones were mistaken for birds by both low-cost and near-frontier LLMs, being dropped from pursuit in 59-61% of near-frontier decisions (1-16% before). However, a one-sentence forewarning in the prompt lowered it to 0-25%. We find that the LLM team with a dedicated external conflict-resolution protocol had the fewest breaches. The code is available at https://github.com/the-kas-lab/consensus-within-competition-between.

cs.AI · cs.LG · cs.RO

版本更新 · v3arXiv:2411.15533

Development of a Low-Cost Prosthetic Hand Using Electromyography and Machine Learning

Mosab Diab, Ashraf Mohammed, Yinlai Jiang, Sharief Babikir

原文摘要

Electromyography (EMG) is a measure of muscular electrical activity and is used in many clinical/biomedical disciplines and modern human computer interaction. Myo-electric prosthetics analyze and classify the electrical signals recorded from the residual limb. The classified output is then used to control the position of motors in a robotic hand and a movement is produced. The aim of this project is to develop a low-cost and effective myo-electric prosthetic hand that would meet the needs of amputees in developing countries. The proposed prosthetic hand should be able to accurately classify five different patterns (gestures) using EMG recordings from three muscles and control a robotic hand accordingly. The robotic hand is composed of two servo motors allowing for two degrees of freedom. After establishing an efficient signal acquisition and amplification system, EMG signals were thoroughly analyzed in the frequency and time domains. Features were extracted from both domains and a shallow neural network was trained on the two sets of data. Results yielded an average classification accuracy of 94.8% and 93.2% for the time and frequency domains respectively. Furthermore, results showed a faster computation and response for the time domain analysis; hence, it was adopted for the classification system. A wrist rotation mechanism was designed and tested to add significant functionality to the prosthetic. The mechanism is controlled by two of the five gestures, one for each direction. Which added a third degree of freedom to the overall design. Finally, a tactile sensory feedback system which uses force sensors and vibration motors was developed to enable sensation of the force inflicted on the hand for the user.

cs.RO · cs.HC

新论文 · v1arXiv:2611.19572

Spectral Steering: Local Inverse Geometry for Frozen Diffusion Generators

Angelo Moroncelli, Alessandro Leanza, Loris Roveda

原文摘要

A frozen diffusion generator may need to change several output properties while preserving others. We study when guidance faithfully realizes such a structured request and whether local inverse correction improves its direction. Our account links this benefit to the interaction between the request and the objective-specific remaining-sampler Jacobian: conditioning alone is insufficient. Spectral steering combines a request-specific cosine-gain predictor with damped inverse correction and fixed-time relinearization for finite edits. Scalar nulls and matched-movement controls test the predicted mechanism. Across 8,923 instances in a retrospectively identified local regime, predicted and realized gains correlate at Spearman 0.79, versus -0.11 for condition number. Experiments span images, audio, robotics, and control and planning, including complex character motion generation. On 50 paired inpainting cases, four corrections improve unobserved-pixel PSNR over Adam by 3.25 dB at equal total derivative traversals. Together, the analysis and controls connect request-dependent geometry to directional gains and establish where these gains translate into downstream performance.

cs.LG · cs.AI · cs.CV · cs.RO

版本更新 · v2arXiv:2610.26979

Latent Projection for Latent Planning

Yoo Sung Jang, Michael Ryoo

原文摘要

Latent world models let an agent plan in latent space. The planner searches for an action sequence that minimizes a cost, and that cost is often the $\ell_2$ distance between the predicted terminal latent and the goal latent. The latent, however, is partly noise: the regularizer that prevents collapse during world model training promotes variance in the directions carrying no signal. We show that the same regularizer also supplies the structure to prune that noise in the latent. SIGReg drives the latent marginal to a standard Gaussian at every step, and assuming further that the latent sequence is a Gauss-Markov process, the transition is linear with its covariance pinned by the map itself, $z' \mid z \sim N(Az,\, I - AA^\top)$. Fitting the state dynamics $A$ therefore yields the covariance at no additional cost, and in its eigenbasis each latent direction carries its own uncertainty. Furthermore, constraining $A$ to be a normal matrix fixes that basis across timesteps, so one eigenbasis measures the uncertainty over an arbitrary number of steps and serves as a stable basis to compute the planning cost on. We then restrict the $\ell_2$ cost to the $k$-dimensional subspace the state dynamics carry forward with the least uncertainty. The correction is post hoc: the encoder, the predictor, the rollout, and the planner are all left untouched, and only a single constrained matrix, $A$, is fit. We evaluate across six environments and show large gains on the 2D navigation tasks, TwoRoom and PointMaze, raising success rate over the plain $\ell_2$ cost by up to $+54.7$ points under a harder planning setting. Code is available at https://github.com/yjang43/lp2.

cs.LG · cs.RO

新论文 · v1arXiv:2611.19559

ResWAM: Learning What Changes, Not What Stays, for World-Action Models

Chongyu Fan, Gaowen Liu, Ramana Rao Kompella, Jason Corso, Sijia Liu

原文摘要

World-action models (WAMs) augment vision-language-action policies (VLAs) with future prediction so that the policy can anticipate how its actions will affect the physical world. Existing WAMs supervise complete future observations. However, observations typically change little over an action chunk, so the future loss is dominated by copying static background content rather than learning task-relevant robot and object motion, limiting policy learning. Moreover, existing WAMs rely on large video generation models or separate world models, incurring heavy training and inference compute that hinders closed-loop deployment. To improve policy performance, we propose visual residual learning, supervising only the feature change from the current frame to the future frame in a frozen DINOv3 space. Compared with complete future features, this residual learning target yields a higher feature rank with richer and more diverse representations of scene changes, and spatially concentrates on the robot arm and manipulated objects rather than the background. To further improve computational efficiency, we present ResWAM, a residual world-action model with 2.3B parameters in total, including a lightweight 0.5B dual-branch DiT that jointly trains an action branch and a residual branch via conditional flow matching. Without robot pretraining (i.e., pretraining on external robot action trajectories beyond the benchmark demonstrations), ResWAM achieves top average success rates on LIBERO-Plus (81.1%), RoboTwin 2.0 (93.8%), and RoboDojo (9.40% success, 13.41 score) at the lowest training (2.7-4.0 TFLOPs) and inference compute (1.0-1.4 TFLOPs), and leads real-robot Open, Put, and Stack (80.0% average, +10.0% over $π_{0.5}$) at 92.7 ms latency.

cs.RO

版本更新 · v2arXiv:2607.15898

Orbis 2: A Hierarchical World Model for Driving

Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso, Karim Farid, Johannes Dienert, Rajat Sahay, Muhammad Ali, Thomas Brox

原文摘要

Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required for real-world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations. We further show that pretraining with a diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing -- predicting only the next frame from clean context -- produces more stable autoregressive rollouts. We therefore introduce a generic two-stage training paradigm that pretrains the model with diffusion forcing and fine-tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter. Our approach achieves state-of-the-art results across the standard suite of driving world model evaluations on established benchmarks, including long-horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality. Project page with code, demo, checkpoints and qualitative results: https://lmb-freiburg.github.io/orbis2.github.io/

cs.CV · cs.AI · cs.LG · cs.RO

新论文 · v1arXiv:2611.19535

Cooperative Gradient Estimation for Multi-Robot Source Seeking with Heteroskedastic Measurements

Alessandro Renzaglia, Lara Briñón-Arranz

原文摘要

This paper proposes a novel approach for localizing the source of a three-dimensional scalar field using a team of heterogeneous cooperative mobile robots. Each robot measures the field intensity at its location and shares the information within the team. Measurement noise is explicitly considered, and heterogeneity arises from robot-dependent sensing accuracies, leading to heteroskedastic data. The proposed framework combines a weighted centroidal Voronoi tessellation-based formation strategy, defined on a spherical surface and parameterized by sensor noise variances, with a Weighted Least Squares estimator to compute the field gradient at the formation center. By explicitly accounting for sensor heterogeneity in both formation design and estimation, the method achieves robust gradient estimation under noisy measurements. In particular, we show analytically that this construction is nearly optimal. The covariance of the gradient estimate is derived in closed form, providing a lower bound on the attainable estimation variance for any formation on the sphere, and the proposed tessellation is shown to asymptotically satisfy the conditions under which this bound is attained. The estimated gradient is then exploited within a gradient-ascent strategy to guide the team toward the source location. Simulation results demonstrate the effectiveness of the proposed approach compared to alternative strategies and highlight its intrinsic ability to reconfigure online in response to robot failures, thereby enhancing its adaptability.

cs.RO · eess.SY

新论文 · v1arXiv:2611.19534

Depth-Aware Path Planning and Collision Avoidance for Robotic Nursing Assistants Using Real-Time 3D Vision

Peeyush Thakur, Dharmendra Sharma, Archit Sharma, Sandeep Gupta, Anuj Nandanwar, Laxmidhar Behera

原文摘要

This paper presents a depth-aware path planning and collision avoidance framework for the safe navigation of robotic nursing assistants in cluttered environments with low-height obstacles. Conventional LiDAR-based navigation systems, while accurate for mapping and localization, fail to detect transparent or low-height obstacles, whereas vision-based systems are prone to noise and occlusion. The proposed framework integrates RGB-D depth sensing, rule-based obstacle avoidance, and the NavfnROS global path planner to drive the robot from an initial point to a goal point. The proposed approach is implemented on the Husky A200 robot, equipped with a Kinova Gen3 arm, to form a complete setup for a robotic nursing assistant. This system features a multi-layer architecture for global planning, perception fusion, and real-time control in the ROS-Gazebo environment. Experimental evaluations in a simulated cluttered environment demonstrate up to 40\% higher success rate and 35\% improvement in minimum clearance compared with single-sensor baselines. Furthermore, real-world experimental results validate the proposed approach as a computationally efficient and safety-aware navigation solution for autonomous healthcare robots.

cs.RO

新论文 · v1arXiv:2611.19469

Representing Differential Inclusions as Monads: A Framework for Compositional Controller Synthesis

Ahmad Ahmad, Kevin LEahy

原文摘要

Sequential composition of safety-certified controllers for hybrid and non-smooth robotic systems is well studied piecewise. Yet no existing framework reduces this question to a single algebraic condition. We construct a \emph{trajectory monad} for Filippov-regular differential inclusions. Its Kleisli category treats controllers as morphisms and sequential execution as trajectory concatenation. Control Barrier Function (CBF) certification, a guarantee of forward invariance, corresponds to a sub-monad of this category. Joint safety of composed controllers then follows from one algebraic condition. All controllers must land in a common sub-monad, which is closed under the monad multiplication. The sub-monad condition recasts recasts existing CBF composition practice categorically. It explains why a smooth intersection of multiple barriers supports safe composition, while enforcing barriers phase by phase does not. For the phase-by-phase case, we derive a closed-form Bernoulli model of the failure rate. We validate it over a randomized sweep of 3{,}000 instances. The same sweep shows that the standard baseline degrades under constraint conflict as the barrier count grows. This baseline enforces all barriers simultaneously as constraints in a single quadratic program (QP). We instantiate the framework on a four-mode piecewise-affine system and on a unicycle reduced-order model over heterogeneous terrain.

eess.SY · cs.RO

新论文 · v1arXiv:2611.19466

Successive Convexification in Learned World Models for Safe Offline Planning

Samuel Younger

原文摘要

World models learned from offline data allow an agent to plan without further interaction with the environment, but the sampling-based planners used with them handle safety constraints only by scoring or rejecting sampled action sequences. We instead pose safe planning in the latent space of a learned world model as a successive convex program (SCP), in which a learned discrete-time control barrier function is imposed as a constraint at every step of the horizon. SCP requires smooth dynamics, so we build the networks from smoothed MaxMin activations and spectrally normalised layers, and bound the resulting linearisation error in terms of the Lipschitz bound and the smoothing width. As SCP is a local method, we start it from a sampled plan. On two Safety-Gymnasium navigation tasks with offline data, this planner has fewer violations than MPPI and CEM drawing 128 times as many samples at equal wall-clock time in our main comparison, though not significantly, and earns greater return at longer horizons, and is much safer than SCP without a sampled start. SCP converged more often on Lipschitz-bounded models. Unlike sampling, it also reports at each step whether its plan is a feasible, first-order optimal KKT point of the planning problem in the model, or that no feasible plan was found nearby. Our implementation is publicly available at https://github.com/youngers2006/scwm.

cs.LG · cs.RO · eess.SY

新论文 · v1arXiv:2611.19463

Semantics First, Pixels Later: Mission-Driven Event Streams for Drone Video over Bandwidth-Starved Links

Cong Xu

原文摘要

A surveillance drone on a bandwidth-starved link cannot send the video its operator would need, yet the operator's questions are often not about pixels: did three people converge, when, and where? We study what should cross a link of 1 Mbps down to 1 kbps instead, and in what order. BPS, a behavior-primitive stream, sends semantics first and pixels later: it runs a detector and a tracker onboard and sends short records (25 bytes on average) of multi-frame events selected by a mission card of 16-125 bytes, ahead of positions and pictures. We measure it by what a ground-side language model answers from it, under a preregistered protocol with a held-out set, against H.265 read by the vision fine-tune of the same model, H.265 on which the ground runs our own perception, a stream of quantized tracks, and onboard captions. On the development set (84% of questions about convergence, mostly one location) the event stream answers 74.3% of questions at 0.13 kbps. A reader that receives nothing answers 57.4%, and the video reader 59.6% at 100 kbps and 64.7% on the source frames. When the ground instead runs our perception on the video, video matches the stream at 100 kbps, about 800 times the stream's rate, and falls behind at 64 kbps and below. A track stream matches it at 96 times the bytes and loses 0.03 at 17 times. On held-out data the result is weaker: with events taken from annotated tracks, an oracle, the stream beats H.265 at 100 kbps (+0.13), but with onboard perception it does not (+0.01 [-0.04, +0.06]) and is no better than receiving nothing. Sent alone through emulated links the event records answer 0.739 at 1 Mbps and still 0.724 at 0.3 kbps; an every-frame track stream holds down to 1 kbps and video with ground-side perception down to about 100 kbps. On a Jetson Orin Nano the unchanged pipeline runs at 4.5 fps and misses most events.

cs.RO · cs.CV

版本更新 · v4arXiv:2506.02849

Learned Controllers for Agile Quadrotors in Pursuit-Evasion Games

Alejandro Sanchez Roncero, Yixi Cai, Olov Andersson, Petter Ogren

原文摘要

We study one-versus-one (1v1) quadrotor pursuit-evasion, where a pursuer and an evader are trained with reinforcement learning (RL). Direct adversarial RL tends to be unstable because opponent policies change during training, causing non-stationarity, forgetting, and strategy cycling. We propose Asynchronous Multi-Stage Population-Based training, inspired by Policy-Space Response Oracles (PSRO). Proximal Policy Optimization (PPO) trains responses against previously learned opponents held fixed during each stage. The uniform variant, AMSPB-U, samples opponents equally, while AMSPB-P combines hard prioritized fictitious self-play (PFSP) with uniform exploration to emphasize currently difficult opponents. In cross-evaluations against learned opponents, both variants improve pursuit-evasion balance over latest-opponent training. AMSPB-P scores highest while retaining competence against earlier strategies. Across varied starting positions and after larger-arena retraining, both population variants retain higher average success across the two roles than latest-opponent training. Hardware experiments with two 10 cm, approximately 40 g Crazyflie brushless quadrotors demonstrate zero-shot sim-to-real transfer against baseline controllers and learned adversaries across 42 flights. Recorded flights include captures of evaders reaching robust peak speeds of 3.31 m/s and accelerations of 17.95 m/s^2, with pursuer flight speeds up to 4.87 m/s.

cs.RO · cs.LG

版本更新 · v4arXiv:2609.34982

ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuning

Di Zhu, Ziheng Yan, Fang Wan

原文摘要

Vision-Language-Action (VLA) models have shown great promise for robotic manipulation by mapping multi-modal semantics to physical actions. However, this mapping inherently struggles to align these coarse-grained semantics with fine-grained temporal execution. It leaves VLA models with limited generalization and insufficient robustness in cluttered environments. To overcome this issue, we propose ActionUNet, an efficient multi-scale fine-tuning framework that enhances pre-trained VLA models with minimal computational cost. ActionUNet first constructs a lightweight temporal U-Net within the temporal-aligned action feature space to fuse hierarchical structural priors, effectively bridging the scale gap between semantics and temporal executions. Recognizing that multi-scale modeling can disrupt microscopic temporal continuity and cause mechanical oscillations, ActionUNet then employs a conditional SIREN as a continuous action decoder. Equipped with explicit second-order smoothness constraints, this decoder guarantees temporal continuity and reduces high-frequency motion jitter. By smoothing temporal discontinuities from multi-scale fusion, this continuous formulation reduces mechanical execution failures while preserving the base VLA model's generalization and manipulation robustness. Extensive experiments on RoboTwin 2.0 and LIBERO-Plus benchmarks, together with real-world hard evaluations, demonstrate that ActionUNet significantly improves π0.5 success rates by absolute 9.8%, 6.1%, and 11.4%, respectively, while also generalizing to the regression-based OpenVLA-OFT backbone, highlighting its effectiveness and efficiency as a fine-tuning strategy. Code and implementation details are available at https://github.com/Di-Zhu123/ActionUNet.

cs.CV · cs.RO

版本更新 · v5arXiv:2502.04873

Training-free Task-oriented Grasp Generation

Jiaming Wang, Jizhuo Chen, Diwen Liu, Harold Soh

原文摘要

This paper presents a training-free pipeline for task-oriented grasp generation that combines pre-trained grasp generation models with vision-language models (VLMs). Unlike traditional approaches that focus solely on stable grasps, our method incorporates task-specific requirements by leveraging the semantic reasoning capabilities of VLMs. We evaluate five querying strategies, each utilizing different visual representations of candidate grasps, and demonstrate significant improvements over a baseline method in both grasp success and task compliance rates, with absolute gains of up to 36.9\% in overall success rate. Our results underline the potential of VLMs to enhance task-oriented manipulation, providing insights for future research in robotic grasping and human-robot interaction.

cs.RO

新论文 · v1arXiv:2611.19387

Long-Horizon Causal Inertial Odometry in Multistory Buildings

Tianzi Zhang, Yan Wang, Hao Qiao, Jian Kuang, Xiaoji Niu

原文摘要

In indoor 3D localization inside multistory buildings, an elevator or an escalator accelerates for a few seconds and then cruises at constant velocity for tens of seconds. Gait excitation disappears, the specific force degenerates into a constant gravity reaction, and instantaneous velocity becomes unobservable within a 1-2 s window. The output of short-window learned inertial odometry consequently degrades toward the conditional mean of the training set, and vertical displacement is systematically underestimated. We propose Local-Token Causal Attention (LTCA), a long-horizon architecture that compresses each 1 s inertial window into a 1 Hz motion token and sustains, over a bounded 60 s causal cache, the motion state established at the onset event. We further introduce a pluggable barometer branch that, when online, supplies a direct elevation constraint to the vertical channel through local differential height observations, and we train it with paired online/fallback conditional training so that the barometer also regularizes the representation shared by the two modes during training. On MSIO, our benchmark of nine buildings, the RoNIN and TLIO instances reach 3D trajectory errors (ATE-3D) of 2.83/4.18 m and 2.44/5.12 m on the Seen and Unseen splits, respectively, with the barometer online. With the barometer disabled at inference, the same weights fall back to pure inertial inference and reach 3.24/4.28 m and 2.64/5.23 m, still below every native short-window baseline. We further validate the framework on the public RoNIN and TLIO benchmarks.

cs.RO

新论文 · v1arXiv:2611.19381

AcousticBEV: Acoustic BEV Object Localization for Intelligent Vehicles

Ziang Liu, Shiming Wang, Joseph Adeola, Julian F. P. Kooij

原文摘要

Modern autonomous driving and driver assistance systems rely predominantly on cameras, radars and LiDARs for environmental perception. However, these modalities face different environmental limitations, including poor illumination for cameras and adverse weather for LiDAR. Acoustic sensing, particularly using microphone arrays, could provide complementary perception capabilities for autonomous driving, but has remained largely underexplored. In this work, we design a horizontally arranged microphone array that functions as the "ears" of a vehicle, enabling localization of nearby moving vehicles in a bird's-eye view (BEV) representation that captures both their relative direction and distance. Using this array, we collect a real-world driving dataset, AcousticScenes, which contains 2,117 static and 2,863 dynamic samples with synchronized spatial acoustic signals, LiDAR point clouds and 3D bounding box annotations. Building on this dataset, we propose AcousticBEV, a learning-based method for acoustic BEV object localization, which estimates the center locations of moving traffic participants in BEV from the microphone-array signals, conditioned on the ego vehicle's own speed. Experimental results demonstrate that AcousticBEV outperforms both classical signal-processing baselines and other learning-based methods, achieving F1 scores of $47.7 \pm 1.2$% on "Easy" scenarios and $45.1 \pm 1.4$% on "Difficult" cases. Our findings demonstrate that acoustic signals from a low-cost, vehicle-mounted microphone array can localize nearby moving vehicles in both azimuth and distance.

cs.CV · cs.RO

版本更新 · v2arXiv:2610.00921

World-Model Gradient Refinement in CEM: An Empirical Study

Shizhe Cai, Frieder Stolzenburg, Oliver Obst

原文摘要

The cross-entropy method (CEM) uses world-model predictions both to select action sequences and to determine where to sample next. We isolate the effect of an early update by replacing only the first model-ranked elite set with environment-ranked elites. This fixed-state intervention lowers the selected sequence's realised tracking cost in all 12 Walker/Cheetah task-seed units. We then study short gradient updates to top-ranked candidates as a way to improve search under the model's own objective. Across 52 frozen TD-MPC2 checkpoints from 32 tasks, we compare refinement with additional CEM sampling at the same planning budget, tuning both planners' initial proposal widths. We fix the groups before this comparison using an exploratory model-only diagnostic. Refinement gains 67.8 return points on average (95\% CI [31.9, 103.6]) on ten checkpoints with large model-predicted improvements, and 0.2 ([$-2.9$, 3.4]) on the remaining 42. The diagnostic's rank test on 24 new checkpoints fails, so the association does not establish a predictive rule. Offline analyses show local benefits on some tasks and positive estimates for both proposal refitting and final selection. End-of-search polishing also helps, leaving the need for repeated refitting unresolved.

cs.LG · cs.RO

版本更新 · v2arXiv:2608.13904

Communication in modular robotic motor control: Bilateral controllers under realistic constraints

Jingwen Li, Levin Kuhlmann, Jason Friedman, Gideon Kowadlo

原文摘要

Robotic motor control requires accurate movement and stable posture under signal-dependent noise and energetic constraints. Inspired by bilateral organisation in biological motor control, we study whether two recurrent controllers outperform a monolithic controller and how learned inter-module communication and its delay affect control. We train two GRU-based modules with a learnable, delayed communication channel in a two-arm musculoskeletal simulator. Success rate combines reaching accuracy (endpoint error below 2 cm) and holding stability (variance below 2.5e-5 m^2). The modular architecture achieves 87.6% +/- 13.8% success, compared with 68.9% +/- 27.8% for a parameter-matched monolithic controller. Within the communicating bilateral architecture, non-zero delays increase success rate by 11.3 percentage points and reduce energetic cost by approximately 17% relative to zero delay. Compared with a bilateral controller without communication, learned communication reduces endpoint and trajectory errors by approximately 27% and 52%, movement variance by 13%, and co-contraction index by 4%. These findings suggest that modular organisation supports motor control under noise and energetic constraints, while communication and its timing influence trade-offs among precision, stability, and energetic cost.

cs.RO

新论文 · v1arXiv:2611.19324

MARS: Dynamic Routing of Reinforcement-Learned Atomic Skills for Multi-Stage Robotic Manipulation

Zequan Zeng, Junqin Yang, Kai Wang

原文摘要

Vision-Language-Action (VLA) models have demonstrated strong capabilities in multi-task robotic manipulation. However, they still face limitations in control precision and execution stability during critical manipulation stages, such as object grasping, alignment and insertion. To combine general-purpose manipulation capabilities with task-specific fine-grained control, while enabling effective coordination between generalist and specialized policies throughout multi-stage manipulation tasks, this paper proposes MARS (Modular Atomic-skill Routing System), a dynamic routing framework for reinforcement learning-based atomic skills. MARS integrates a frozen generalist VLA policy with multiple independently trained atomic skill policies. Using a shared RL Token state representation and the reference action of VLA, specialized control policies are optimized through reinforcement learning for different cr`itical manipulation stages. For high-level policy selection, we develop a dynamic router that integrates temporal context and skill semantics. The router employs Mamba-based recurrent encoding and a historical feature window to capture state evolution during manipulation. Skill descriptions of candidate policies are encoded into semantic queries, and attention mechanisms are used to establish correspondences between skill semantics and temporal state features. Experimental results show that MARS achieves an average Macro-F1 score of 90.47% across four LIBERO task suites under fixed-trajectory evaluation, outperforming an MLP-based router by 7.08 percentage points. In the simulated Tube Transport task, the full-task success rate increases from 20.04% to 85.67%. These results demonstrate that stage-specific reinforcement learning and temporal state-aware dynamic policy routing can improve multi-policy coordination and overall task execution performance in multi-stage robotic manipulation.

cs.RO