Yuheng Zhang, Mingyue Huo, Minghao Zhu, Mengxue Zhang, Nan Jiang
Reward models (RMs) are widely used as optimization targets in reinforcement learning from human feedback (RLHF). We show that leading RMs can assign higher scores to gibberish than to meaningful reference answers. To systematically discover such inputs, we introduce Token Mapping Perturbation Attack (TOMPA), a framework that uses a perturbed token mapping to extend an attack LLM's search beyond natural language. The mapping changes the linguistic meaning of the attack LLM's natural language responses, producing non-linguistic text for the target RM to score. We then use these scalar scores as rewards to train the attack LLM through reinforcement learning to discover non-linguistic adversarial inputs. Experiments against Llama and Qwen RMs demonstrate such attacks using two instantiations: identity token mapping across different tokenizers and random permutation with a shared tokenizer. Against Skywork-Reward-V2-Llama-3.1-8B, ranked first on RewardBench 2, the two mappings achieve mean rewards of +33.64 and +43.78, respectively, compared with +17.48 for GPT-5 reference answers. Their attack success rates are 97.6% and 100%, measured as the percentages of individual responses scoring above their corresponding GPT-5 references. These results show that strong benchmark performance does not ensure robustness to non-linguistic adversarial inputs, exposing a fundamental vulnerability in using RMs as optimization targets.
Learning controllable world dynamics from heterogeneous interaction videos is challenging because robot embodiments use incompatible action spaces, while human videos lack native control labels. We present RynnWorld-Latent, a cross-embodiment video world model that uses the 3D-grounded latent actions introduced by RynnLAM as a shared conditioning interface. This representation enables joint pretraining on human interaction videos, robot demonstrations, and simulation rollouts without requiring a common native action space. Built on a pretrained video transformer, RynnWorld-Latent combines global action tokens with an identity-initialized local pathway that aligns actions with future video latents. We pretrain on 23 datasets comprising approximately 553.5M frames and adapt the model to target robots through post-training. The framework supports two data-generation routes: digital teleoperation uses target robot commands to generate action-video demonstrations, while latent-action synthesis transfers source motion to an edited initial frame and pairs the resulting rollout with retargeted pseudo robot actions. Experiments show improved video prediction metrics on public benchmarks and two real-robot platforms, alongside qualitative action following when different action sequences share the same initial observation. Synthetic mid-training improves policy success rates on LIBERO and LIBERO-Plus, and both generation routes improve real-world policy success rates over real-demonstration-only training in the evaluated settings. These results support universal latent actions as a scalable interface for learning controllable visual dynamics across heterogeneous embodiments.
Yerong Wu Southeast University, Tianxing Wu, Minghao Zhu Alibaba Group, Hangyu Sha Southeast University, Haofen Wang Tongji University
Achieving realistic human-like conversation for virtual characters requires not only a simple memorization and recall of past events, but also the strategic utilization of memory to meet factual needs and social engagement. Current memory utilization relevant (e.g., memory-augmented generation, long-term dialogue, and etc.) benchmarks overlook this nuance, treating memory primarily as a static repository of facts rather than a dynamic resource to be strategically deployed in dialogues. To address this gap, we design StratMem-Bench, a new benchmark to evaluate strategic memory use in character-centric dialogues. This dataset comprises 657 instances where virtual characters must navigate heterogeneous memory pools containing required, supportive, and irrelevant memories. We also propose a framework with different evaluation metrics including Strict Memory Compliance, Memory Integration Quality, Proactive Enrichment Score and Conditional Irrelevance Rate, and to evaluate strategic memory use capabilities of virtual characters. Experiments on StratMem-Bench which leverage the state-of-the-art large language models as virtual characters show that all models perform well at distinguishing between required and irrelevant memories, but struggle once supportive memories are introduced into the decision process.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表