跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+197新增论文
+151其中 AI 相关
+16已有论文修订
+91新增 AI 导读

arXiv 最近完成扫描:2026-11-21 12:19 · 全站最近入库:2026-11-21 12:19各信源进度 →

统计口径与数据新鲜度

统计 2026-11-21 04:29 — 2026-11-22 04:29(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-21 12:19最近发现新增版本:2026-11-21 12:19库内论文最新更新时间:2026-11-20 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

29,158 已收录论文24,181 AI 相关8,992 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 17 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-21 12:17(北京时间)
版本更新 · v2arXiv:2608.27496

ROPE:面向间接提示注入的路由来源策略执行

ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection

Xinhang Ma, Chaowei Xiao, William Yeoh, Ning Zhang, Yevgeniy Vorobeychik

AI 速读 基于摘要

ROPE 利用系统信任根为敏感工具参数按任务生成来源策略,并在工具执行前做确定性校验,以在抵御间接提示注入的同时尽量少损失效用。

cs.CR

查看原文摘要

Tool-using LLM agents execute consequential operations on behalf of users. Because the LLM driving an agent processes trusted user instructions and untrusted content in the same context, agents are vulnerable to indirect prompt injection attacks that induce unintended actions by mixing malicious instructions into benign data. Defenses based on prompting or content detection are easily evaded by paraphrasing attacks, while system-level defenses that constrain tool calls using inferred user intent, or restrict which sources can supply values to sensitive tools, greatly compromise utility. We present ROPE (Routed Origin Policy Enforcement), a defense that greatly improves the utility-security tradeoff by leveraging system roots of trust to create and enforce per-task policies on sensitive tool parameters. Our key insight is that authentication and integrity protection provide a basis for identifying the source of content independently of instructions embedded in it. Before a task unfolds, an LLM router reads only trusted task input to set a provenance rule for each sensitive tool parameter, accounting for the nature of the task and the values and sources the user specified. As the agent works, an origin labeler records where each piece of content came from, using the user's request, platform metadata, and protected record fields as trust anchors. Before any sensitive tool executes, a deterministic verifier requires each of its origin-guarded values to match a value in a source its rule accepts. We show that ROPE provably guarantees parameter value provenance to sensitive tool calls and is immune to paraphrasing attacks. Against eleven defenses on AgentDojo and AgentDyn under static, adaptive, and long-horizon attacks, ROPE exhibits near-perfect security while compromising far less utility than the best baseline defenses. Code and data are available at https://github.com/xhOwenMa/ROPE.

版本更新 · v3arXiv:2509.10780

How K-12 Teachers Adapt Generative AI to Local Classroom Demands: Cultural Distance Between GenAI Defaults and Everyday Teaching Practices

Qing Xiao, Ruiwei Xiao, Hanqi Jane Li, Phenyo Phemelo Moletsane, Xinying Hou, John Stamper, Hong Shen

原文摘要

Generative AI (GenAI) is entering K-12 classrooms, but its default culture does not always fit local teaching. Existing accounts either treat this misfit as macro-level structural bias or examine it through single-case studies; neither captures how much work teachers must do to make outputs usable across tasks. This exploratory, theory-building qualitative study introduces cultural distance, the gap between GenAI's default cultural repertoire and the situated demands of teachers' practice. Through interviews with 30 K-12 teachers who frequently use GenAI, 10 each from South Africa, Taiwan, and the United States, we identified three levels defined by the effort teachers invested and whether it yielded a usable output: low (minor edits), mid (sustained prompting and revision), and high (structural gaps rooted in unsupported languages, undigitized knowledge, or policy filters that no teacher effort can close). We present cultural distance as a promising framework requiring further examination and draw implications for designers, policymakers, and educators.

cs.HC · cs.AI

新论文 · v1arXiv:2611.20291

Where the Savings Go: The Limits of Hiding Cross-Tenant Cache Hits in LLM Serving

Bowen Sun, Chaowei Xiao

原文摘要

LLM serving engines cache prompt prefixes; a cache hit skips computation and returns faster, so a shared cache reveals whether others have sent a given text. Per-account salting closes this channel but forgoes cross-tenant reuse and isolates only whole accounts, even where access is redistributed or trust domains are subdivided below them. Defenses that share globally but hide hits promise reuse without relying on account boundaries. Hiding a hit requires everything tenants observe afterwards to be distributed as if it had missed. The defenses we examine target the hit-miss gap; we instead follow where the saved time goes. Step-based schedulers that never idle while work waits pass it in full to every request queued behind the hit, shifting each earlier by the same amount. An attacker within ordinary rate limits brackets a probe with two groups of listeners that each return one token, sends them in one short burst, and counts in how many steps they finish. Replaying a production trace on one vLLM instance at up to half capacity, a single probe detects cached texts of 256 tokens or more with advantage at least 0.98. Hiding only the hit request falls to one probe on vLLM and SGLang; random delays must reach about ten times the shift or more to cut the advantage to 0.1; narrowing what is shared gives no guarantee against this inference. Since any observable tenant benefit is also a signal, we prove that a rate-limited tenant's gain from content written by k other accounts is at most of order k times the leakage allowed, whatever the provider does with identities; at zero leakage, only the provider keeps the savings. Exact hiding is attainable in principle on one instance, but mechanisms that only pause or hold work know skipped work's duration only approximately, so under saturated prefill they must pad every step that may hold a hit, at a price that grows with time rather than with hits.

cs.CR

新论文 · v1arXiv:2611.17859

Per-Agent-Safe Navigation of Globally Driven Robot Collectives via Affine Field Design

Seunghwan Jang, Steven Ceron, Wei Xiao

原文摘要

Safe navigation of robot collectives requires coordinating motion and deformation so that the collective can traverse constrained environments while ensuring each robot's safety. Existing controllers for swarmalator collectives largely rely on local control, where each robot receives an individual command, which causes the control and communication requirements to grow with the number of agents. We introduce a broadcast control formulation in which a single spatially affine field governs both the motion and deformation of the collective. The scalar component of the field regulates collective size, and its spatial gradient generates collective translation through non-reciprocal interactions. Both components enter the collective dynamics directly, which allows per-robot wall, actuation, and cohesion constraints to be enforced using control barrier functions within a single quadratic program for the whole collective. The resulting controller continuously adapts the collective size to the local environment, and enables safe passage through corridors narrower than the unconstrained collective without pre-planned resizing. We further derive how the available actuation budget determines the reachable speed and minimum passable corridor width, and we demonstrate that two independently driven collectives can simultaneously traverse a shared junction using distinct fixed phase labels without inter-collective coordination.

cs.RO

版本更新 · v2arXiv:2602.16033

衔接生成式AI政策与实践:简短指令可提升编程入门课程中以学习为导向的提示能力

Bridging GenAI policy and practice: Brief instruction improves learning-oriented prompting in an introductory programming course

Ruiwei Xiao, Runlong Ye, Xinying Hou, Jessica Wen, Harsh Kumar, Michael Liut, John Stamper

AI 速读 基于摘要

一项随机对照试验发现,在编程入门课程中,从两句话提醒到多步骤建构任务的四种教学干预都能提升学生以学习为导向的提示能力,效果持续六周并随设计投入程度递增,但未体现在期末考试成绩上。

cs.HC · cs.AI

查看原文摘要

Generative artificial intelligence (GenAI) is now commonplace in higher education, yet unguided use can harm learning. Universities have responded with policies asking students to use GenAI for learning, not solutions, but how to translate these policies into classroom instruction remains untested. Here we report a randomized controlled trial with a six-week, end-of-semester follow-up (N = 431) in an introductory programming course comparing four instructional approaches, from a two-sentence reminder (Condition 1) to a multi-step constructive task (Condition 4). We found: (1) all four approaches produced learning-oriented prompting gains that persisted six weeks later and scaled with the designed engagement level, from 8 to 51 percentage points; (2) students in every condition self-reported sustained voluntary use, and confidence grew in step with skill; and (3) final examination performance did not differ between conditions, placing the possible boundary of the effects at the pathway from prompting skill to achievement. These findings provide instructors with an empirically calibrated set of options for translating GenAI-use policies into classroom practice.

版本更新 · v3arXiv:2606.22673

Agent MechSuits: Mechanistic Subspace Safety Steering for Multi-Turn CLI Agents

Weidi Luo, Qiming Zhang, Yihao Quan, Mingyu Jin, Jie Cai, Chaowei Xiao, Jingcheng Niu, Zhen Xiang

原文摘要

Command-Line Interface (CLI) agents based on large language models (LLMs) demonstrate remarkable autonomous capabilities, but they also introduce significant safety and misuse risks during multi-turn interactions with external environments. Existing safety mechanisms mainly rely on external guardrails, which have a limited ability to perform fine-grained behavioral control during execution. Meanwhile, recent mechanistic interpretability methods for LLM safety are mostly confined to single-turn or jailbreak-style QA settings, limiting their ability to capture the evolving risk dynamics of multi-turn agent execution. In this paper, we investigate the safety of multi-turn CLI agents from an internal perspective. We propose Agent MechSuits (Mechanistic Subspace Intervention and Steering), a white-box defense framework that performs runtime safety detection and representation-level mitigation for CLI agents. Unlike conventional agent guardrails, Agent MechSuits detects harmful execution states from step-level hidden representations and mitigates unsafe behavior by intervening in a 10-dimensional subspace within a single layer. To support this research, we introduce the Mechanistic Agent Safety (MAS) benchmark, comprising comprehensively annotated multi-turn execution trajectories across 194 tasks using LLaMA-3.1-8B, Qwen-2.5-7B, and Gemma-2-9B. Extensive experiments show that Agent MechSuits achieves strong safety detection performance, provides preliminary evidence for lookahead risk anticipation, and substantially reduces harmful actions of the CLI agent, establishing a foundation for applying mechanistic interpretability to dynamic LLM agent safety.

cs.AI · cs.SE

新论文 · v1arXiv:2611.13741

Retargetable Compilation for Tensor Kernels via a Unified Layout Abstraction

Likai Ma, Renze Chen, Tianqi Li, Yuhao Luo, Youwei Xiao, Size Zheng, Xiuhong Li, Yun Liang

原文摘要

Modern hardware accelerators achieve high performance through specialized instructions coupled with tightly managed memory hierarchies. These instructions impose strict layout requirements on their operands: where tensor fragments reside, how they are organized, and how parallel units access them. Such instruction-visible layout is critical to kernel performance, as it determines whether instructions execute correctly and how much data movement they incur. However, existing compilers either scatter layout requirements across handwritten programs and local scheduling rules, or provide layout representations and conversions specific to a single class of targets. Both approaches expose the same abstraction gap: the absence of a unified model for instruction layout requirements across backends. This paper presents Arknife, a retargetable compiler that treats instruction-visible layout as first-class instruction semantics. Its central idea is a constraint-based layout algebra: tensor layouts map logical axes to hardware axes spanning parallel levels and memory spaces. Instruction operands declare layout requirements, and target-provided repairs resolve conflicts. Layouts are then solved globally across the program. ISA-specific layout facts are supplied by target-side instruction descriptions rather than hardcoded in the compiler. Retargeting reuses the layout algebra and solver while defining the new target's instructions and their operand requirements. We evaluate Arknife across x86 CPU, Hexagon NPU, and NVIDIA A100 GPU. Arknife matches cuBLAS on A100, and on x86 and Hexagon outperforms vendor-optimized libraries by $2.17\times$ and $3.78\times$ and the retargetable compiler MISAAL by $5.08\times$ and $4.53\times$, respectively.

cs.PL

新论文 · v1arXiv:2611.13496

Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation

Jiawei Xiao, Boya Wang, Longtian Qiu, Shan Ning, Xuming He

原文摘要

Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: https://artanic30.github.io/project_pages/RaRe.

cs.CV · cs.AI

版本更新 · v2arXiv:2608.00635

FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation

Runqing Wang, Ding Yu, Pengyuan Min, Xinhong Zhang, Wei Xiao, Yu Hu, Jie Chen, Fu Zhang 等 9 位作者

原文摘要

We present FlowPilot, the first world-action model for real-time onboard UAV navigation from depth. Unlike map-then-optimize pipelines that require local reconstruction or end-to-end policies that lack explicit scene prediction, FlowPilot jointly denoises future depth observations and executable trajectories with flow matching. A dual-stream mixture-of-transformers couples video and action experts through shared attention, allowing future-scene prediction and trajectory generation to inform each other. At deployment, the model runs action-centrically and outputs only a trajectory. To generate smooth, state-consistent references, actions are parameterized as degree-7 Bernstein polynomials: the current state constrains the initial control points, and the network predicts five free control points, yielding C^2-continuous references with closed-form velocity, acceleration and jerk. FlowPilot is trained on a three-level depth pyramid spanning high-throughput simulation, photorealistic simulation, and real onboard data. In closed-loop simulation, it outperforms learning- and optimization-based baselines under increasing clutter and commanded speeds up to 10 m/s. On a physical quadrotor, FlowPilot achieves a mean inference latency of 10.2 ms on a Jetson Orin NX and reaches a peak speed of 10 m/s during outdoor flight using only onboard sensing and computation.

cs.RO

新论文 · v1arXiv:2611.01483

$TF^2$-Steering: Action Refinement with Tactile Forecasts and Feedback via Velocity-Field Steering

Yiming Chen, Xiaofei Wei, Chun Gu, Wei Xiao, Peidong Liu, Li Zhang

原文摘要

Tactile sensing plays a crucial role in everyday human interaction. Consider inserting a plug into a tight socket, vision alone provides limited guidance during the final insertion stage, and we instead rely on both anticipated contact and realized tactile feedback to ajust the insertion motion. The same capability is equally important for robots, particularly in dexterous manipulation tasks. Yet many contemporary robot policies remain vision-dominant. This visual bias, compounded by limited tactile data and sparse contact events, can cause directly incorporated tactile signals to be ignored or even compromise the policy's pretrained capabilities, yielding degrading performance as observed in our experiments. We present $TF^2$-Steering, which refines actions via learned residual steering of the flow-matching action expert's velocity field, using a tactile mixture-of-experts architecture to integrate both tactile forecasts and real-time tactile feedback. Together with specific designed three stages training recipe, $TF^2$-Steering enables tactile signals to correct and, when necessary, redirect the flow-matching action-generation process, while building on the strong action-generation capacity of the vision-based foundation policy. Extensive experiments across six simulated and six real-world manipulation tasks demonstrate the effectiveness of $TF^2$-Steering. It outperforms the strongest evaluated baselines by 10.7 percentage points in average simulation success rate and 9.1 percentage points in average real-world progress success rate. Project page: https://yiming-cc.github.io/tf2-steering/.

cs.RO

版本更新 · v2arXiv:2602.20076

面向安全关键系统的鲁棒Taylor-Lagrange控制:保证可行性

Robust Taylor-Lagrange Control for Safety-Critical Systems with Guaranteed Feasibility

Wei Xiao, Christos Cassandras, Anni Li

AI 速读 基于摘要

提出鲁棒Taylor-Lagrange控制(rTLC),通过Taylor展开与Lagrange余项将安全函数展开到高于相对阶的阶数,以缓解离散化下的可行性保持问题并用于最优控制。

eess.SY · cs.AI · cs.RO

查看原文摘要

The Control Barrier Function (CBF) method has been widely adopted in solving safety-critical control problems. However, the existence of a CBF is only a sufficient condition for system safety. The recently proposed Taylor-Lagrange Control (TLC) method addresses this limitation by providing a necessary and sufficient condition, but it is still vulnerable to the feasibility preservation problem (e.g., due to the inter-sampling effect when time is discretized). In this paper, we propose a robust TLC (rTLC) method to address the feasibility preservation problem and show how to use it to solve optimal control problems. Specifically, the rTLC method expands the safety function to an order higher than its relative degree using Taylor's expansion with Lagrange remainder, which allows the control to explicitly show up at the current time instead of a future time in the original TLC method. This implies that the safety constraint remains satisfied over the entire discretization time interval that follows. Moreover, the rTLC method naturally addresses the feasibility preservation problem with only a single hyper-parameter (the discretization time interval size during implementation), compared to significantly more parameters for its counterparts. Finally, we illustrate the effectiveness of the proposed rTLC method through an adaptive cruise control problem, and compare it with existing safety-critical control methods.

版本更新 · v4arXiv:2603.22016

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

Xinyan Wang, Xiaogeng Liu, Ming Pei, Chaowei Xiao

原文摘要

Large reasoning models (LRMs) often suffer from overthinking, continuing with redundant verification and repeated attempts even after reaching a correct solution. This wastes computation and can overturn an initially correct answer. We show that the transition from productive to redundant reasoning is decodable from late-layer hidden states. These representations separate the two phases around first-correct-solution boundaries beyond what position and lexical controls explain, and distinguish correct from incorrect solution endings. Building on this finding, we propose ROM (Real-time Overthinking Mitigation), which uses a lightweight streaming detector (0.07--0.25\% of backbone parameters) to monitor a frozen LRM and intervene at well-formed reasoning boundaries without answer extraction or probe decoding. Counterfactual Self-Correction (CSC) augments detector training with wrong$\rightarrow$correct trajectories to preserve useful self-correction. Heads are trained separately for each backbone using shared MATH500 supervision and transfer zero-shot to four unseen benchmarks. Evaluated against ten baselines across five backbones from three families and five benchmarks, ROM$_{\text{CSC}}$ reduces response length by 28--77\% (45\% on average), with significant reductions in all 25 settings and accuracy matching or exceeding vanilla decoding in 22. Measurements on two backbones further show 20--70\% lower end-to-end latency with less than 0.5\% detector overhead per decoding step. Code is available at https://github.com/SaFo-Lab/ROM.

cs.LG · cs.AI · cs.CL

新论文 · v1arXiv:2610.39122

Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection

Xinyan Wang, Xiaogeng Liu, Jun Shao, Chaowei Xiao

原文摘要

Self-evolving personal agents act for users and learn from user feedback. However, if an agent obeys an instruction that an untrusted third party hides in content, the third party gains control of the agent. Such indirect prompt injection (IPI) poses severe risks. We investigate whether self-evolution degrades the robustness of personal agents to IPI. To study this question, we design a controlled two-phase methodology. In the adaptation phase, the user asks the agent to read external content and complete the task described in it, a common request in daily use. The agent then learns from the user's feedback and improves itself as designed. In the test phase, a third party writes a similar task into new external content, but the user's test request does not authorize the injected task. Using this methodology, we build multiple tasks with diverse test instances in different domains for a real personal agent. Our extensive experiments reveal that self-evolution substantially degrades robustness to IPI. Specifically, compared with the base agent, which does not evolve from user feedback, the average attack success rate rises by 14.0--48.8 percentage points under skill evolution and by 3.0--39.3 percentage points under reinforcement learning, posing safety risks. The code is available at https://github.com/SaFo-Lab/Evolve-IPI

cs.CR · cs.AI

新论文 · v1arXiv:2610.38895

FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models

Xiaogeng Liu, Xinyan Wang, Chaowei Xiao

原文摘要

Typed decision models are an emerging class of language models that make decisions instead of generating free-text responses. They answer in a single fast and cheap query and are increasingly used, including in security-sensitive tasks. However, red teaming of these models leaves two gaps: existing adaptive attacks do not use their speed, and few studies have searched for universal adversarial examples. We fill both gaps on Jev, the representative typed decision model, with a quick black-box query-based attack that calls no attacker language model. We conduct extensive experiments on 12 datasets with different data types and application scenarios. The results show that Jev resists common prompt injections: no strategy changes a correct answer in 10,240 paired tests. In contrast, our attack finds short suffixes that flip confident answers within minutes. These suffixes are universal: one suffix per dataset flips 43% of unseen documents. Because typed decision models, unlike classic NLP models, can serve many tasks, we test whether these suffixes transfer across tasks. We find that one suffix generated from one task can flip 90-100% of the samples in other tasks. We therefore believe the evaluation of typed decision models must include strict adaptive attacks to fully reveal their robustness.

cs.CR · cs.AI

新论文 · v1arXiv:2610.35093

Improving Instruction Hierarchy by Learning from Goal-Goal Conflicts

Sizhe Chen, Sanjay Kariyappa, Liwei Jiang, Chong Xiang, Jihye Choi, Chaowei Xiao, G. Edward Suh

原文摘要

The recent incident in which OpenAI agents autonomously compromised Hugging Face's infrastructure illustrates the importance of instruction hierarchy (IH): an LLM should follow lower-priority instructions, such as user requests, only when they do not conflict with higher-priority instructions, such as system policies. IH can be effectively improved by fine-tuning LLMs on simulated examples of instruction conflicts, whose diversity is key to teaching LLMs to properly handle conflicts at test time. However, our decomposition of a task into goals and constraints reveals a conflict type -- goal-goal conflicts -- that is absent from existing IH training sets from OpenAI and NVIDIA. We show the importance of this conflict type: fine-tuning Qwen3.5-9B and Gemma4-12B-it on examples of this conflict type significantly improves IH on a wide range of real-world tasks unseen in training; adding these examples to existing IH training sets for fine-tuning also gives better IH adherence. The IH gains come with trivial mean utility changes on no-conflict tasks across eight benchmarks. Our work advocates including this new conflict type in IH training, even if these examples do not reflect typical LLM use.

cs.AI

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-demo.2ACL Anthology · ACL 2026

Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks

Guangwei Zhang, Jianing Zhu University of Texas at Austin, Cheng Qian University of Illinois at Urbana-Champaign, Neil Zhenqiang Gong Duke University, Rada Mihalcea University of Michigan, Zhaozhuo Xu Workato and Stevens Institute of Technology, Jingrui He University of Illinois at Urbana-Champaign, Jiaqi W. Ma University of Illinois Urbana-Champaign 等 14 位作者

原文摘要

We present Copyright Detective, the first interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs. The system treats copyright infringement versus compliance as an evidence discovery process rather than a static classification task due to the complex nature of copyright law. It integrates multiple detection paradigms, including content recall testing, paraphrase-level similarity analysis, persuasive jailbreak probing, and unlearning verification, within a unified and extensible framework. Through interactive prompting, response collection, and iterative workflows, our system enables systematic auditing of verbatim memorization and paraphrase-level leakage, supporting responsible deployment and transparent evaluation of LLM copyright risks even with black-box access. In our experiments with GPT-4o-mini, we demonstrate that the specific persuasive strategy “Pathos” shifts the leakage distribution from about 0.1 (ROUGE-L) to 0.7. Our live system is hosted on Streamlit server, with a demonstration video included as supplementary material.

cs.CL

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) · 2026 · 正式发表

首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.502ACL Anthology · ACL 2026

Defenses Against Prompt Attacks Learn Surface Heuristics

Li Li University of Southern California, Chenxiao Yu University of Southern California, Zhiyu Ni University of California, Berkeley, Hao Li Washington University, Saint Louis, Charith Peris Amazon, Chaowei Xiao, Yue Zhao University of Southern California

原文摘要

Large language models (LLMs) are increasingly deployed in security-sensitive applications, where they must follow system- or developer-specified instructions that define the intended task behavior, while completing benign user requests. When adversarial instructions appear in user queries or externally retrieved content, models may override intended logic. Recent defenses rely on supervised fine-tuning with benign and malicious labels. Although these methods achieve high attack rejection rates, we find that they rely on narrow correlations in defense data rather than harmful intent, leading to systematic rejection of safe inputs. We analyze three recurring shortcut behaviors induced by defense fine-tuning. Position bias arises when benign content placed later in a prompt is rejected at much higher rates; across reasoning benchmarks, suffix-task rejection rises from below 10% to as high as 90%. Token trigger bias occurs when strings common in attack data raise rejection probability even in benign contexts; inserting a single trigger token increases false refusals by up to 50%. Topic generalization bias reflects poor generalization beyond the defense data distribution, with defended models suffering test-time accuracy drops of up to 40%. These findings suggest that current prompt-injection defenses frequently respond to attack-like surface patterns rather than the underlying intent. We introduce controlled diagnostic datasets and a systematic evaluation across two base models and multiple defense pipelines, highlighting limitations of supervised fine-tuning for reliable LLM security.

cs.CL

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表