← 研踪 PaperTrailarXiv:2607.11070 · v2
MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignment
首次提交 2026-07-13 12:19
版本更新 2026-11-09 12:58
雷达观测 2026-11-10 13:18
摘要
Modern large language models (LLMs) operate in interactive multi-turn settings, making multi-turn jailbreaking a realistic threat model and an important setting for automated red teaming. A core challenge in learning multi-turn jailbreak attackers is credit assignment: different turns contribute differently to the final outcome, yet existing learning signals are often too coarse to identify their individual contributions. We propose decomposed credit GRPO (DC-GRPO), a unified turn-level credit assignment framework for Group Relative Policy Optimization in multi-turn jailbreak learning. DC-GRPO assigns a separate group-relative learning signal to each turn by combining immediate and future credit, avoiding the credit misassignment induced by broadcasting a single trajectory-level score across the dialogue. We instantiate this framework with static and dynamic weighting rules that differ in how the two credit sources are balanced while sharing the same turn-level structure. Across four victim LLMs and three benchmarks, DC-GRPO achieves up to 98.26% average ASR_5@3, substantially outperforming the state-of-the-art methods, including SEMA (86.58%) and TROJail (86.23%), demonstrating the effectiveness of our turn-level group-relative credit assignment. Warning: This paper contains examples of harmful content.
作者备注：34 pages. Warning: This paper contains examples of harmful content
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-09 12:58 · 最新观测版本
MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignment
Modern large language models (LLMs) operate in interactive multi-turn settings, making multi-turn jailbreaking a realistic threat model and an important setting for automated red teaming. A core challenge in learning multi-turn jailbreak attackers is credit assignment: different turns contribute differently to the final outcome, yet existing learning signals are often too coarse to identify their individual contributions. We propose decomposed credit GRPO (DC-GRPO), a unified turn-level credit assignment framework for Group Relative Policy Optimization in multi-turn jailbreak learning. DC-GRPO assigns a separate group-relative learning signal to each turn by combining immediate and future credit, avoiding the credit misassignment induced by broadcasting a single trajectory-level score across the dialogue. We instantiate this framework with static and dynamic weighting rules that differ in how the two credit sources are balanced while sharing the same turn-level structure. Across four victim LLMs and three benchmarks, DC-GRPO achieves up to 98.26% average ASR_5@3, substantially outperforming the state-of-the-art methods, including SEMA (86.58%) and TROJail (86.23%), demonstrating the effectiveness of our turn-level group-relative credit assignment. Warning: This paper contains examples of harmful content.
查看此版本解读 →查看此版本原文 ↗