跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2608.15341 · v2

TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models

首次提交 2026-08-16 01:46
版本更新 2026-11-18 13:03
雷达观测 2026-11-19 19:08
arXiv ↗PDF ↗

摘要

Text-to-image diffusion models can be misused to generate harmful content through adversarial or paraphrased prompts that bypass built-in safety mechanisms. Existing concept erasure methods often suffer from limited robustness against adversarial prompts, degradation of benign generation quality, or reliance on inference-time interventions that introduce persistent computational overhead. To address these limitations, we formulate concept erasure as a domain alignment problem in the text representation space. We propose a lightweight Text Encoder Alignment framework (TEA) that fine-tunes only the text encoder while keeping the generative backbone fully frozen. Given concept-anchor prompt pairs, our method trains a discriminator to distinguish token-level representations of concept-bearing prompts from those of safe anchor prompts, while updating the text encoder to make these representations indistinguishable. TEA introduces zero inference-time overhead and trains in seconds with only a few fine-tuning steps, making it practical for large-scale deployment. Despite this efficiency, TEA achieves state-of-the-art erasure robustness against black-box and white-box adversarial attacks on Stable Diffusion v1.4, while preserving generation quality on benign prompts. Furthermore, TEA is model-agnostic and achieves the lowest attack success rate on Stable Diffusion v3.5, extending concept erasure to Rectified Flow Transformer architectures with T5 conditioning, which remain largely unexplored by prior methods. Code is available at https://github.com/alirezafarashah/TEA

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-18 13:03 · 最新观测版本

TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models

Text-to-image diffusion models can be misused to generate harmful content through adversarial or paraphrased prompts that bypass built-in safety mechanisms. Existing concept erasure methods often suffer from limited robustness against adversarial prompts, degradation of benign generation quality, or reliance on inference-time interventions that introduce persistent computational overhead. To address these limitations, we formulate concept erasure as a domain alignment problem in the text representation space. We propose a lightweight Text Encoder Alignment framework (TEA) that fine-tunes only the text encoder while keeping the generative backbone fully frozen. Given concept-anchor prompt pairs, our method trains a discriminator to distinguish token-level representations of concept-bearing prompts from those of safe anchor prompts, while updating the text encoder to make these representations indistinguishable. TEA introduces zero inference-time overhead and trains in seconds with only a few fine-tuning steps, making it practical for large-scale deployment. Despite this efficiency, TEA achieves state-of-the-art erasure robustness against black-box and white-box adversarial attacks on Stable Diffusion v1.4, while preserving generation quality on benign prompts. Furthermore, TEA is model-agnostic and achieves the lowest attack success rate on Stable Diffusion v3.5, extending concept erasure to Rectified Flow Transformer architectures with T5 conditioning, which remain largely unexplored by prior methods. Code is available at https://github.com/alirezafarashah/TEA

查看此版本解读 →查看此版本原文 ↗