隐形微调：通过自动生成的CoT打破RVLMs的对齐
Stealth Fine-Tuning: Efficiently Breaking Alignment in RVLMs Using Self-Generated CoT
AI总结：
本文提出隐形微调方法，通过分段干扰和自动生成输出，有效绕过RVLMs的安全对齐防御，实现更高的ASR性能同时保持推理能力。
摘要：
增强推理的视觉-语言模型（RVLMs）依赖于安全对齐以防止有害行为，但它们暴露的推理链（CoT）痕迹引入了新的攻击面。在本工作中，我们发现RVLMs的安全对齐可以通过一种新的攻击方法“隐形微调”轻易打破。我们的方法通过分段干扰来激发有害推理痕迹，并重用自动生成的输出作为监督微调数据。为此，我们引入了一种基于回合加权的损失，以最小化分布偏移。在我们的实验中，仅使用499个样本和3小时在单个A100（QLoRA）上训练，隐形微调在ASR上优于IDEATOR 38.66%，同时保持一般推理能力，因为微调后的模型保留了原始表示分布。在AdvBench和多个通用基准测试中的实验表明，隐形微调是一种低成本且高效的绕过对齐防御的方法。免责声明：本文包含可能令人不安或冒犯的内容。
英文摘要：
Reasoning-augmented Vision-Language Models (RVLMs) rely on safety alignment to prevent harmful behavior, yet their exposed chain-of-thought (CoT) traces introduce new attack surfaces. In this work, we find that the safety alignment of RVLMs can be easily broken through a novel attack method termed \textbf{Stealth Fine-Tuning}. Our method elicits harmful reasoning traces through \textbf{segment-level interference} and reuses the self-generated outputs as supervised fine-tuning data. To facilitate this, we introduce a \textbf{turn-based weighted} loss that minimizes distribution shift. In our experiment, with only 499 samples and under 3 hours on a single A100 (QLoRA), Stealth Fine-Tuning outperforms IDEATOR by 38.66\% ASR while preserving general reasoning ability, as the tuned model retains the original representation distribution. Experiments on AdvBench and several general benchmarks demonstrate that Stealth Fine-Tuning is a low-cost and highly effective way to bypass alignment defenses. \textcolor{red}{\textbf{Disclaimer: This paper contains content that may be disturbing or offensive.}}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
