动态对抗强化学习用于鲁棒多模态大语言模型
Dynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Models
AI总结：
本文提出AOT方法，通过对抗训练提升多模态大语言模型的鲁棒性和可靠性。
摘要：
尽管具有惊人的能力，多模态大语言模型（MLLMs）在面对视觉复杂场景时表现出感知脆弱性。这种弱点源于对有限训练数据集的依赖，这些数据集扩展成本高昂且限制了模型的鲁棒性。我们引入了AOT-SFT，一个大规模对抗数据集，用于促进MLLMs的鲁棒性。在此基础上，我们提出了AOT（对抗对手训练），一种自我博弈框架，通过生成自身训练数据来增强MLLMs的鲁棒性。我们的方法协调了图像编辑攻击者和防御性MLLM之间的共进化，其中攻击者生成多样且动态的图像操纵课程，迫使防御者适应和改进。大量实验表明，AOT增强了防御者的感知鲁棒性并减少了幻觉，建立了一种可扩展的训练更可靠MLLMs的范式。
英文摘要：
Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) exhibit perceptual fragility when confronted with visually complex scenes. This weakness stems from a reliance on finite training datasets, which are prohibitively expensive to scale and impose a ceiling on model robustness. We introduce \textbf{AOT-SFT}, a large-scale adversarial dataset for bootstrapping MLLM robustness. Building on this, we propose \textbf{AOT (Adversarial Opponent Training)}, a self-play framework that forges MLLM robustness by creating its own training data. Our method orchestrates a co-evolution between an image-editing Attacker and a Defender MLLM, where the Attacker generates a diverse and dynamic curriculum of image manipulations, forcing the Defender to adapt and improve. Extensive experiments demonstrate that AOT enhances the Defender's perceptual robustness and reduces hallucinations, establishing a scalable paradigm for training more reliable MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
