TARS：最小最大令牌自适应偏好策略用于减少多模态大语言模型中的幻觉
TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction in MLLMs
AI总结：
TARS通过最小最大优化问题重构直接偏好优化，通过对抗性令牌扰动减少多模态大语言模型中的幻觉，通过频域对齐损失提升隐藏表征，优于标准DPO并超越数据增强方法。
摘要：
多模态大语言模型（MLLMs）容易产生看似合理但视觉上不连贯的输出，部分原因是直接偏好优化（DPO）在静态偏好监督下容易过拟合表面语言线索。我们提出TARS，一种令牌自适应偏好策略，将DPO重新表述为一个原则性的最小最大优化问题。内部最大化选择性扰动视觉无关令牌以诱导最坏情况分布偏移，而外部最小化则强制对齐因果视觉信号而非表面模式。一种新的频域对齐损失进一步通过快速傅里叶变换（FFT）在频域正则化隐藏表征，保留全局语义结构而不依赖刚性令牌级对应。我们评估了TARS在多个幻觉基准上的表现。仅使用4.8k偏好样本而不依赖专家反馈，TARS将幻觉率从26.4%降至13.2%，认知分数从2.5降至0.4，显著优于标准DPO。值得注意的是，TARS超越了基于5倍LLM的数据增强方法（Hal-Rate：16.0% vs. 13.2%），证明通过对抗性令牌扰动重塑优化景观比扩大训练数据更有效。TARS进一步在关键指标上缩小了与GPT-4o的差距。
英文摘要：
Multimodal large language models (MLLMs) are prone to hallucinations, generating plausible but visually ungrounded outputs, partly because direct preference optimization (DPO) overfits to superficial linguistic cues under static preference supervision. We propose TARS, a token-adaptive preference strategy that reformulates DPO as a principled min-max optimization problem. The inner maximization selectively perturbs visual-agnostic tokens to induce worst-case distributional shifts, while the outer minimization enforces alignment with causal visual signals rather than surface-level patterns. A novel spectral alignment loss further regularizes hidden representations in the frequency domain via the Fast Fourier Transform (FFT), preserving global semantic structure without rigid token-level correspondence. We evaluate TARS across multiple hallucination benchmarks. Using only 4.8k preference samples without expert feedback, TARS reduces hallucination rates from 26.4\% to 13.2\% and cognition scores from 2.5 to 0.4, outperforming standard DPO by a large margin. Notably, TARS surpasses $5\times$ LLM-based data augmentation trained on 28.8k samples (Hal-Rate: 16.0\% vs.\ 13.2\%), demonstrating that reshaping the optimization landscape via adversarial token perturbation is fundamentally more effective than scaling training data. TARS further narrows the gap with GPT-4o on key metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
