VACoT：重新思考视觉数据增强与VLMs
VACoT: Rethinking Visual Data Augmentation with VLMs
AI总结：
VACoT通过在推理过程中动态调用图像增强，提升VLMs在挑战性和分布外输入中的鲁棒性，特别是在OCR对抗场景中，通过高效的智能强化学习减少训练开销并增强感知任务的推理效率。
摘要：
尽管视觉数据增强仍然是训练鲁棒视觉模型的基石，但在视觉语言模型（VLMs）中，它却受到了有限的关注，因为VLMs主要依赖大规模真实数据采集或合成多样性。因此，它们可能在基本感知任务上遇到困难，而传统模型能够可靠地处理这些任务。鉴于预训练和微调VLMs的高昂成本，继续在增强数据上进行训练带来的收益有限且逐渐减少。在本文中，我们提出了视觉增强链式思维（VACoT），这是一种在模型推理过程中动态调用图像增强的框架。通过引入事后转换，如去噪，VACoT在具有挑战性和分布外输入的情况下显著提高了鲁棒性，特别是在OCR相关的对抗场景中。与以往局限于局部裁剪的方法不同，VACoT整合了一组结构化的通用视觉增强，从而扩展了查询图像的视角，同时通过高效的智能强化学习减少了训练复杂性和计算开销。我们提出了一种条件奖励方案，鼓励必要的增强并惩罚冗长的响应，确保在感知任务中具有简洁而有效的推理。我们通过在13个感知基准测试中进行广泛的实验来展示VACoT的优势，并进一步引入AdvOCR来突出事后视觉增强在对抗场景中的泛化优势。
英文摘要：
While visual data augmentation remains a cornerstone for training robust vision models, it has received limited attention in visual language models (VLMs), which predominantly rely on large-scale real data acquisition or synthetic diversity. Consequently, they may struggle with basic perception tasks that conventional models handle reliably. Given the substantial cost of pre-training and fine-tuning VLMs, continue training on augmented data yields limited and diminishing returns. In this paper, we present Visual Augmentation Chain-of-Thought (VACoT), a framework that dynamically invokes image augmentations during model inference. By incorporating post-hoc transformations such as denoising, VACoT substantially improves robustness on challenging and out-of-distribution inputs, especially in OCR-related adversarial scenarios. Distinct from prior approaches limited to local cropping, VACoT integrates a structured collection of general visual augmentations, broadening the query image views while reducing training complexity and computational overhead with efficient agentic reinforcement learning. We propose a conditional reward scheme that encourages necessary augmentation while penalizing verbose responses, ensuring concise and effective reasoning in perception tasks. We demonstrate the superiority of VACoT with extensive experiments on 13 perception benchmarks and further introduce AdvOCR to highlight the generalization benefits of post-hoc visual augmentations in adversarial scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
