可废止视觉蕴含：基准、评估器与奖励驱动优化
Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimization
AI总结：
针对视觉蕴含未探索的可废止性问题，提出DVE任务、推理感知评估器及奖励驱动优化方法，以捕捉更新对图像-文本蕴含关系的修改，提升多应用中模型的准确性与可靠性。
摘要：
我们提出一项名为可废止视觉蕴含（DVE）的新任务，目标是允许基于额外更新修改图像前提与文本假设之间的蕴含关系。尽管该概念在自然语言推理中已确立，但在视觉蕴含领域仍未被探索。从整体看，DVE使模型能优化初始解释，提升检测图像误导信息、增强视觉问答、改进自主系统决策流程等多应用的准确性与可靠性。现有指标无法充分捕捉更新带来的蕴含关系变化，为此我们提出新型推理感知评估器，通过成对对比学习与分类信息学习捕捉更新引发的蕴含强度变化；还引入奖励驱动更新优化方法，进一步提升多模态模型生成的更新质量。实验结果证明了所提评估器与优化方法的有效性。
英文摘要：
We introduce a new task called Defeasible Visual Entailment (DVE), where the goal is to allow the modification of the entailment relationship between an image premise and a text hypothesis based on an additional update. While this concept is well-established in Natural Language Inference, it remains unexplored in visual entailment. At a high level, DVE enables models to refine their initial interpretations, leading to improved accuracy and reliability in various applications such as detecting misleading information in images, enhancing visual question answering, and refining decision-making processes in autonomous systems. Existing metrics do not adequately capture the change in the entailment relationship brought by updates. To address this, we propose a novel inference-aware evaluator designed to capture changes in entailment strength induced by updates, using pairwise contrastive learning and categorical information learning. Additionally, we introduce a reward-driven update optimization method to further enhance the quality of updates generated by multimodal models. Experimental results demonstrate the effectiveness of our proposed evaluator and optimization method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
