arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.16441cs.CVcs.AIcs.CL
收藏

VisExMEM:基于视觉证据的可解释多模态评估指标

VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric

University of Technology Nuremberg(纽伦堡工业大学)

Subhadeep Roy, Christoph Leiter, Steffen Eger

AI总结:

VisExMEM是基于视觉证据的无参考可解释多模态评估指标,经多任务SFT等训练后,在8个基准中表现优于VQAScore,推出AGITA-Bench数据集,分解式提示可提升评估性能,其解释接近GPT-5.6 Sol。

摘要:

当前图像-文本评估通常依赖全局分数,难以明确图像支持哪些语义细节。可解释指标虽能提供依据,但支撑这些依据的视觉证据仍难以验证。我们提出VisExMEM,这是一种无参考评估器,它将字幕分解为原子主张、定位视觉证据、验证每个主张,并将主张级支持度平均为全局分数,解释采用相同的证据和决策。我们在10万个样本上对基于Qwen3.5的4B和9B模型进行多任务SFT、反事实优化和证据感知GDPO训练。在8个基准测试中,VisExMEM-9B在我们的对比中达到开源评估器中最高的描述性综合得分,超过VQAScore 11.7分,同时与GPT-5.6 Sol具有竞争力。我们还推出AGITA-Bench,包含600个人类标注的图像-文本对和8700个原子主张,用于全局和主张级评估。对于未微调的Qwen3.5和GPT-5.6 Sol对照,分解式提示相比整体式提示使综合得分提高3.6至5.1分。人类评估显示,VisExMEM的解释优于现有可解释指标,接近GPT-5.6 Sol。

英文摘要:

Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explainable metrics provide rationales, but the visual evidence supporting those rationales can still be difficult to verify. We introduce VisExMEM, a reference-free evaluator that decomposes captions into atomic claims, localizes visual evidence, verifies each claim, and averages claim-level support into a global score. Explanations use the same evidence and decisions. We train Qwen3.5-based 4B and 9B models on 100K samples with multi-task SFT, counterfactual refinement, and evidence-aware GDPO. Across eight benchmarks, VisExMEM-9B achieves the highest descriptive composite among open-source evaluators in our comparison, exceeding VQAScore by 11.7 points while remaining competitive with GPT-5.6 Sol. We also introduce AGITA-Bench, containing 600 human-annotated image-text pairs and 8.7K atomic claims for global and claim-level evaluation. For untuned Qwen3.5 and GPT-5.6 Sol controls, decomposed prompting raises the same composite by 3.6-5.1 points relative to holistic prompting. Human evaluation shows that VisExMEM explanations outperform prior explainable metrics and approach GPT-5.6 Sol.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑