CV-Arena: 面向教学计算机视觉问题求解的开放基准与人类-AI协作偏好
CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferences
AI总结：
提出CV-Arena基准，包含12K高分辨率真实图像指令对，覆盖16种任务类型，并采用Active Elo协议结合人类与AI偏好评估21个系统，揭示指令遵循、物理推理等方面的差距，同时开发CV-Agent代理模型展示闭环推理的潜力。
摘要：
指令引导的图像编辑正成为视觉工作的通用接口，然而现有基准仍主要聚焦于狭窄的外观编辑，未能充分捕捉专业工作流程中真实图像任务的多样性。在此，我们将教学计算机视觉问题求解定义为图像编辑的更广泛形式：给定真实输入图像和自然语言指令，系统必须生成编辑后的输出，实现所要求的变换，同时满足明确的保持性、几何、物理和可用性约束。我们引入了CV-Arena，一个旨在以专业规模评估此能力的开放基准。CV-Arena包含12K高分辨率真实图像指令对，涵盖16种基于指令的视觉任务类型，通过CogRetriever构建，这是一个结合目标网络搜索、代理查询精化、验证和可追溯性的双轨检索与筛选流水线。为了在保持人类保真度的同时大规模评估模型，我们提出了Active Elo，一种人类-AI协作偏好协议，利用CV-Judge（一个逻辑门控、多维度VLM评估器）拒绝明显失败并解决高置信度比较，并将接近的高质量比较路由给专家评分者。然后通过可靠性加权的Elo更新聚合混合的人类和AI监督。我们对21个系统（包括专有、开源和代理模型）在CV-Arena上的全面评估揭示了指令遵循、物理推理、结构控制和细粒度细节保持方面的持续差距。我们进一步开发了CV-Agent，一个轻量级代理模型，结合了规划、编辑和验证，并证明了闭环推理是专业级指令遵循视觉编辑的一个有前景的方向。
英文摘要：
Instruction-guided image editing is becoming a general interface for visual work, yet existing benchmarks still focus largely on narrow appearance edits and do not fully capture the diversity of real-image tasks in professional workflows. Here, we define instructional computer vision problem solving as a broader formulation of image editing: given a real input image and a natural-language instruction, a system must produce an edited output that realizes the requested transformation while satisfying explicit preservation, geometric, physical, and usability constraints. We introduce CV-Arena, an open benchmark designed to evaluate this capability at professional scales. CV-Arena contains 12K high-resolution real-image instruction pairs spanning 16 instruction-based visual task types, constructed using CogRetriever, a dual-track retrieval-and-curation pipeline that combines targeted web search, agentic query refinement, verification, and traceability. To evaluate models at scale while preserving human fidelity, we propose Active Elo, a human-AI collaborative preference protocol that leverages CV-Judge, a logic-gated, multi-dimensional VLM evaluator, to reject clear failures and resolve high-confidence comparisons; and to route close, high-quality comparisons to expert raters. Mixed human and AI supervision is then aggregated through reliability-weighted Elo updates. Our comprehensive evaluation of 21 systems, including proprietary, open-source, and agentic models, on CV-Arena reveals persistent gaps in instruction adherence, physical reasoning, structural control, and fine-grained detail preservation. We further develop CV-Agent, a lightweight agentic model that combines planning, editing, and verification, and demonstrate that closed-loop reasoning is a promising direction for professional-grade instruction-following visual editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
