将生成式验证建立在3D内容的基础上
Grounding Generative Verification in 3D Content
AI总结:
本文针对3D内容生成的正确性验证挑战,推出基准3DVerifierBench,构建含14126个样本的训练数据流程,验证反馈可提升3D生成的事实成功率,为3D系统的可靠验证奠定基础。
摘要:
多模态大语言模型(MLLMs)与3D生成领域的最新进展拓展了3D理解与合成能力,但除视觉保真度外,基于3D证据的任务特定正确性仍是重大挑战。本文认为,构建可靠的3D系统不仅需要更强的生成与推理能力,还需能识别未满足需求并解释违规情况的验证反馈,使系统可反思并优化输出;此类反馈需整合多视图观测结果,区分真实不一致与投影依赖外观。本文贡献有三:其一,推出涵盖10项任务的基准3DVerifierBench,评估决策、弃权(不执行)解释及对规格与候选对象受控变化的响应;人类研究显示,人类与测试的最强模型GPT-6 Astra存在21.6个百分点的差距,而感知解释的分析暴露了弃权可靠性的缺陷。其二,分析细粒度事实类别的验证情况,揭示数值与目标绑定错误;开发可扩展流程,结合多视图事实提取、受控3D编辑及原生标注,生成含矛盾解释与关联反事实的14126个训练样本。其三,研究验证如何通过候选选择与递归优化支持3D生成:在人工标注的16个候选池中,最优选择将完整事实成功率从26.7%提升至90.0%;在另一项8输入研究中,人工引导的递归优化满足所有输入的7项要求,而并行Oracle@8仅满足3项。综上,本文的基准、分析与数据流程为模型通过可靠验证反馈反思3D内容奠定了基础。
英文摘要:
Recent advances in multimodal large language models (MLLMs) and 3D generation have expanded 3D understanding and synthesis. However, beyond visual fidelity, task-specific correctness grounded in 3D evidence remains a major challenge. We argue that building reliable 3D systems requires not only stronger generation and reasoning, but also verification feedback that identifies unmet requirements and explains their violations, allowing systems to reflect on and refine their outputs. Such feedback must integrate multi-view observations and distinguish genuine inconsistencies from projection-dependent appearances. We make three contributions. First, we introduce 3DVerifierBench, a benchmark spanning 10 tasks that evaluates decisions, rejection explanations, and responses to controlled changes in specifications and candidates. A human study reveals a 21.6 percentage-point gap between humans and the strongest tested model, GPT-6 Astra, while explanation-aware analysis exposes weaknesses in rejection reliability. Second, we analyze verification across fine-grained factual categories, exposing errors in values and target binding. We develop a scalable pipeline combining multi-view fact extraction, controlled 3D editing, and native annotations to produce 14,126 training examples with contradiction explanations and linked counterfactuals. Third, we examine how verification could support 3D generation through candidate selection and recursive refinement. In human-labeled sixteen-candidate pools, oracle selection improves complete factual success from 26.7% to 90.0%. In a separate eight-input study, human-guided recursive refinement satisfies all seven requirements for every input, compared with three under parallel Oracle@8. Together, our benchmark, analysis, and data pipeline establish a foundation for models to reflect on 3D content through reliable verification feedback.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。