arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.30561cs.AIcs.CVcs.HC
收藏

人类创造力基准

The Human Creativity Benchmark

Massachusetts Institute of Technology(麻省理工学院);Contra

Aspen Hopkins, Allison Nulty, Alexandria Minetti, Anoop Pakki, Angad Singh

AI总结:

提出人类创造力基准(HCB),通过收集领域专家的成对偏好、标量评分和定性理由,分离收敛(共享最佳实践)和发散(个体品味差异)信号,评估创意AI。

摘要:

现代AI评估框架将评估者的分歧视为需要解决的噪声。在创意领域,专业分歧反映了品味的真实差异,而非测量误差。我们认为,评估创意AI需要保留两种不同的信号:收敛,即专业人士围绕共享最佳实践达成一致;以及发散,即个体品味合理变化。我们提出了人类创造力基准(HCB),该基准通过收集来自领域专家的成对偏好、关于提示遵循性、可用性和视觉吸引力的标量评分以及定性理由,实现了这种分离。跨越五个创意领域和三个工作流阶段(构思、原型、细化)的15,000个专业判断中,我们发现收敛集中在可验证的维度上,如技术正确性和视觉层次,而发散集中在品味驱动的维度上,如美学方向和概念风险。没有模型在所有阶段都表现优异。将这些信号合并为一个单一的质量指标会丢弃最可操作的信息:模型必须在哪些方面正确,以及模型在哪些方面应保持可引导性。

英文摘要:

Modern AI evaluation frameworks treat evaluator disagreement as noise to be resolved. In creative domains, professional disagreement reflects genuine differences in taste, not measurement error. We argue that evaluating creative AI requires preserving two distinct signals: convergence, where professionals align around shared best practices, and divergence, where individual taste legitimately varies. We present the Human Creativity Benchmark (HCB), a benchmark that operationalizes this separation by collecting pairwise preferences, scalar ratings on prompt adherence, usability, and visual appeal, and qualitative rationale from domain professionals. Across 15,000 professional judgments spanning five creative domains and three workflow phases (ideation, mockup, refinement), we find that convergence concentrates on verifiable dimensions like technical correctness and visual hierarchy, while divergence concentrates on taste-driven dimensions like aesthetic direction and conceptual risk. No model excels uniformly across all phases. Collapsing these signals into a single quality metric discards the most actionable information: where models must be correct versus where they should remain steerable.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑