CreativeBench: 通过自我进化挑战基准测试和增强机器创造力
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
AI总结：
提出CreativeBench基准，基于认知框架通过代码生成评估机器创造力，包含组合与探索两个子集，利用逆向工程和自我博弈自动生成挑战，并通过质量与新颖性乘积的指标区分创造与幻觉。
摘要：
高质量预训练数据的饱和已将研究焦点转向能够持续生成新颖产物的进化系统，从而促成了AlphaEvolve的成功。然而，此类系统的进展因缺乏严格、量化的评估而受阻。为应对这一挑战，我们引入了CreativeBench，这是一个基于经典认知框架、用于评估代码生成中机器创造力的基准。该基准包含两个子集——CreativeBench-Combo和CreativeBench-Explore，通过利用逆向工程和自我博弈的自动化流程，分别针对组合创造力和探索创造力。通过利用可执行代码，CreativeBench通过一个统一指标（定义为质量与新颖性的乘积）客观地区分创造力与幻觉。我们对最先进模型的分析揭示了不同的行为：(1) 规模扩展显著提升了组合创造力，但对探索的收益递减；(2) 更大的模型表现出“规模收敛”，即变得更正确但更少发散；(3) 推理能力主要有利于受约束的探索而非组合。最后，我们提出了EvoRePE，一种即插即用的推理时引导策略，通过内化进化搜索模式来持续增强机器创造力。
英文摘要：
The saturation of high-quality pre-training data has shifted research focus toward evolutionary systems capable of continuously generating novel artifacts, leading to the success of AlphaEvolve. However, the progress of such systems is hindered by the lack of rigorous, quantitative evaluation. To tackle this challenge, we introduce CreativeBench, a benchmark for evaluating machine creativity in code generation, grounded in a classical cognitive framework. Comprising two subsets -- CreativeBench-Combo and CreativeBench-Explore -- the benchmark targets combinatorial and exploratory creativity through an automated pipeline utilizing reverse engineering and self-play. By leveraging executable code, CreativeBench objectively distinguishes creativity from hallucination via a unified metric defined as the product of quality and novelty. Our analysis of state-of-the-art models reveals distinct behaviors: (1) scaling significantly improves combinatorial creativity but yields diminishing returns for exploration; (2) larger models exhibit ``convergence-by-scaling,'' becoming more correct but less divergent; and (3) reasoning capabilities primarily benefit constrained exploration rather than combination. Finally, we propose EvoRePE, a plug-and-play inference-time steering strategy that internalizes evolutionary search patterns to consistently enhance machine creativity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
