CresOWLve：基于现实世界知识的创造性问题解决基准测试
CresOWLve: Benchmarking Creative Problem-Solving Over Real-World Knowledge
AI总结：
本文提出CresOWLve基准测试，评估创造性问题解决能力，通过现实世界知识中的谜题，发现模型在事实性问题与创造性问题上的表现差异显著。
摘要：
创造性问题解决需要结合多种认知能力，包括逻辑推理、侧向思维、类比和常识知识，以发现连接看似无关信息的洞察。然而，现有大型语言模型（LLM）基准主要评估单一过程组件，且许多创造力导向的基准依赖人工构造的谜题或人为场景，无法反映现实中的创造性问题解决。为解决这一差距，我们引入CresOWLve，一个基于现实世界知识的创造性问题解决评估基准。CresOWLve中的问题要求运用多种创造性思维策略，从不同领域检索事实，并创造性地结合它们以得出解决方案。评估几种前沿非思考和思考LLM后，我们发现CresOWLve仍极具挑战性。我们的分析揭示了一致的表现差距：模型在事实性问题上表现优于创造性问题（高达-17%的下降）。尽管模型能经常检索相关知识，但它们在整合信息并得出正确答案所需的非显而易见的创造性连接上面临困难。
英文摘要：
Creative problem-solving requires combining multiple cognitive abilities, including logical reasoning, lateral thinking, analogy-making, and commonsense knowledge, to discover insights that connect seemingly unrelated pieces of information. However, most existing benchmarks for large language models (LLMs) evaluate only specific components of this process. Moreover, many creativity-oriented benchmarks rely on artificially constructed brainteasers or contrived scenarios that do not reflect how creative problem-solving occurs in real-world settings. To address this gap, we introduce CresOWLve (containing 2,061 examples across five difficulty levels), a benchmark for evaluating creative problem-solving using puzzles grounded in real-world knowledge. Problems in CresOWLve require employing multiple creative thinking strategies, retrieving facts from diverse domains, and creatively combining them to arrive at a solution. Evaluating several frontier non-thinking and thinking LLMs, we show that CresOWLve remains highly challenging. Our analysis reveals a consistent performance gap: models perform substantially better on factual questions than on creative ones (up to -17% drop). While models can often retrieve the relevant knowledge, they struggle to form the non-obvious creative connections required to integrate the knowledge and arrive at the correct answer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
