DiG-bench：游戏中的发现
DiG-bench: Discovery in Games
浏览论文内容
中文总结 AI 辅助
本研究发布DiG-bench基准，含70个需智能体通过交互实验发现规则的游戏，设7个难度级，部分公开部分私有，用于评估智能体在目标未知环境中发现新知识的能力。
中文摘要 AI 辅助
发现——即构建新的概括——是科学过程的核心部分。尽管其重要性不言而喻，但当前AI基准领域存在一个缺口：很少有基准能直接探测智能体在目标未知的受控环境中通过实验发现新知识的能力。为解决这一缺口，我们发布了新基准DiG-bench（Discovery in Games）。DiG-bench包含70个独立游戏，每个游戏编码为短字符串，具有独特的变换规则，需通过交互与实验发现。游戏关卡提供一系列挑战，用于测试规则是否被发现，且每个关卡的胜利条件也未知。我们为AI智能体提供7个难度级别的游戏：最低级可被多个模型常规解决，最高级则挑战智能体框架中的最优模型。全部70个游戏至少有一人首次尝试即可解决，其中21个游戏公开，其余保留为私有用于安全评估。
英文摘要
Discovery---formulating novel generalizations---is a central part of the scientific process. Despite its importance, there is a gap in the current AI benchmark landscape, with few benchmarks directly probing the capacity for discovering new knowledge with experimentation in controlled environments where the objective is unknown. To address this gap, we release a new benchmark: DiG-bench (Discovery in Games). DiG-bench consists of a set of 70 independent games. Each game is encoded as a short string and has unique transformation rules that must be discovered through interaction and experimentation. The levels of the game present a series of challenges to test whether the rules have been discovered, where the win conditions for each level are also unknown. We provide games at seven tiers of difficulty for AI agents. The lowest tier is routinely solvable by multiple models, while the highest tier challenges the best models in agentic harnesses. All 70 games were solved by at least one human on first attempt. A subset of 21 games is released publicly, and the remainder is held private for secure evaluation.
