CellARC:利用元胞自动机测量智能
CellARC: Measuring Intelligence with Cellular Automata
AI总结:
该研究提出基于一维元胞自动机的抽象推理基准CellARC,通过多维度可控任务空间与大规模数据集,评估多类模型,发现小型Transformer优于递归模型,突显神经符号互补性。
摘要:
我们引入CellARC,这是一个基于多色一维元胞自动机(CA)构建的抽象与推理合成基准。每个任务包含5个支持对和1个查询,序列化后共256个标记,可在小型模型上实现快速迭代,同时通过字母表大小k、半径r、规则家族、兰顿λ值、查询覆盖率和细胞熵等显式控制旋钮,提供可控的任务空间。我们发布95k训练任务及两个1k测试集(插值/外推),并评估符号、递归、卷积、Transformer、递归和LLM基线模型。CellARC将泛化能力与拟人先验解耦,支持无限难度可控采样,能在预算有限的情况下可重复研究模型推断新规则的速度。我们最强的小型模型基线(10M参数的 vanilla Transformer)优于最新递归模型(TRM、HRM),在插值/外推测试集上达到58.0%/32.4%的逐标记准确率;大型闭源模型(GPT-5 High)在100个测试任务子集上达到62.3%/48.1%。在每个任务中选择Transformer与最佳符号基线的集成模型达到65.4%/35.5%的准确率,突显神经符号互补性。排行榜:https://cellarc.mireklzicar.com
英文摘要:
We introduce CellARC, a synthetic benchmark for abstraction and reasoning built from multicolor 1D cellular automata (CA). Each episode has five support pairs and one query serialized in 256 tokens, enabling rapid iteration with small models while exposing a controllable task space with explicit knobs for alphabet size k, radius r, rule family, Langton's lambda, query coverage, and cell entropy. We release 95k training episodes plus two 1k test splits (interpolation/extrapolation) and evaluate symbolic, recurrent, convolutional, transformer, recursive, and LLM baselines. CellARC decouples generalization from anthropomorphic priors, supports unlimited difficulty-controlled sampling, and enables reproducible studies of how quickly models infer new rules under tight budgets. Our strongest small-model baseline (a 10M-parameter vanilla transformer) outperforms recent recursive models (TRM, HRM), reaching 58.0%/32.4% per-token accuracy on the interpolation/extrapolation splits, while a large closed model (GPT-5 High) attains 62.3%/48.1% on subsets of 100 test tasks. An ensemble that chooses per episode between the Transformer and the best symbolic baseline reaches 65.4%/35.5%, highlighting neuro-symbolic complementarity. Leaderboard: https://cellarc.mireklzicar.com
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。