GraphARC：基于图的抽象推理综合基准
GraphARC: A Comprehensive Benchmark for Graph-Based Abstract Reasoning
AI总结：
提出GraphARC基准，将抽象推理扩展到图结构数据，通过少样本变换学习任务评估模型在局部、全局和层次图变换上的泛化能力，并揭示语言模型的理解-执行差距和规模扩展障碍。
摘要：
关系推理是智能的核心，但现有基准通常局限于网格或文本格式。我们引入了GraphARC，一个用于图结构数据抽象推理的基准。GraphARC推广了抽象与推理语料库（ARC）的少样本变换学习范式。每个任务需要从几个输入-输出对中推断变换规则，并将其应用于新的测试图，涵盖局部、全局和层次图变换。与基于网格的ARC不同，GraphARC实例可以在不同的图族和规模上大规模生成，从而能够系统评估泛化能力。我们在GraphARC上评估了最先进的语言模型，并观察到明显的局限性。模型能够回答关于图属性的问题，但往往无法解决完整的图变换任务，揭示了理解-执行差距。在更大实例上性能进一步下降，暴露了规模扩展障碍。更广泛地说，通过将节点分类、链接预测和图生成的方面结合在一个单一框架内，GraphARC为未来的图基础模型提供了一个有前景的测试平台。
英文摘要：
Relational reasoning lies at the heart of intelligence, but existing benchmarks are typically confined to formats such as grids or text. We introduce GraphARC, a benchmark for abstract reasoning on graph-structured data. GraphARC generalizes the few-shot transformation learning paradigm of the Abstraction and Reasoning Corpus (ARC). Each task requires inferring a transformation rule from a few input-output pairs and applying it to a new test graph, covering local, global, and hierarchical graph transformations. Unlike grid-based ARC, GraphARC instances can be generated at scale across diverse graph families and sizes, enabling systematic evaluation of generalization abilities. We evaluate state-of-the-art language models on GraphARC and observe clear limitations. Models can answer questions about graph properties but often fail to solve the full graph transformation task, revealing a comprehension-execution gap. Performance further degrades on larger instances, exposing scaling barriers. More broadly, by combining aspects of node classification, link prediction, and graph generation within a single framework, GraphARC provides a promising testbed for future graph foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
