图机：将边机制作为归纳偏置的探索
Graph Machine: Exploring Edge Mechanisms as an Inductive Bias
AI总结：
该研究提出Graph Machine架构，含边增强注意力与以边为中心的引用两种边机制，在数独任务上优于Transformer基线，证明显式边机制是有前景的架构设计。
AI中文摘要：
Transformer提供了基于全局内容匹配的强大架构，但推理问题可能受益于对潜在关系迭代遍历的更强归纳偏置。我们提出Graph Machine（图机）架构，包含两种显式的基于边的机制：边增强注意力，其中边调制节点间的注意力；以及以边为中心的引用，其中节点交换地址以更新自身的边。从概念上讲，这使模型能跨层动态且可微地构建和修正关系图。我们在受控设置下使用数独研究该归纳偏置，发现Graph Machine优于Transformer基线模型；消融研究和机制分析表明，性能提升源于边机制。令人惊讶的是，该模型为发现了一种紧凑的基于边的数独几何构造。我们的结果支持显式边机制是一种有前景的架构设计，值得更广泛的评估。
英文摘要：
Transformers provide a powerful architecture for global content-based matching, but reasoning problems may benefit from a stronger inductive bias toward iterative traversal of latent relations. We introduce Graph Machine, an architecture with two explicit edge-based mechanisms: Edge-augmented attention, in which edges modulate attention between nodes, and edge-centric referral, in which nodes exchange addresses to update their edges. Conceptually, this enables the model to dynamically and differentiably construct and revise relational graphs across layers. We study this inductive bias using Sudoku under controlled settings and find that Graph Machine outperforms Transformer baselines, with ablation studies and mechanistic analysis attributing the gains to the edge mechanisms. Surprisingly, we found that the model discovers a compact edge-based construction for Sudoku geometry. Our results support explicit edge mechanisms as a promising architectural design, motivating broader evaluation.
