CAT：面向几何感知学习的曲率自适应Transformer
CAT: Curvature-Adaptive Transformers for Geometry-Aware Learning
AI总结：
针对Transformer隐含欧氏几何假设、难以适配混合几何结构数据的问题，提出曲率自适应Transformer CAT，通过可微分门控动态按token路由到不同几何注意力分支，在知识图谱补全基准上以低开销实现约10%的性能提升，为混合几何架构提供可扩展可解释的基础。
摘要：
Transformer在多个领域都取得了出色的性能，但其注意力机制隐含了欧几里得几何假设，这限制了它在具有非欧结构的数据上的有效性。虽然近期针对双曲空间和球面空间的扩展方法分别在层级模式和循环模式上展现出潜力，但这类方法需要预先选定单一几何结构，当数据具备混合几何特性时灵活性不足。我们提出了曲率自适应Transformer（Curvature-Adaptive Transformer，CAT）——一种新型架构，它通过轻量可微的门控机制，动态学习每个token在三个几何注意力分支间的路由。与固定几何结构的方法不同，CAT支持自适应的几何特化，可根据token的局部关系结构将其路由到对应曲率的分支。路由网络能够提供可解释的曲率偏好，而每个分支都采用针对其对应流形优化的几何专属操作。在知识图谱补全基准（FB15k-237、WN18RR）上，CAT相较于固定几何基线在MRR和Hits@10指标上实现了约10%的提升，且额外开销极低（参数仅增加5%，推理速度相当）。这些结果表明，对于复杂关系推理任务，可学习的几何自适应方法优于任何单一固定几何结构，同时也确立了CAT作为可扩展、可解释的混合几何架构基础的地位，可应用于语言、视觉和多模态领域。
英文摘要：
Transformers achieve strong performance across diverse domains but implicitly assume Euclidean geometry in their attention mechanisms, limiting their effectiveness on data with non-Euclidean structure. While recent extensions to hyperbolic and spherical spaces show promise for hierarchical and cyclical patterns, respectively, they require committing to a single geometry a priori, reducing flexibility when data exhibits mixed geometric properties. We introduce the Curvature-Adaptive Transformer (CAT), a novel architecture that dynamically learns per-token routing across three geometric attention branches through a lightweight, differentiable gating mechanism. Unlike fixed-geometry approaches, CAT enables adaptive geometric specialization, routing tokens to the appropriate curvature based on their local relational structure. The routing network provides interpretable curvature preferences while each branch employs geometry-specific operations optimized for its respective manifold. On knowledge graph completion benchmarks (FB15k-237, WN18RR), CAT achieves approximately 10% improvements in MRR and Hits@10 over fixed-geometry baselines with minimal overhead (5% parameter increase, comparable inference time). These results demonstrate that learned geometric adaptation outperforms any single fixed geometry for complex relational reasoning, establishing CAT as a scalable and interpretable foundation for mixture-of-geometry architectures across language, vision, and multimodal domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
