arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10195cs.LGcs.AIhep-th
收藏

Versor:一种几何序列架构

Versor: A Geometric Sequence Architecture

Trương Minh Huy Independent Researcher(Trương Minh Huy 独立研究者);Instituto de Matemática, Estatística e Computação Científica University of Campinas (UNICAMP), Brazil(巴西坎皮纳斯大学数学、统计与科学计算研究所)

Truong Minh Huy, Edward Hirst

AI总结:

Versor通过共形几何代数实现结构泛化和高效任务处理,优于Transformer等基线模型。

摘要:

一种新颖的序列架构被引入,称为Versor,它使用共形几何代数(CGA)取代传统的线性操作,以实现结构泛化并在各种任务上获得显著的性能提升,同时提供改进的可解释性和效率。通过将状态嵌入到$Cl_{4,1}$流形中,并通过几何变换(旋子)来演变它们,Versor原生地表示$SE(3)$等价关系,而无需显式结构编码。Versor在混沌N体动力学、拓扑推理和标准多模态基准(CIFAR-10,WikiText-103)上进行了验证,始终优于Transformer、图网络和几何基线(GATr,EGNN)。关键结果包括:参数数量减少数个数量级(200倍优于Transformer);可解释的注意力分解为接近性和方向性成分;零样本规模泛化(0.993 vs. 0.070 MCC for ViT);以及具有递归旋子累加器(RRA)的$O(L)$线性时间复杂度动态系统,和具有几何积注意力(GPA)机制的$O(L^{2})$全局关系建模,允许根据所需规模进行任务特定的架构剪枝或混合。在非分布测试中,Versor保持稳定预测,而Transformer则灾难性失败。自定义克莱因核通过位掩码收缩和专用矩阵同构核实现累计超过100倍的速度提升,将每步延迟降低到1.05毫秒,并优于高度优化的Transformer基线。

英文摘要:

A novel sequence architecture is introduced, Versor, which uses Conformal Geometric Algebra (CGA) in place of traditional linear operations to achieve structural generalization and significant performance improvements on a variety of tasks, while offering improved interpretability and efficiency. By embedding states in the $Cl_{4,1}$ manifold and evolving them via geometric transformations (rotors), Versor natively represents $SE(3)$-equivariant relationships without requiring explicit structural encoding. Versor is validated on chaotic N-body dynamics, topological reasoning, and standard multimodal benchmarks (CIFAR-10, WikiText-103), consistently outperforming Transformers, Graph Networks, and geometric baselines (GATr, EGNN). Key results include: orders-of-magnitude fewer parameters ($200\times$ vs. Transformers); interpretable attention decomposing into proximity and orientational components; zero-shot scale generalization (0.993 vs. 0.070 MCC for ViT); and featuring a Recursive Rotor Accumulator (RRA) for $O(L)$ linear temporal complexity in dynamical systems, and a Geometric Product Attention (GPA) mechanism for $O(L^{2})$ global relational modeling, allowing for task-specific architectural pruning or hybridization depending on the required scale. In out-of-distribution tests, Versor maintains stable predictions while Transformers fail catastrophically. Custom Clifford kernels achieve a cumulative over $100\times$ speedup via bit-masked contraction and specialized Matrix Isomorphism kernels, reducing per-step latency to 1.05 ms and outperforming highly-optimized Transformer baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑