Trương Minh Huy Independent Researcher(Trương Minh Huy 独立研究者);Instituto de Matemática, Estatística e Computação Científica University of Campinas (UNICAMP), Brazil(巴西坎皮纳斯大学数学、统计与科学计算研究所)
一种新颖的序列架构被引入,称为Versor,它使用共形几何代数(CGA)取代传统的线性操作,以实现结构泛化并在各种任务上获得显著的性能提升,同时提供改进的可解释性和效率。通过将状态嵌入到$Cl_{4,1}$流形中,并通过几何变换(旋子)来演变它们,Versor原生地表示$SE(3)$等价关系,而无需显式结构编码。Versor在混沌N体动力学、拓扑推理和标准多模态基准(CIFAR-10,WikiText-103)上进行了验证,始终优于Transformer、图网络和几何基线(GATr,EGNN)。关键结果包括:参数数量减少数个数量级(200倍优于Transformer);可解释的注意力分解为接近性和方向性成分;零样本规模泛化(0.993 vs. 0.070 MCC for ViT);以及具有递归旋子累加器(RRA)的$O(L)$线性时间复杂度动态系统,和具有几何积注意力(GPA)机制的$O(L^{2})$全局关系建模,允许根据所需规模进行任务特定的架构剪枝或混合。在非分布测试中,Versor保持稳定预测,而Transformer则灾难性失败。自定义克莱因核通过位掩码收缩和专用矩阵同构核实现累计超过100倍的速度提升,将每步延迟降低到1.05毫秒,并优于高度优化的Transformer基线。
英文摘要:
A novel sequence architecture is introduced, Versor, which uses Conformal Geometric Algebra (CGA) in place of traditional linear operations to achieve structural generalization and significant performance improvements on a variety of tasks, while offering improved interpretability and efficiency. By embedding states in the $Cl_{4,1}$ manifold and evolving them via geometric transformations (rotors), Versor natively represents $SE(3)$-equivariant relationships without requiring explicit structural encoding. Versor is validated on chaotic N-body dynamics, topological reasoning, and standard multimodal benchmarks (CIFAR-10, WikiText-103), consistently outperforming Transformers, Graph Networks, and geometric baselines (GATr, EGNN). Key results include: orders-of-magnitude fewer parameters ($200\times$ vs. Transformers); interpretable attention decomposing into proximity and orientational components; zero-shot scale generalization (0.993 vs. 0.070 MCC for ViT); and featuring a Recursive Rotor Accumulator (RRA) for $O(L)$ linear temporal complexity in dynamical systems, and a Geometric Product Attention (GPA) mechanism for $O(L^{2})$ global relational modeling, allowing for task-specific architectural pruning or hybridization depending on the required scale. In out-of-distribution tests, Versor maintains stable predictions while Transformers fail catastrophically. Custom Clifford kernels achieve a cumulative over $100\times$ speedup via bit-masked contraction and specialized Matrix Isomorphism kernels, reducing per-step latency to 1.05 ms and outperforming highly-optimized Transformer baselines.
GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
HSE University(莫斯科高等经济大学);Skolkovo Institute of Science(斯克尔科沃科学研究院);Institute for Information Transmission Problems of the Russian Academy of Sciences(俄罗斯科学院信息传输问题研究所)
Convolutional neural network architecture for geometric matching
École normale supérieure(巴黎高等师范学院);CNRS(法国国家科学研究中心);PSL Research University(巴黎文理研究大学);Czech Technical University in Prague(布拉格捷克理工大学);DeepMind;INRIA(法国国家信息与自动化研究所)
GeoMFormer: A General Architecture for Geometric Molecular Representation Learning
School of EECS, Peking University(北京大学工程科学学院);National Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);Microsoft Research AI4Science(微软研究院AI4Science部门);Center for Machine Learning Research, Peking University(北京大学机器学习研究中心)
Comparison of Optimised Geometric Deep Learning Architectures, over Varying Toxicological Assay Data Environments
King’s College London(伦敦国王学院);Istituto di Ricerche Farmacologiche Mario Negri IRCCS(马里奥·内格里药理研究所);European Food Safety Authority (EFSA)(欧洲食品安全局);Food Standards Agency(食品标准局)
GRIP: Algorithm-Agnostic Machine Unlearning for Mixture-of-Experts via Geometric Router Constraints
School of Computer Science(计算机科学学院);Georgia Institute of Technology(佐治亚理工学院);Department of Electrical Engineering(电气工程系);Tsinghua University(清华大学);School of Electrical and Computer Engineering(电气与计算机工程学院)
Understanding Graph Neural Networks with Generalized Geometric Scattering Transforms
University of California, Los Angeles(加州大学洛杉矶分校);Université de Montréal(蒙特利尔大学);Mila – Quebec AI Institute(米拉-魁北克人工智能研究所);Columbia University(哥伦比亚大学);Michigan State University(密歇根州立大学)
How the Optimizer Shapes Learned Solutions in Equivariant Neural Networks
University of Stuttgart, Germany(斯图加特大学);International Max Planck Research School for Intelligent Systems, Germany(国际马克斯·普朗克智能系统研究学校);Tudor Vianu High School of Computer Science, Romania(托尔德·维安乌计算机科学高中)
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
Imperial College London(伦敦帝国学院);USI IDSIA(瑞士意大利语区人工智能研究所);Twitter(推特);New York University(纽约大学);Qualcomm AI Research(高通AI研究院);Qualcomm Technologies, Inc.(高通技术公司);DeepMind(深度思维)
Equivariant geometric convolutions for emulation of dynamical systems
Johns Hopkins University(约翰斯·霍普金斯大学);New York University(纽约大学);Max-Planck-Institut für Astronomie(马克斯·普朗克天文研究所);Flatiron Institute(弗拉蒂伦研究所);Universidad Autónoma de Madrid(马德里自治大学);Mathematical Institute for Data Science, Johns Hopkins University(约翰斯·霍普金斯大学数据科学数学研究所)
Geometric Neural Network based on Phase Space for BCI-EEG decoding
Université Côte d’Azur(蔚蓝海岸大学);Inria(法国国家信息与自动化研究所);Université Paris-Saclay(巴黎萨克雷大学);Universidade Federal do ABC(ABC联邦大学);Paris Brain Institute(巴黎脑研究所)
Mathematical Insights into Protein Architecture: Persistent Homology and Machine Learning Applied to the Flagellar Motor
Department of Mathematics, Faculty of Sciences, Ibn Tofail University(数学系,科学学院,伊本·图法易大学);Department of Mathematics, Faculty of Sciences and Techniques, Abdelmalek Essaâdi University(数学系,科学与技术学院,阿卜杜勒马利克·艾萨迪大学);Department of Computer and Information Science, Fordham University(计算机与信息科学系,福尔敦大学)