发表机构
Technical University of Munich; Munich Center for Machine Learning; Massachusetts Institute of Technology（慕尼黑工业大学; 慕尼黑机器学习中心; 麻省理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究普通Transformer通过有限数据增强学习点云对称性的能力，识别出从非保角到保角再到基础子群的可学习性递增顺序，并揭示诱导不变性的可解释机制，为等变性学习提供基础。
AI 中文摘要
使用有限数据增强训练基于Transformer的架构已成为几何机器学习中日益流行的方法。尽管其经验成功，Transformer架构、对不同对称性的不变性以及增强预算之间的相互作用仍未得到充分探索。在本文中，我们研究了普通Transformer通过有限数据增强学习点云数据集上各种对称性的能力。我们确定了以下对称群中可学习性递增的顺序：（i）非保角对称性，（ii）保角对称性，以及（iii）基础保角子群，如平移、旋转和缩放。对于基础保角群，我们进一步研究了Transformer的外推行为，并对训练模型进行了结构分析，从而识别出诱导不变性的可解释机制。最后，我们将分析扩展到等变函数，并表明所检测到的近似不变性机制也可以为学习到的等变性提供关键构建块。我们的项目页面可在以下网址获取：此https URL。
英文摘要
Training Transformer-based architectures with finite data augmentation has become an increasingly popular approach in geometric machine learning. Despite its empirical success, the interplay between the Transformer architecture, invariance to different symmetries, and augmentation budgets remains underexplored. In this paper, we study the ability of a vanilla Transformer to learn various symmetries through finite data augmentation for point cloud datasets. We identify an ordering of increasing learnability across the following symmetry groups: (i) non-angle-preserving symmetries, (ii) angle-preserving symmetries, and (iii) base angle-preserving subgroups, such as translation, rotation, and scale. For the base angle-preserving groups, we further investigate the Transformer's extrapolation behavior and conduct a structural analysis of the trained models, allowing us to identify interpretable mechanisms that induce invariance. Finally, we extend our analysis to equivariant functions and show that the detected mechanisms for approximate invariance can also provide a key building block for learned equivariance. Our project page is available at https://transformers-learn-symmetries.github.io/
CommentsAccepted at NeurIPS 2026
