面向对撞机物理的轻量级基础模型：多领域适配
A Lightweight Foundation Model for Collider Physics with Multi-Domain Adaptation
浏览论文内容
中文总结 AI 辅助
该研究提出轻量级基础模型NEXUS，以300万参数的全连接自编码器为架构，通过对撞机数据预训练实现多领域适配，在下游任务中提升准确率且计算复杂度低于同规模Transformer。
中文摘要 AI 辅助
我们提出了一种轻量级基础建模方法（NEXUS），利用从对撞机物理数据中学习到的预训练知识，迁移到其他科学数据集的域外任务，该方法采用参数规模约300万的全连接自编码器模型。该模型在大型强子对撞机（LHC）生成的、以带电粒子径迹特征建模的大规模碰撞数据集上进行无监督预训练。在预训练模型权重的基础上开发了对撞机分析的下游任务，如运动学回归和事例分类，与从头训练的等效架构相比，仅使用少量标注数据集即可实现更高的准确率。此外，通过潜在空间解释及在引力波、洪水预报、神经活动等其他领域的应用，验证了预训练的优势；与同等规模的Transformer方法相比，NEXUS的计算复杂度更低，为科学实验中基础模型的高能效推理及实时或边缘应用打开了大门。
英文摘要
We present a lightweight approach to foundation modeling (\textbf{NEXUS}) that leverages pre-trained learning from collider physics data towards out-of-domain tasks in other scientific datasets, using a fully connected autoencoder model with approximately 3 million parameters. The model pre-trains with no supervision over a large-scale collision dataset from the Large Hadron Collider modeled by charged particle track features. Downstream tasks for collider analyses, such as kinematic regression and event classification, are developed on pre-trained model weights and achieve improved accuracy with only small labeled datasets when compared to equivalent architectures trained from scratch. The benefits of pre-training are additionally investigated through latent space interpretation and application to other domains, including gravitational waves, flood forecasting, and neural activity. Furthermore, the relative computational simplicity of NEXUS is demonstrated compared to transformer approaches at comparable scale, opening the door to power-efficient inference and real-time or edge applications of foundation models in scientific experiments.
发表机构
- Stanford University（斯坦福大学）
- SLAC National Accelerator Laboratory（SLAC国家加速器实验室）
机构由 AI 辅助整理，请以论文原文为准。
