RINO:无标签的重正化组蒸馏
RINO: Renormalization Group Distillation with No Labels
AI总结:
本研究针对高能物理中监督学习依赖带标签模拟数据的偏差问题,提出 RINO 方法,利用喷注的重正化组结构进行无标签自监督学习,在 JetNet 数据集上实现了优异的分布外分类性能。
摘要:
高能物理(HEP)中的监督机器学习需要对粒子碰撞的带标签模拟数据进行训练,而这类模拟数据已知与 underlying 物理过程存在偏差,这导致训练好的模型迁移到真实数据时出现显著的不准确性和偏差,限制了深度学习方法的应用效果。自监督学习(SSL)是直接在真实无标签数据上进行预训练的有前景的替代方案,但现有技术大多采用图像、自然语言等领域的归纳偏置。我们提出 RINO(Renormalization Group DIstillation with NO Labels,无标签重正化组蒸馏),首次利用高能喷注形成的多尺度重正化组结构,作为基于物理的 SSL 视图来源。我们通过不同层级的层次聚类在能量尺度上定义教师-学生表示,采用 DINO 风格的自蒸馏进行对齐。为了反映真实对撞机数据以背景为主的特性,RINO 仅在量子色动力学(QCD)过程产生的喷注上进行预训练,并在稀有信号类别的分布外(OOD)迁移任务上进行评估。在我们的可控数据效率研究中,RINO 在仅对 1% 的带标签数据进行微调后,性能可与 JetNet 上的全监督训练相媲美;使用完整带标签数据集时,RINO 对顶夸克喷注的 OOD 分类准确率提升 5.9%,且无需重新训练,对未见过的希格斯玻色子衰变通道 $H \ o 4q$ 的 AUC 较最强的 SSL 基准提升 4.1%。尺度结构诊断显示,DINO 风格的自蒸馏使我们的骨干网络保留了尺度依赖结构, ablation 研究表明,这一特性使得 RINO 相较于 SimCLR 等其他 SSL 范式,OOD 准确率差距缩小了 8.6%。代码可在该 https URL 获取。
英文摘要:
Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel $H \to 4q$ by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://github.com/zichunhao/RINO.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。