高能物理中多模态数据的精准分词
Precision Tokenization for Multi-Modal Data in High Energy Physics
AI总结:
本文针对高能物理中基础模型所用分词器研究不足的问题,将分词视为率失真问题,提出适用于多模态HEP数据的设计规则,开发开源库heptokens,实现4-11倍压缩下保留关键信息且喷注$p_T$分辨率低于1%。
摘要:
近年来,基础模型在高能物理(HEP)等领域受到广泛关注,然而将探测器数据转换为这类模型所需离散输入的分词器却鲜有研究。本文针对这一空白,学习HEP数据集的离散且压缩的分词表示。我们在覆盖两种不同复杂度模拟探测器的四种模态上验证了该方法,将分词视为率失真问题,通过系统的超参数扫描得到适用于所有研究的集合型HEP数据模态的四条分词设计规则。在喷注$p_T$回归、粒子流重建和味道标记等任务上评估分词代价,发现压缩比约为4-11×时,分词保留了这些任务所需的大部分信息,且喷注$p_T$分辨率保持在1%以下。我们还推出了用于HEP数据分词的开源库heptokens。
英文摘要:
Foundation models have recently gained significant attention in fields such as high energy physics (HEP). The tokenizers that convert detector data into discrete inputs for such models, however, are comparatively unexplored. This paper addresses this gap by learning discrete and compressed tokenized representations of HEP datasets. We demonstrate the approach on four modalities covering two different simulated detectors that vary in complexity. Treating tokenization as a rate-distortion problem, a systematic hyperparameter scan yields four design rules for tokenizing HEP data that hold across all set-based modalities studied. The cost of tokenization is evaluated on tasks including jet $p_T$ regression, particle flow reconstruction, and flavor tagging. We find that tokenization preserves much of the information needed for these tasks at compression factors of approximately 4-11$\times$ while keeping the jet $p_T$ resolution below 1%. We also present heptokens, an open-source library for tokenizing HEP data.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。