TransformerPayne：通过捕获长程相关性提升光谱仿真精度与数据效率
TransformerPayne: enhancing spectral emulation accuracy and data efficiency by capturing long-range correlations
AI总结：
本文提出TransformerPayne光谱仿真器，利用Transformer捕获光谱长程相关性，在精度、数据效率和可解释性上均优于现有方法，并支持跨网格微调以减少训练数据需求。
摘要：
恒星光谱仿真器通常依赖大规模网格，并且往往在仿真精度上达到平台期，导致在推断恒星性质时产生显著的系统误差。我们的研究探索了使用Transformer模型来捕获光谱中的长程信息，并将其性能与The Payne仿真器（一个全连接多层感知机）、The Payne的扩展版本以及一个基于卷积的仿真器进行了比较。我们在合成光谱网格上测试了这些模型，通过分析仿真残差和评估光谱参数推断质量来评价其性能。新提出的TransformerPayne仿真器优于所有其他测试模型，在完整网格上训练时达到了约0.15%的平均绝对误差（MAE）。最显著的改进出现在包含1000到10000条光谱的网格中，TransformerPayne的性能比扩大规模的The Payne版本好2到5倍。此外，TransformerPayne展示了优越的微调能力，允许在一个光谱模型网格上进行预训练，然后迁移到另一个网格。与从头训练的模型相比，这种微调方法使训练网格规模最多减少了十倍。对TransformerPayne注意力图的分析表明，它们编码了所选元素的许多谱线中共同的可解释特征。虽然将The Payne扩展到更大的网络使其在完整数据集上训练时的MAE从1.2%降低到0.3%，但TransformerPayne在所有测试中始终取得最低的MAE。TransformerPayne仿真器的归纳偏置在光谱仿真方面相较于现有方法提升了精度、数据效率和可解释性。
英文摘要：
Stellar spectra emulators often rely on large grids and tend to reach a plateau in emulation accuracy, leading to significant systematic errors when inferring stellar properties. Our study explores the use of Transformer models to capture long-range information in spectra, comparing their performance to The Payne emulator (a fully connected multilayer perceptron), an expanded version of The Payne, and a convolutional-based emulator. We tested these models on synthetic spectra grids, evaluating their performance by analyzing emulation residuals and assessing the quality of spectral parameter inference. The newly introduced TransformerPayne emulator outperformed all other tested models, achieving a mean absolute error (MAE) of approximately 0.15% when trained on the full grid. The most significant improvements were observed in grids containing between 1000 and 10,000 spectra, with TransformerPayne showing 2 to 5 times better performance than the scaled-up version of The Payne. Additionally, TransformerPayne demonstrated superior fine-tuning capabilities, allowing for pretraining on one spectral model grid before transferring to another. This fine-tuning approach enabled up to a tenfold reduction in training grid size compared to models trained from scratch. Analysis of TransformerPayne's attention maps revealed that they encode interpretable features common across many spectral lines of chosen elements. While scaling up The Payne to a larger network reduced its MAE from 1.2% to 0.3% when trained on the full dataset, TransformerPayne consistently achieved the lowest MAE across all tests. The inductive biases of the TransformerPayne emulator enhance accuracy, data efficiency, and interpretability for spectral emulation compared to existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
