NIERT：使用Transformer编码器统一离散数据表示的精确数值插值
NIERT: Accurate Numerical Interpolation through Unifying Scattered Data Representations using Transformer Encoder
摘要：
散乱数据的插值问题是数值分析中的经典问题，在理论和实践上都有着悠久的历史。最近的进展利用深度神经网络来构造插值器，表现出优异的和可推广的性能。然而，它们仍然在两个方面不足：\textbf{1) inadequate representation learning}，由流行的编码器-解码器框架中的观察点和目标点的单独嵌入引起，以及\textbf{2) limited generalization power}，由忽略跨不同域共享的先前插值知识引起。为了克服这些局限性，我们提出了一种使用\textbf{E}编码器\textbf{R}表示\textbf{T}变换器（称为\textbf{NIERT}）的\textbf{N}数值\textbf{I}插值方法。一方面，NIERT利用仅编码器框架而不是编码器-解码器结构。通过这种方式，NIERT可以将观察点和目标点嵌入到统一的编码器表示空间中，从而有效地利用它们之间的相关性并获得更精确的表示。另一方面，我们建议在大规模综合数学函数上预训练NIERT以获得先验插值知识，并将其转移到具有一致性能增益的多个插值域。在合成和真实世界的数据集上，NIERT都比现有的方法有很大的优势，即，4.3 $\sim$ 14.3 $\times$降低TFRD子集的MAE，1.7/1.8/8.7 $\times$降低Mathit/PhysioNet/PTV数据集的MSE。NIERT的源代码可在https://github.com/DingShizhe/NIERT上获得。
英文摘要：
Interpolation for scattered data is a classical problem in numerical analysis, with a long history of theoretical and practical contributions. Recent advances have utilized deep neural networks to construct interpolators, exhibiting excellent and generalizable performance. However, they still fall short in two aspects: \textbf{1) inadequate representation learning}, resulting from separate embeddings of observed and target points in popular encoder-decoder frameworks and \textbf{2) limited generalization power}, caused by overlooking prior interpolation knowledge shared across different domains. To overcome these limitations, we present a \textbf{N}umerical \textbf{I}nterpolation approach using \textbf{E}ncoder \textbf{R}epresentation of \textbf{T}ransformers (called \textbf{NIERT}). On one hand, NIERT utilizes an encoder-only framework rather than the encoder-decoder structure. This way, NIERT can embed observed and target points into a unified encoder representation space, thus effectively exploiting the correlations among them and obtaining more precise representations. On the other hand, we propose to pre-train NIERT on large-scale synthetic mathematical functions to acquire prior interpolation knowledge, and transfer it to multiple interpolation domains with consistent performance gain. On both synthetic and real-world datasets, NIERT outperforms the existing approaches by a large margin, i.e., 4.3$\sim$14.3$\times$ lower MAE on TFRD subsets, and 1.7/1.8/8.7$\times$ lower MSE on Mathit/PhysioNet/PTV datasets. The source code of NIERT is available at https://github.com/DingShizhe/NIERT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
