通过受扩散启发的时间 Transformer 算子(DiTTO)实现实时推理与外推
Real-time Inference and Extrapolation via a Diffusion-inspired Temporal Transformer Operator (DiTTO)
AI总结:
本文提出 DiTTO 算子学习方法,结合潜在扩散条件机制与 Transformer,在无时间离散化条件下连续求解时间相关 PDE,并实现实时推理、长时外推和零样本时间超分辨。
摘要:
外推仍然是深度神经网络在所有应用领域中的一项重大挑战。我们提出一种算子学习方法,用于连续求解含时间的偏微分方程(PDE),并在时间上进行外推,而无需任何时间离散化。所提出的方法名为 Diffusion-inspired Temporal Transformer Operator(DiTTO,受扩散启发的时间 Transformer 算子),其灵感来自潜在扩散模型及其条件机制;我们利用该机制纳入 PDE 的时间演化,并结合 transformer 架构中的元素以增强其能力。训练完成后,DiTTO 可以进行实时推理。我们通过估算数年内的全球气温,在一个气候问题上展示了其外推能力;同时还将其用于模拟双锥体周围的高超声速流动。我们提出了涉及时间捆绑和子采样的不同训练策略,并在多个基准上展示了性能提升,实现了长时间间隔外推以及时间上的零样本超分辨率。
英文摘要:
Extrapolation remains a grand challenge in deep neural networks across all application domains. We propose an operator learning method to solve time-dependent partial differential equations (PDEs) continuously and with extrapolation in time without any temporal discretization. The proposed method, named Diffusion-inspired Temporal Transformer Operator (DiTTO), is inspired by latent diffusion models and their conditioning mechanism, which we use to incorporate the temporal evolution of the PDE, in combination with elements from the transformer architecture to improve its capabilities. Upon training, DiTTO can make inferences in real-time. We demonstrate its extrapolation capability on a climate problem by estimating the temperature around the globe for several years, and also in modeling hypersonic flows around a double-cone. We propose different training strategies involving temporal-bundling and sub-sampling and demonstrate performance improvements for several benchmarks, performing extrapolation for long time intervals as well as zero-shot super-resolution in time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。