LASS-ODE：将ODE计算扩展以连接基础模型与动态物理系统
LASS-ODE: Scaling ODE Computations to Connect Foundation Models with Dynamical Physical Systems
AI总结：
LASS-ODE通过局部线性ODE表示和跨系统注意力机制，提升物理系统动态预测的可扩展性和知识共享效率。
摘要：
基础模型已改变了语言、视觉和时间序列数据的分析，但对物理系统的动态预测进展有限。鉴于物理约束的复杂性，有两个挑战突出。（i）物理计算可扩展性：物理引导的学习可以强制物理正则化，但其计算（例如ODE积分）无法扩展到大规模系统。（ii）知识共享效率：注意力机制主要在每个系统内计算，这限制了系统间共享ODE结构的提取。我们表明，强制ODE一致性不需要昂贵的非线性积分：逐token的局部线性ODE表示在保持物理保真度的同时能够扩展到基础模型领域。因此，我们提出了新的token表示，以尊重局部线性ODE演进。这种线性性大大加速了积分，同时准确地近似了局部数据流形。第二，我们引入了一种简单而有效的跨系统注意力，通过一个存储共享token的公共结构库（CSH）来增强注意力，从而在系统间聚合知识。所得到的模型称为LASS-ODE（LArge-Scale SmaLL ODE），在我们的40GB ODE轨迹集合上进行预训练，以实现强大的领域内性能、跨不同ODE系统的零样本泛化能力，以及通过微调进一步改进。
英文摘要：
Foundation models have transformed language, vision, and time series data analysis, yet progress on dynamic predictions for physical systems remains limited. Given the complexity of physical constraints, two challenges stand out. $(i)$ Physics-computation scalability: physics-informed learning can enforce physical regularization, but its computation (e.g., ODE integration) does not scale to extensive systems. $(ii)$ Knowledge-sharing efficiency: the attention mechanism is primarily computed within each system, which limits the extraction of shared ODE structures across systems. We show that enforcing ODE consistency does not require expensive nonlinear integration: a token-wise locally linear ODE representation preserves physical fidelity while scaling to foundation-model regimes. Thus, we propose novel token representations that respect locally linear ODE evolution. Such linearity substantially accelerates integration while accurately approximating the local data manifold. Second, we introduce a simple yet effective inter-system attention that augments attention with a common structure hub (CSH) that stores shared tokens and aggregates knowledge across systems. The resulting model, termed LASS-ODE (\underline{LA}rge-\underline{S}cale \underline{S}mall \underline{ODE}), is pretrained on our $40$GB ODE trajectory collections to enable strong in-domain performance, zero-shot generalization across diverse ODE systems, and additional improvements through fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
