大语言模型学习动力系统的支配原理，揭示上下文内神经标度律
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law
AI总结：
本文研究LLaMA 2在无微调条件下对物理动力系统时间序列的外推能力，发现其预测精度随上下文窗口长度提升，揭示上下文内神经标度律，并提出从LLM提取多位数字概率密度函数的算法。
摘要：
预训练大语言模型（LLMs）在零样本任务中表现出惊人的有效性，包括时间序列预测。然而，由于模型的复杂性，理解这种能力背后的机制仍然极具挑战性。我们研究了LLMs外推动力系统行为的能力，这些系统的演化由具有物理意义的原理支配。我们的结果表明，LLaMA 2这一主要基于文本训练的语言模型，无需微调或提示工程即可实现对动力系统时间序列的准确预测。此外，所学物理规则的准确性随着输入上下文窗口长度的增加而提高，揭示了一种上下文内版本的神经标度律。在此过程中，我们提出了一种灵活且高效的算法，可直接从LLMs中提取多位数字的概率密度函数。
英文摘要：
Pretrained large language models (LLMs) are surprisingly effective at performing zero-shot tasks, including time-series forecasting. However, understanding the mechanisms behind such capabilities remains highly challenging due to the complexity of the models. We study LLMs' ability to extrapolate the behavior of dynamical systems whose evolution is governed by principles of physical interest. Our results show that LLaMA 2, a language model trained primarily on texts, achieves accurate predictions of dynamical system time series without fine-tuning or prompt engineering. Moreover, the accuracy of the learned physical rules increases with the length of the input context window, revealing an in-context version of neural scaling law. Along the way, we present a flexible and efficient algorithm for extracting probability density functions of multi-digit numbers directly from LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
