PhysNLU：用于评估物理学中自然语言理解与解释连贯性的语言资源
PhysNLU: A Language Resource for Evaluating Natural Language Understanding and Explanation Coherence in Physics
AI总结：
本文构建PhysNLU数据集集合，从句子排序、位置、章节预测和话语连贯性等方面评估语言模型对物理解释的理解能力，并通过基线揭示现有模型的不足。
摘要：
为了使语言模型能够辅助物理学研究，它们必须首先对数学话语和自然语言话语进行表征编码，从而产生连贯的解释，并保证陈述具有正确的顺序和相关性。我们提出了一组数据集，用于评估语言模型在这方面的表现，测量其在句子排序、位置、章节预测和话语连贯性方面的能力。对数据的分析揭示了物理学话语中最常见的方程和子学科，以及方程和表达式在句子层面的出现频率。我们给出的基线表明，即使在数学自然语言目标上进行训练，当代语言模型仍会受到物理学中与连贯性相关任务的挑战。
英文摘要：
In order for language models to aid physics research, they must first encode representations of mathematical and natural language discourse which lead to coherent explanations, with correct ordering and relevance of statements. We present a collection of datasets developed to evaluate the performance of language models in this regard, which measure capabilities with respect to sentence ordering, position, section prediction, and discourse coherence. Analysis of the data reveals equations and sub-disciplines which are most common in physics discourse, as well as the sentence-level frequency of equations and expressions. We present baselines that demonstrate how contemporary language models are challenged by coherence related tasks in physics, even when trained on mathematical natural language objectives.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
