本文提出 APOLLO，利用逻辑推理关键词筛选 Wikipedia 子集，并结合词性选择性掩码与蕴含/矛盾句子分类损失进行自适应预训练，从而在不依赖任务格式的情况下提升语言模型的通用逻辑推理能力。
摘要：
文本的逻辑推理是一项重要能力，它要求理解文本中呈现的信息及其相互联系，然后通过这些信息进行推理以推断新的结论。以往提升语言模型逻辑推理能力的工作需要对训练数据进行复杂处理（例如，将符号知识与文本对齐），从而产生了针对特定任务的数据增强方案，限制了通用逻辑推理技能的学习。在本工作中，我们提出了 APOLLO，这是一种经过自适应预训练、具备更强逻辑推理能力的语言模型。我们基于一组逻辑推理关键词选取 Wikipedia 的一个子集，用于语言模型的持续预训练。我们使用两种自监督损失函数：一种是改进的掩码语言建模损失，其中仅掩码特定词性的词，这些词相比基础语言理解可能需要更多推理；另一种是句子级分类损失，用于教模型区分句子的蕴含与矛盾类型。所提出的训练范式既简单，又独立于任务格式。我们在两个逻辑推理数据集上将 APOLLO 与以往基线进行比较，证明了其有效性。APOLLO 在 ReClor 上表现相当，并在 LogiQA 上优于基线。代码库已公开发布。
英文摘要：
Logical reasoning of text is an important ability that requires understanding the information present in the text, their interconnections, and then reasoning through them to infer new conclusions. Prior works on improving the logical reasoning ability of language models require complex processing of training data (e.g., aligning symbolic knowledge to text), yielding task-specific data augmentation solutions that restrict the learning of general logical reasoning skills. In this work, we propose APOLLO, an adaptively pretrained language model that has improved logical reasoning abilities. We select a subset of Wikipedia, based on a set of logical inference keywords, for continued pretraining of a language model. We use two self-supervised loss functions: a modified masked language modeling loss where only specific parts-of-speech words, that would likely require more reasoning than basic language understanding, are masked, and a sentence-level classification loss that teaches the model to distinguish between entailment and contradiction types of sentences. The proposed training paradigm is both simple and independent of task formats. We demonstrate the effectiveness of APOLLO by comparing it with prior baselines on two logical reasoning datasets. APOLLO performs comparably on ReClor and outperforms baselines on LogiQA. The code base has been made publicly available.
Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension
Microsoft Cloud and AI（微软云与人工智能）；School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
CodePMP: Scalable Preference Model Pretraining for Large Language Model Reasoning
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Xiaohongshu Inc（小红书公司）
Progressive Residual Warmup for Language Model Pretraining
The Hong Kong University of Science（香港科学与技术大学）；NVIDIA, Santa Clara, US（NVIDIA公司）；The University of Hong Kong, Hong Kong, China（香港大学）；University of Surrey, Guildford, UK（萨里大学）
What do Language Models Learn and When? The Implicit Curriculum Hypothesis
Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）；Department of Computer Science, Data Science and AI Institute, Johns Hopkins University（约翰霍普金斯大学计算机科学系、数据科学与人工智能研究所）；Khoury College of Computer Science, Northeastern University（东北大学Khoury计算机科学学院）；Department of Computer Science, University of Southern California（南加州大学计算机科学系）
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
Michigan State University（密歇根州立大学）；Lehigh University（理海大学）；Beihang University（北京航空航天大学）；Macquarie University（麦考瑞大学）；Nanyang Technological University（南洋理工大学）；University of California San Diego（加利福尼亚大学圣地亚哥分校）；Salesforce AI Research（Salesforce人工智能研究院）；Duke University（杜克大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Curing Miracle Steps in LLM Mathematical Reasoning with Rubric Rewards
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（数据科学学院，香港中文大学（深圳））；UC Berkeley（加州大学伯克利分校）；Zhejiang University（浙江大学）；Johns Hopkins University（约翰霍普金斯大学）；Renmin University of China（中国人民大学）；Xiaohongshu Inc.（小红书公司）
