Optimizing Language Model's Reasoning Abilities with Weak Supervision
University of California, San Diego(加利福尼亚大学圣迭戈分校);University of Southern California(南加利福尼亚大学);University of Pennsylvania(宾夕法尼亚大学);Yale University(耶鲁大学);Washington University in St. Louis(圣路易斯华盛顿大学)
While Large Language Models (LLMs) have demonstrated proficiency in handling complex queries, much of the past work has depended on extensively annotated datasets by human experts. However, this reliance on fully-supervised annotations poses scalability challenges, particularly as models and data requirements grow. To mitigate this, we explore the potential of enhancing LLMs' reasoning abilities with minimal human supervision. In this work, we introduce self-reinforcement, which begins with Supervised Fine-Tuning (SFT) of the model using a small collection of annotated questions. Then it iteratively improves LLMs by learning from the differences in responses from the SFT and unfinetuned models on unlabeled questions. Our approach provides an efficient approach without relying heavily on extensive human-annotated explanations. However, current reasoning benchmarks typically only include golden-reference answers or rationales. Therefore, we present \textsc{PuzzleBen}, a weakly supervised benchmark that comprises 25,147 complex questions, answers, and human-generated rationales across various domains, such as brainteasers, puzzles, riddles, parajumbles, and critical reasoning tasks. A unique aspect of our dataset is the inclusion of 10,000 unannotated questions, enabling us to explore utilizing fewer supersized data to boost LLMs' inference capabilities. Our experiments underscore the significance of \textsc{PuzzleBen}, as well as the effectiveness of our methodology as a promising direction in future endeavors. Our dataset and code will be published soon on \texttt{Anonymity Link}.
LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机科学学院,北京大学);Pattern Recongnition Center, WechatAI, Tencent Inc(图案识别中心,微信AI,腾讯公司);Dept. of Comp. Sci. & Tech., Tsinghua University(计算机科学与技术系,清华大学)
Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiaohongshu Inc(小红书公司)
Self-Evolving Critique Abilities in Large Language Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院);Shenzhen International Center for Industrial and Applied Mathematics(深圳国际工业与应用数学中心);Qwen Team, Alibaba Inc.(阿里云通义团队)
In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement
Center for Advanced Intelligence Project, RIKEN(RIKEN先进智能项目中心);Northeastern University(东北大学);UNC-Chapel Hill(北卡罗来纳大学教堂山分校);Center for Advanced Intelligence Project, RIKEN Graduate School of Frontier Sciences, The University of Tokyo(RIKEN前沿科学研究生院、东京大学先进智能项目中心)
An Efficient and Precise Training Data Construction Framework for Process-supervised Reward Model in Mathematical Reasoning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Exploring the Syntactic Abilities of RNNs with Multi-task Learning
PSL Research University(巴黎文理研究大学);Bar Ilan University(巴伊兰大学);CNRS(法国国家科学研究中心);EHESS(法国社会科学高等研究院);Johns Hopkins University(约翰斯·霍普金斯大学)
最近的工作已经探索了RNN的句法能力,使用主语-动词一致性任务,诊断对句子结构的敏感性。RNN在常见情况下很好地执行了这项任务,但在复杂句子中却表现不佳(Linzen et al.,2016年)。我们测试这些错误是否是由于固有的限制架构或相对间接的监督提供的大多数协议依赖于语料库。我们训练了一个RNN来执行协议任务和一个额外的任务,无论是CCG超级标记还是语言建模。多任务训练导致出错率显著降低,特别是在复杂句子上,这表明RNN有能力进化出比以前更复杂的句法表示。我们还表明,容易获得的协议训练数据可以提高其他语法任务的性能,特别是当只有有限的训练数据可用于这些任务。多任务范例还可以被利用来将语法知识注入到语言模型中。
Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Chandar Research Lab(Chandar研究实验室);Mila – Quebec AI Institute(魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);Microsoft Research(微软研究院);Polytechnique Montréal(蒙特利尔理工学院);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
KnowRA: Knowledge Retrieval Augmented Method for Document-level Relation Extraction with Comprehensive Reasoning Abilities
Nanjing Normal University, School of Computer Science and Electronic Informatics(南京师范大学计算机科学与电子信息学院);State Key Laboratory for Novel Software Technology at Nanjing University(南京大学新型软件技术国家重点实验室);Nanjing University, School of Computer Science(南京大学计算机科学学院)
Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reports
DAMO Academy, Alibaba Group(阿里巴巴集团达摩学院);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Hupan Lab(湖畔实验室);Sun Yat-sen University Cancer Center(中山大学肿瘤防治中心);South China Hospital, Medical School, Shenzhen University(深圳大学医学院华南医院)