The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);University of Cambridge(剑桥大学);University of Edinburgh(爱丁堡大学);City University of Hong Kong(香港城市大学)
In this work, we propose a novel method named \textbf{Auto}mated \textbf{P}rocess-\textbf{S}upervised \textbf{V}erifier (\textbf{\textsc{AutoPSV}}) to enhance the reasoning capabilities of large language models (LLMs) by automatically annotating the reasoning steps. \textsc{AutoPSV} begins by training a verification model on the correctness of final answers, enabling it to generate automatic process annotations. This verification model assigns a confidence score to each reasoning step, indicating the probability of arriving at the correct final answer from that point onward. We detect relative changes in the verification's confidence scores across reasoning steps to automatically annotate the reasoning process, enabling error detection even in scenarios where ground truth answers are unavailable. This alleviates the need for numerous manual annotations or the high computational costs associated with model-induced annotation approaches. We experimentally validate that the step-level confidence changes learned by the verification model trained on the final answer correctness can effectively identify errors in the reasoning steps. We demonstrate that the verification model, when trained on process annotations generated by \textsc{AutoPSV}, exhibits improved performance in selecting correct answers from multiple LLM-generated outputs. Notably, we achieve substantial improvements across five datasets in mathematics and commonsense reasoning. The source code of \textsc{AutoPSV} is available at \url{https://github.com/rookie-joe/AutoPSV}.
AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models
Alibaba Group(阿里巴巴集团);School of Computer Science, Fudan University(复旦大学计算机科学技术学院);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海市智能信息处理重点实验室)
Bolzano: From Expert-Guided Proof Search to Automated Open-Problem Solving
Computer Science Institute, Charles University(查尔斯大学计算机科学研究所);Institute of Formal and Applied Linguistics, Charles University(查尔斯大学形式与应用语言学研究所);Department of Applied Mathematics, Charles University(查尔斯大学应用数学系);Institute of Mathematics, Czech Academy of Sciences(捷克科学院数学研究所)
StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning
Department of Computer Science\&Engineering, University of Minnesota-Twin Cities, Minnesota, USA;Department of Electrical Engineering, University of Minnesota-Twin Cities, Minnesota, USA
Efficient Process Reward Modeling via Contrastive Mutual Information
Department of Electrical and Computer Engineering, Seoul National University(首尔大学电气与计算机工程系);Department of Computer Science and Engineering, Konkuk University(建国大学计算机科学与工程系)
FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);City Univeristy of Hong Kong(香港城市大学);Sun Yat-sen University(中山大学);The University of Hong Kong(香港大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);MBZUAI(Mohamed bin Zayed 人工智能大学);DarkMatter AI Research(DarkMatter AI 研究院)