To enhance the reasoning capabilities of large language models (LLMs), self-consistency has become a popular approach, combining multiple samplings with majority voting. However, current methods are computationally expensive and time-consuming due to the need for numerous samplings. To address this, this paper introduces path-consistency, which leverages the confidence of earlier-generated answers to identify the most promising prefix and guide the generation of subsequent branches. By dynamically guiding the generation of subsequent branches based on this prefix, path-consistency mitigates both the errors and redundancies from random or less useful sampling in self-consistency. This approach reduces errors and redundancies from random sampling, significantly accelerating inference by minimizing token consumption. Our extensive empirical results demonstrate that path-consistency improves inference latency by up to 40.5\%, while maintaining task accuracy across various tasks, including mathematical reasoning, commonsense reasoning, and symbolic reasoning.
Nagoya University(名古屋大学);Nara Institute of Science and Technology(奈良科学技術大學);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);RIKEN AIP(理化学研究所(AIP))
Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller
Shenzhen University(深圳大学);Rensselaer Polytechnic Institute(伦斯勒理工学院);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Tsinghua University(清华大学);The University of Hong Kong(香港大学);California Institute of Technology(加州理工学院)
A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Zhejiang University(浙江大学)
Internal Consistency and Self-Feedback in Large Language Models: A Survey
School of Information, Renmin University of China(中国人民大学信息学院);Large Language Model Center, Institute for Advanced Algorithms Research(高级算法研究院大语言模型中心);School of Computer Science and Technology, Beijing Institute of Technology(北京理工大学计算机科学与技术学院);State Key Laboratory of Media Convergence Production Technology and Systems, Xinhua News Agency(新华社媒体融合生产技术与系统国家重点实验室)
Stepwise Reasoning Enhancement for LLMs via External Subgraph Generation
School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院);School of Computer Science and Technology, Chongqing University of Posts and Telecommunications(重庆邮电大学计算机科学与技术学院)
Understanding Temporal Logic Consistency in Video-Language Models through Cross-Modal Attention Discriminability
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China(北京理工大学计算机科学与技术学院,北京,中国);Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing Institute of Technology, Beijing, China(高性能语言信息处理与云计算应用北京工程研究中心,北京理工大学,北京,中国)
Enhancing Retrieval-Augmented LMs with a Two-stage Consistency Learning Compressor
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Artificial Intelligence Laboratory, China Mobile Communication Group Tianjin Co., Ltd.(中国移动通信集团天津有限公司人工智能实验室);University College London(伦敦大学学院)