Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enabling them to understand the logic in natural language and generate logic-consistent responses. However, the representational differences between unstructured and structured knowledge make LLMs inherently struggle to maintain logic consistency, leading to \textit{Logic Drift} challenges in structured knowledge reasoning tasks such as Knowledge Graph Question Answering (KGQA). Existing methods address this limitation by designing complex workflows embedded in prompts to guide LLM reasoning. Nevertheless, these approaches only provide input-level guidance and fail to fundamentally address the \textit{Logic Drift} in LLM outputs. Additionally, their inflexible reasoning workflows cannot adapt to different tasks and knowledge graphs. To enhance LLMs' logic consistency in structured knowledge reasoning, we specifically target the logits output from the autoregressive generation process. We propose the \textit{Logits-to-Logic} framework, which incorporates logits strengthening and logits filtering as core modules to correct logical defects in LLM outputs. Extensive experiments show that our approach significantly improves LLMs' logic consistency in structured knowledge reasoning and achieves state-of-the-art performance on multiple KGQA benchmarks.
Empowering LLMs with Logical Reasoning: A Comprehensive Survey
Institute for Logic, Language and Computation, University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所);Center for Data Science, Peking University(北京大学数据科学中心);Tsinghua-UvA JRC for Logic, Department of Philosophy, Tsinghua University(清华大学哲学系);Machine Learning Department, MBZUAI(MBZUAI机器学习部门);Department of Philosophy, CMU(卡内基梅隆大学哲学系);State Key Lab of General AI, School of Intelligence Science and Technology, Peking University(北京大学人工智能通用实验室);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究所);Pazhou Laboratory (Huangpu), Guangzhou(广州黄埔实验室)
R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Beijing Normal University(北京师范大学)
Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrast
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Renmin University of China(中国人民大学);Xiaohongshu Inc.(小红书公司)
LR^2Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhongguancun Academy(中关村学院);Wuhan AI Research(武汉人工智能研究院)
Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
Fordham University(福特汉姆大学);Algoverse AI Research(Algoverse AI研究);Rice University(稻子大学);UC San Diego(圣地亚哥大学);Santa Clara University(圣克拉拉大学);LMU Munich(慕尼黑路德维希-马克西米利安大学)
Identifying Semantic Induction Heads to Understand In-Context Learning
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)