Improving the reasoning capabilities of large language models (LLMs) has attracted considerable interest. Recent approaches primarily focus on improving the reasoning process to yield a more precise final answer. However, in scenarios involving contextually aware reasoning, these methods neglect the importance of first identifying logical relationships from the context before proceeding with the reasoning. This oversight could lead to a superficial understanding and interaction with the context, potentially undermining the quality and reliability of the reasoning outcomes. In this paper, we propose an information re-organization (InfoRE) method before proceeding with the reasoning to enhance the reasoning ability of LLMs. Our re-organization method involves initially extracting logical relationships from the contextual content, such as documents or paragraphs, and subsequently pruning redundant content to minimize noise. Then, we utilize the re-organized information in the reasoning process. This enables LLMs to deeply understand the contextual content by clearly perceiving these logical relationships, while also ensuring high-quality responses by eliminating potential noise. To demonstrate the effectiveness of our approach in improving the reasoning ability, we conduct experiments using Llama2-70B, GPT-3.5, and GPT-4 on various contextually aware multi-hop reasoning tasks. Using only a zero-shot setting, our method achieves an average absolute improvement of 4% across all tasks, highlighting its potential to improve the reasoning performance of LLMs. Our source code is available at https://github.com/hustcxx/InfoRE.
CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs
School of Computer Science & Engineering, Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（计算机科学与工程学院，新一代人工智能技术及其交叉应用关键实验室（东南大学），教育部，中国）；Data Science & AI Department at Faculty of IT, Monash University, Australia（信息学院数据科学与人工智能部门，墨尔本大学，澳大利亚）
FERA: Uncertainty-Aware Federated Reasoning for Large Language Models
Indiana University（印第安纳大学）；The University of New South Wales（新南威尔士大学）；The Chinese University of Hong Kong（香港中文大学）；University of California San Diego（加州大学圣地亚哥分校）；Adobe Research（Adobe研究院）
Hypothesis Testing Prompting Improves Deductive Reasoning in Large Language Models
Shanghai Jiao Tong University（上海交通大学）；MoE Key Lab of Artificial Intelligence（教育部人工智能重点实验室）；AI Institute（人工智能研究院）；State Key Lab of Advanced Optical Communication System and Network（先进光通信系统与网络国家重点实验室）
