Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for Large Language Models (LLMs) to address knowledge-intensive queries requiring domain-specific or up-to-date information. To handle complex multi-hop questions that are challenging for single-step retrieval, iterative RAG approaches incorporating reinforcement learning have been proposed. However, existing iterative RAG systems typically plan to decompose questions without leveraging information about the available retrieval corpus, leading to inefficient retrieval and reasoning chains that cascade into suboptimal performance. In this paper, we introduce Early Knowledge Alignment (EKA), a simple but effective module that aligns LLMs with retrieval set before planning in iterative RAG systems with contextually relevant retrieved knowledge. Extensive experiments on six standard RAG datasets demonstrate that by establishing a stronger reasoning foundation, EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency. Our analysis from an entropy perspective demonstrate that incorporating early knowledge reduces unnecessary exploration during the reasoning process, enabling the model to focus more effectively on relevant information subsets. Moreover, EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models. Generalization tests across diverse datasets and retrieval corpora confirm the robustness of our approach. Overall, EKA advances the state-of-the-art in iterative RAG systems while illuminating the critical interplay between structured reasoning and efficient exploration in reinforcement learning-augmented frameworks. The code is released at \href{https://github.com/yxzwang/EarlyKnowledgeAlignment}{Github}.
Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models
University of Colorado Boulder(科罗拉多大学博尔德分校);University of Central Florida(中佛罗里达大学);University of Maryland College Park(马里兰大学帕克分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)
National Institute of Informatics(情报学研究所);Nantes Université(南特大学);The Graduate University for Advanced Studies(高级研究大学院大学);The University of Tokyo(东京大学);EPFL(洛桑联邦理工学院)
Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学计算机科学与技术系);Alibaba Group, China(阿里巴巴集团)
Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?
ASUS Intelligent Cloud Services (AICS)(华硕智能云服务(AICS));National University of Singapore(新加坡国立大学);University of Manchester(曼彻斯特大学);Imperial Global Singapore(帝国理工学院新加坡分部)
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
University of Science and Technology of China(中国科学技术大学);City University of Hong Kong(香港城市大学);Zhejiang University(浙江大学);City University of Hong Kong, Shenzhen Research Institute(香港城市大学深圳研究院)
Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models
State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);University of Science and Technology of China(中国科学技术大学);The Hong Kong University of Science and Technology(香港科技大学)
SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning
School of Software Engineering, Sun Yat-sen University(中山大学软件工程学院);School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院);Merchants Union Consumer Finance Company Limited(招商局联融消费金融股份有限公司)
AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));State University of New York at Buffalo(纽约州立大学布法罗分校);Huawei International, Singapore(新加坡华为国际)
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Microsoft Research(微软研究院);University of Michigan(密歇根大学);Shanghai Jiao Tong University(上海交通大学);CUHKSZ(香港中文大学(深圳));THU(清华大学)
Understanding Reasoning in Chain-of-Thought from the Hopfieldian View
King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Soochow University(苏州大学);University of Southampton(南安普顿大学);Arizona State University(亚利桑那州立大学)
Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
National Institute of Advanced Industrial Science and Technology(产业技术综合研究所);National Institute of Informatics(国立信息学研究所);The Graduate University for Advanced Studies(综合研究大学院大学)