Retrieval-Augmented Generation (RAG) has become a standard approach for knowledge-intensive question answering, but existing systems remain brittle on multi-hop questions, where solving the task requires chaining multiple retrieval and reasoning steps. Key challenges are that current methods represent reasoning through free-form natural language, where intermediate states are implicit, retrieval queries can drift from intended entities, and errors are detected by the same model that produces them making self-reflection an unreliable, ungrounded signal.
We observe that multi-hop question answering is a typical form of step-by-step computation, and that this structured process aligns closely with how code-specialized language models are trained to operate. Motivated by this, we introduce \pyrag, a framework that reformulates multi-hop RAG as program synthesis and execution. Instead of free-form reasoning trajectories, \pyrag represents the reasoning process as an executable Python program over retrieval and QA tools, exposing intermediate states as variables, producing deterministic feedback through execution, and yielding an inspectable trace of the entire reasoning process. This formulation further enables compiler-grounded self-repair and execution-driven adaptive retrieval without any additional training.
Experiments on five QA benchmarks (PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, and Bamboogle) show that \pyrag consistently outperforms strong baselines under both training-free and RL-trained settings, with especially large gains on compositional multi-hop datasets. Our code, data and models are publicly available at https://github.com/GasolSun36/PyRAG.
SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
Midea Group(美的集团);Beijing University of Posts and Telecommunications(北京邮电大学);Donghua University(东华大学);The University of Sydney(悉尼大学);Peking University(北京大学)
SEAL: Self-Evolving Agentic Learning for Conversational Question Answering over Knowledge Graphs
Institute of Big Data and Artificial Intelligence, China Telecom Research Institute(大数据与人工智能研究院,中国电信研究院);Wangxuan Institute of Computer Technology, Peking University(王宣计算机技术研究所,北京大学);School of Artificial Intelligence, China University of Geosciences (Beijing)(人工智能学院,中国地质大学(北京));Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University(认知与神经工效学中心,认知神经科学与学习国家重点实验室,北京师范大学);Institute of Artificial Intelligence and Future Networks, Beijing Normal University(人工智能与未来网络研究院,北京师范大学)
Next-Generation LLM for UAV: From Natural Language to Autonomous Flight
School of Electrical and Computer Engineering, Purdue University(电子与计算机工程学院,普渡大学);School of Aeronautics and Astronautics, Purdue University(航空与航天学院,普渡大学);Department of Computer Science and Engineering, Yonsei University(计算机科学与工程系,延世大学);Polytechnic Institute, Purdue University(技术学院,普渡大学)
ST-Bench: A Spatial-Temporal Benchmark for Multi-Agent System Generation on Scientific Research Tasks
Rutgers University(罗格斯大学);University of Pittsburgh(匹兹堡大学);University of Minnesota(明尼苏达大学);Oak Ridge National Laboratory(橡树岭国家实验室);NEC Labs America(NEC美国实验室);University of Maryland(马里兰大学)