National Institute of Informatics（情报学研究所）；Nantes Université（南特大学）；The Graduate University for Advanced Studies（高级研究大学院大学）；The University of Tokyo（东京大学）；EPFL（洛桑联邦理工学院）
Most existing multi-hop datasets are extractive answer datasets, where the answers to the questions can be extracted directly from the provided context. This often leads models to use heuristics or shortcuts instead of performing true multi-hop reasoning. In this paper, we propose a new multi-hop dataset, MoreHopQA, which shifts from extractive to generative answers. Our dataset is created by utilizing three existing multi-hop datasets: HotpotQA, 2WikiMultihopQA, and MuSiQue. Instead of relying solely on factual reasoning, we enhance the existing multi-hop questions by adding another layer of questioning that involves one, two, or all three of the following types of reasoning: commonsense, arithmetic, and symbolic. Our dataset is created through a semi-automated process, resulting in a dataset with 1,118 samples that have undergone human verification. We then use our dataset to evaluate five different large language models: Mistral 7B, Gemma 7B, Llama 3 (8B and 70B), and GPT-4. We also design various cases to analyze the reasoning steps in the question-answering process. Our results show that models perform well on initial multi-hop questions but struggle with our extended questions, indicating that our dataset is more challenging than previous ones. Our analysis of question decomposition reveals that although models can correctly answer questions, only a portion - 38.7% for GPT-4 and 33.4% for Llama3-70B - achieve perfect reasoning, where all corresponding sub-questions are answered correctly. Evaluation code and data are available at https://github.com/Alab-NII/morehopqa
Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context
Department of Electrical and Computer Engineering, Seoul National University（电子与计算机工程系，首尔国立大学）；AIIS, ASRI, INMC, ISRC, and IPAI, Seoul National University（人工智能研究所、人工智能研究机构、智能网络中心、智能系统研究所以及人工智能与公共政策研究所，首尔国立大学）
BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；Shanghai University of Electric Power（上海电力大学）；Alibaba Group（阿里巴巴集团）；East China Normal University（华东师范大学）
Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
Shanghai Jiao Tong University（上海交通大学）；The University of Sydney（悉尼大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Nanyang Technological University（南洋理工大学）
Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?
ASUS Intelligent Cloud Services (AICS)（华硕智能云服务（AICS））；National University of Singapore（新加坡国立大学）；University of Manchester（曼彻斯特大学）；Imperial Global Singapore（帝国理工学院新加坡分部）
Scaling Reasoning Hop Exposes Weaknesses: Demystifying and Improving Hop Generalization in Large Language Models
University of Science and Technology of China（中国科学技术大学）；City University of Hong Kong（香港城市大学）；Zhejiang University（浙江大学）；City University of Hong Kong, Shenzhen Research Institute（香港城市大学深圳研究院）
Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Models
State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室）；University of Science and Technology of China（中国科学技术大学）；The Hong Kong University of Science and Technology（香港科技大学）
Understanding Reasoning in Chain-of-Thought from the Hopfieldian View
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Soochow University（苏州大学）；University of Southampton（南安普顿大学）；Arizona State University（亚利桑那州立大学）
Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
Chandar Research Lab（Chandar研究实验室）；Mila – Quebec AI Institute（魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；Microsoft Research（微软研究院）；Polytechnique Montréal（蒙特利尔理工学院）；Canada CIFAR AI Chair（加拿大CIFAR人工智能主席）
A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning
KAUST（阿卜杜拉国王科技大学）；Soochow University（苏州大学）；University of Southampton（南安普顿大学）；Tongji University（同济大学）；Universität Hamburg（汉堡大学）；Huazhong University of Science and Technology（华中科技大学）
Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）；National Institute of Informatics（国立信息学研究所）；The Graduate University for Advanced Studies（综合研究大学院大学）
