Long-context language models support ever longer context windows, and evaluations typically treat context length as the main indicator of difficulty. However, context length alone cannot explain why models fail on long-context tasks that require reasoning over complex information. Contexts of equal length can pose very different challenges: a task may be solvable from a single local passage, or it may require integrating evidence distributed across the full context. We quantitatively characterise this difficulty through three properties: evidence scope, how much necessary information must be found; evidence dispersion, how widely that information is scattered within the context; and reasoning depth, how many dependent steps are needed to derive an answer from it. We construct LURCH, a multi-domain question-answering benchmark of 6,550 instances with supporting evidence and reasoning graphs, enabling all three properties to be quantified. Evaluations of proprietary and open-weight models show that: (1) at fixed context length, all our three properties are strongly negatively correlated with model performance; (2) as context length increases, performance degrades sharply on samples with dispersed evidence, whereas it remains relatively stable on samples with localised evidence; and (3) difficulty compounds across properties, as questions with moderately high values on all three are harder than those with an extreme value on just one. These findings establish evidence scope, dispersion and reasoning depth as measures of long-context reasoning difficulty that complement context length.
Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting
Tencent Jarvis Lab(腾讯天衍实验室);Carnegie Mellon University(卡内基梅隆大学);Université de Montréal(蒙特利尔大学);Peking University(北京大学);The University of Sheffield(谢菲尔德大学)
Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning
Department of Computer Science, University College London, UK(英国伦敦大学学院计算机科学系);Centre for Artificial Intelligence, University College London, UK(英国伦敦大学学院人工智能中心)
ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shaping
University of California, Los Angeles(加州大学洛杉矶分校);The Hong Kong University of Science and Technology(香港科技大学);Columbia University(哥伦比亚大学);The Chinese University of Hong Kong(香港中文大学)
HiDiffTIR: Hierarchical Difficulty-Aware Policy Optimization for Multi-Turn Tool-Integrated Reasoning
State Key Laboratory of AI Safety(人工智能安全国家重点实验室);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Meituan(美团)
Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translation
Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);School of Informatics, Xiamen University(厦门大学信息学院);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism(文化和旅游部闽台非物质文化遗产数字化保护与智能处理重点实验室(厦门大学))
Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);AMAP, Alibaba Group(阿里集团AMAP);Xiamen University(厦门大学);Dalian University of Technology(大连理工大学)
Exploring the System 1 Thinking Capability of Large Reasoning Models
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Tongyi Lab, Alibaba Group(阿里巴巴集团通义实验室)
CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density
UiT - The Arctic University of Norway(UiT - 北极大学);Integreat - Norwegian Centre for knowledge-driven machine learning(Integreat - 挪威知识驱动机器学习中心);University of Oslo(奥斯陆大学)
Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Department of Computer Science, National University of Singapore(新加坡国立大学计算机科学系);Ant Group(蚂蚁集团)
SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarios
Center for Chinese Linguistics, Department of Chinese Language and Literature, Peking University(北京大学中文语言文学系中文语言学中心);School of Computer Science, State Key Laboratory of Multimedia Information Processing, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);Huawei Noah’s Ark Lab, China(华为诺亚实验室,中国)
Latent-GRPO: Group Relative Policy Optimization for Latent Reasoning
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全国家重点实验室,计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Tailored Teaching with Balanced Difficulty: Elevating Reasoning in Multimodal Chain-of-Thought via Prompt Curriculum
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Hunan Vanguard Group Corporation Co., Ltd.(湖南 Vanguard集团有限公司);Huaneng Information Technology Co., Ltd.(华能信息技术有限公司);Tsinghua University(清华大学);Zhejiang University(浙江大学);PowerChina Huadong Engineering Co., Ltd.(中国电建华东工程有限公司)