Understanding and reasoning over long contexts is a crucial capability for language models (LMs). Although recent models support increasingly long context windows, their accuracy often deteriorates as input length grows. In practice, models often struggle to keep attention aligned with the most relevant context throughout decoding. In this work, we propose DYSCO, a novel decoding algorithm for improving long-context reasoning. DYSCO leverages retrieval heads--a subset of attention heads specialized for longcontext retrieval--to identify task-relevant tokens at each decoding step and explicitly up-weight them. By doing so, DYSCO dynamically adjusts attention during generation to better utilize relevant context. The method is training-free and can be applied directly to any off-the-shelf LMs. Across multiple instruction-tuned and reasoning models, DYSCO consistently improves performance on challenging long-context reasoning benchmarks, yielding relative gains of up to 25% on MRCR and LongBenchV2 at 128K context length with modest additional compute. Further analysis highlights the importance of both dynamic attention rescaling and retrievalhead guided selection for the effectiveness of the method, while providing interpretability insights into decoding-time attention behavior. Our code is available at https://github.com/princeton-pli/DySCO.
Red Note Hi-Lab(红笔记高研实验室);Shanghai Jiaotong University(上海交通大学);Nanjing University(南京大学);Zhejiang University(浙江大学);Chinese University of Hong Kong(香港中文大学)
T$^\star$: Progressive Block Scaling for Masked Diffusion Language Models Through Trajectory Aware Reinforcement Learning
Shanghai Academy of AI for Science(上海人工智能科学研究院);Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);School of Mathematical Sciences(数学科学学院);Shanghai Jiao Tong University(上海交通大学);Carnegie Mellon University(卡内基梅隆大学)
Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding
DeepMind, London, UK(深度思维公司,伦敦,英国);Google Research, New York, NY, USA(谷歌研究院,纽约,纽约州,美国);Stanford University, Stanford, CA, USA(斯坦福大学,斯坦福,加利福尼亚州,美国);University of Toronto, Toronto, Ontario, Canada(多伦多大学,多伦多,安大略省,加拿大);University of Washington, Seattle, WA, USA(华盛顿大学,西雅图,华盛顿州,美国)
Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models
LARG, Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(LARG,社会计算与交互机器人研究中心,哈尔滨工业大学);School of Computer Science and Engineering, Central South University(计算机科学与工程学院,中南大学);The University of Hong Kong(香港大学);Shanghai Jiao Tong University(上海交通大学);ByteDance Seed (China)(字节跳动种子(中国))
Language Ranker: A Lightweight Ranking framework for LLM Decoding
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(一般人工智能国家重点实验室,智能科学与技术学院,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学);MIT CSAIL, MA, USA(MIT CSAIL,马萨诸塞州,美国);Microsoft Research Asia(微软亚洲研究院)
Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
The Hong Kong Polytechnic University, Hong Kong, China(香港理工大学);ELLIS Institute Tübingen, Tübingen, Germany(图宾根ELLIS研究所);Max Planck Institute for Intelligent Systems, Tübingen, Germany(智能系统马克斯·普朗克研究所);Tübingen AI Center, Tübingen, Germany(图宾根人工智能中心);University of Surrey, Guildford, United Kingdom(萨里大学);The University of North Carolina at Chapel Hill, Chapel Hill, NC, USA(北卡罗来纳大学教堂山分校)
Rep2Text: Decoding Full Text from a Single LLM Token Representation
New Jersey Institute of Technology(新泽西理工学院);National University of Singapore(国立新加坡大学);Cisco Research(思科研究);Wake Forest University(威克森林大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))