Large language model (LLM) inference performance is increasingly bottlenecked by the memory wall. While GPUs continue to scale raw compute throughput, they struggle to deliver scalable performance for memory bandwidth bound workloads. This challenge is amplified by emerging reasoning LLM applications, where long output sequences, low arithmetic intensity, and tight latency constraints demand significantly higher memory bandwidth. As a result, system utilization drops and energy per inference rises, highlighting the need for an optimized system architecture for scalable memory bandwidth.
To address these challenges we present the Reasoning Processing Unit (RPU), a chiplet-based architecture designed to address the challenges of the modern memory wall. RPU introduces: (1) A Capacity-Optimized High-Bandwidth Memory (HBM-CO) that trades capacity for lower energy and cost; (2) a scalable chiplet architecture featuring a bandwidth-first power and area provisioning design; and (3) a decoupled microarchitecture that separates memory, compute, and communication pipelines to sustain high bandwidth utilization. Simulation results show that RPU performs up to 45.3x lower latency and 18.6x higher throughput over an H100 system at ISO-TDP on Llama3-405B.
NUMCoT: Numerals and Units of Measurement in Chain-of-Thought Reasoning using Large Language Models
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Nanjing University of Posts and Telecommunications（南京邮电大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Video question answering is a challenging task, which requires agents to be able to understand rich video contents and perform spatial-temporal reasoning. However, existing graph-based methods fail to perform multi-step reasoning well, neglecting two properties of VideoQA: (1) Even for the same video, different questions may require different amount of video clips or objects to infer the answer with relational reasoning; (2) During reasoning, appearance and motion features have complicated interdependence which are correlated and complementary to each other. Based on these observations, we pro…
IIU: Independent Inference Units for Knowledge-based Visual Question Answering
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；School of Cyberspace Science and Technology, Beijing Institute of Technology（北京理工大学网络空间科学与技术学院）
UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations
School of Computer Science and Technology, University of Chinese Academy of Sciences (UCAS)（中国科学院大学计算机科学与技术学院）；Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS)（中国科学院计算技术研究所）；Beijing Academy of Artificial Intelligence (BAAI)（北京智源人工智能研究院）；School of Artificial Intelligence and Robotics, Hunan University（湖南大学人工智能与机器人学院）
SPARC: Scenario Planning and Reasoning for Automated C Unit Test Generation
Bangladesh University of Engineering and Technology（孟加拉工程科技大学）；National Taiwan University（台湾国立大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Rationale-Grounded In-Context Learning for Time Series Reasoning with Multimodal Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；China Mobile (Jiangxi) Virtual Reality Technology Co., Ltd.（中国移动（江西）虚拟现实技术有限公司）；School of Computer and Information Technology, Beijing Jiaotong University（北京交通大学计算机与信息学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Multi-hop question answering requires a model to connect multiple pieces of evidence scattered in a long context to answer the question. In this paper, we show that in the multi-hop HotpotQA (Yang et al., 2018) dataset, the examples often contain reasoning shortcuts through which models can directly locate the answer by word-matching the question with a sentence in the context. We demonstrate this issue by constructing adversarial documents that create contradicting answers to the shortcut but do not affect the validity of the original answer. The performance of strong baseline models drops si…
Knowledge Capsules: Structured Nonparametric Memory Units for LLMs
Zhejiang Angel Medical AI Technology Co., Ltd.（浙江天使医疗人工智能科技有限公司）；Miti AI Technology Co., Ltd.（米蒂人工智能科技有限公司）；China-Singapore Belt and Road Joint Laboratory on Translational Infection Biology for Diagnostics and Therapies（中英一带一路联合实验室（转化感染生物学诊断与治疗））；State Key Laboratory for Diagnosis and Treatment of Infectious Diseases（传染病诊断与治疗国家重点实验室）；The First Affiliated Hospital（第一附属医院）
Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteria
POSTECH；Seoul National University（首尔国立大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Kyungpook National University（庆北国立大学）；LG AI Research（LG AI研究院）
School of Computing and Information Sciences Florida International University（佛罗里达国际大学计算与信息科学学院）；IBM Research（IBM研究院）
Existing commonsense reasoning datasets for AI and NLP tasks fail to address an important aspect of human life: cultural differences. We introduce an approach that extends prior work on crowdsourcing commonsense knowledge by incorporating differences in knowledge that are attributable to cultural or national groups. We demonstrate the technique by collecting commonsense knowledge that surrounds six fairly universal rituals -- birth, coming-of-age, marriage, funerals, new year, and birthdays -- across two national groups: the United States and India. Our study expands the different types of rel…
Probabilistic reasoning is an essential tool for robust decision-making systems because of its ability to explicitly handle real-world uncertainty, constraints and causal relations. Consequently, researchers are developing hybrid models by combining Deep Learning with probabilistic reasoning for safety-critical applications like self-driving vehicles, autonomous drones, etc. However, probabilistic reasoning kernels do not execute efficiently on CPUs or GPUs. This paper, therefore, proposes a custom programmable processor to accelerate sum-product networks, an important probabilistic reasoning …
