During the deployment of Large Language Models (LLMs), the autoregressive decoding phase on heterogeneous NPU platforms (e.g., Ascend 910B) faces severe memory-bound challenges. This study reveals the ``Model Scaling Paradox'' caused by the static deployment of single-sized models. It also points out the kernel synchronization overhead of fine-grained speculative decoding \cite{leviathan2023fast, chen2023speculative} under NPU computational graph compilation, and the severe limitations of purely relying on micro-level acceleration algorithms like Prompt LookUp Decoding (PLD)
STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU
AMD Research and Advanced Development (RAD)（AMD研究与先进开发）；Integrated Systems Laboratory (IIS)（集成系统实验室）；ETH Zürich（苏黎世联邦理工学院）；Department of Electrical, Electronic and Information Engineering (DEI)（电气电子信息工程系）；University of Bologna（博洛尼亚大学）
Scaling LLM Test-Time Compute with Mobile NPU on Smartphones
Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究院（AIR），清华大学）
To satisfy the compute and memory demands of deep neural networks, neural processing units (NPUs) are widely being utilized for accelerating deep learning algorithms. Similar to how GPUs have evolved from a slave device into a mainstream processor architecture, it is likely that NPUs will become first class citizens in this fast-evolving heterogeneous architecture space. This paper makes a case for enabling address translation in NPUs to decouple the virtual and physical memory address space. Through a careful data-driven application characterization study, we root-cause several limitations of…
FrankenBot: Brain-Morphic Modular Orchestration for Robotic Manipulation with Vision-Language Models
School of Future Technology, South China University of Technology（未来技术学院，华南理工大学）；School of Software Engineering, South China University of Technology（软件工程学院，华南理工大学）；Shien-Ming Wu School of Intelligent Engineering, South China University of Technology（智能工程学院，华南理工大学）
Since very few contributions to the development of an unified memory orchestration framework for efficient management of both host and remote idle memory have been made, we present Valet, an efficient approach to orchestration of host and remote shared memory for improving performance of memory intensive workloads. The paper makes three original contributions. First, we redesign the data flow in the critical path by introducing a host-coordinated memory pool that works as a local cache to reduce the latency in the critical path of the host and remote memory orchestration. Second, Valet utilize…
Danus: Orchestrating Mathematical Reasoning Agents with Fact-Graph Memory
School of Mathematical Sciences, Peking University（北京大学数学科学学院）；Beijing International Center for Mathematical Research, Peking University（北京大学北京国际数学研究中心）；Research Institute for Mathematical Sciences, Kyoto University（京都大学数理解析研究所）；School of Mathematics, Tianjin University（天津大学数学学院）；Zhongguancun Academy（中关村科学院）；Department of Mathematics, Stanford University（斯坦福大学数学系）；Westlake Institute for Advanced Study, Westlake University（西湖大学西湖高等研究院）；School of Mathematical Sciences, Key Laboratory of Intelligent Computing and Applications (Ministry of Education), Tongji University（同济大学数学科学学院（教育部智能计算与应用重点实验室））；Beijing International Center for Mathematical Research and the New Cornerstone Science Laboratory, Peking University（北京大学北京国际数学研究中心与新基石科学实验室）；Center for Machine Learning Research, Peking University（北京大学机器学习研究中心）；Center for Intelligent Computing, Great Bay Institute for Advanced Study, Great Bay University（大湾区大学高等研究院智能计算中心）
TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；School of Artificial Intelligence and Data Science, University of Science and Technology of China（中国科学技术大学人工智能与数据科学学院）；Laoshan Laboratory（崂山实验室）；Ocean University of China（中国海洋大学）
FlexNPU: Transparent NPU Virtualization for Dynamic LLM Prefill-Decode Co-location
Huawei Technologies Co., Ltd（华为技术有限公司）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen)（广东省人工智能与数字经济实验室（深圳））；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）
HeRo: Adaptive Orchestration of Agentic RAG on Heterogeneous Mobile SoC
School of Computer Science, Peking University（北京大学计算机学院）；School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；School of Integrated Circuit Science and Engineering, Beihang University（北京航空航天大学集成电路科学与工程学院）
