Standard Large Language Models (LLMs) are predominantly designed for static inference with pre-defined inputs, which limits their applicability in dynamic, real-time scenarios. To address this gap, the streaming LLM paradigm has emerged. However, existing definitions of streaming LLMs remain fragmented, conflating streaming generation, streaming inputs, and interactive streaming architectures, while a systematic taxonomy is still lacking. This paper provides a comprehensive overview and analysis of streaming LLMs. First, we establish a unified definition of streaming LLMs based on data flow and dynamic interaction to clarify existing ambiguities. Building on this definition, we propose a systematic taxonomy of current streaming LLMs and conduct an in-depth discussion on their underlying methodologies. Furthermore, we explore the applications of streaming LLMs in real-world scenarios and outline promising research directions to support ongoing advances in streaming intelligence. We maintain a continuously updated repository of relevant papers at https://github.com/EIT-NLP/Awesome-Streaming-LLMs.
Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
Meituan Inc.(美团公司);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所)
StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios
School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Innovation Institute(上海创新研究院);Shanghai University of Finance and Economics(上海财经大学);MOE laboratory for National Development and Intelligent Governance, Fudan University(复旦大学国家发展与智能治理实验室);Research Institute of Intelligent Complex Systems, Fudan University(复旦大学智能复杂系统研究院)