ProactiveLLM: 学习流式大语言模型的主动交互
ProactiveLLM: Learning Active Interaction for Streaming Large Language Models
AI总结：
提出ProactiveLLM，利用模型内生状态指导交互决策，通过掩码流式建模和同步特权自蒸馏实现主动交互，减少延迟并保持质量。
摘要：
标准大语言模型（LLMs）遵循先读取后生成的范式，导致不必要的延迟和计算。流式LLMs通过在接收输入的同时生成来缓解这一问题，但仍难以决定何时与流交互。现有方法要么硬编码交互时机，要么依赖昂贵的外部对齐信号，如时间标签、推理轨迹或更强的教师模型。本文提出ProactiveLLM，通过利用模型内生状态来指导交互决策，实现主动交互。模型首先通过两种互补的训练机制学习从部分输入中感知语义充分性：基于掩码的流式建模和同步特权自蒸馏（SPSD）。前者在训练期间对输入应用单调随机掩码，模拟逐步揭示的流式输入，使模型能够从部分输入视角学习局部语义依赖。后者将部分上下文的学生视图与同一演化模型生成的全上下文教师视图对齐，允许特权全上下文证据指导学生在不完整观察下的理解。这些机制共同诱导内生充分性线索，无需外部教师或标注，为多种决策头的即插即用集成提供了通用基础。在文本和语音流式任务上的广泛评估证实，ProactiveLLM在保持质量的同时显著降低了交互延迟，验证了其动态主动交互的能力。代码公开于https://github.com/EIT-NLP/StreamingLLM/tree/main/ProactiveLLM。
英文摘要：
Standard Large Language Models (LLMs) follow a read-then-generate paradigm, causing unnecessary latency and computation. Streaming LLMs alleviate this issue by generating while receiving inputs, but still struggle to decide when to interact with the stream. Existing methods either hard-code interaction timing or rely on costly external alignment signals, such as timing labels, reasoning trajectories, or stronger teachers. In this paper, we propose ProactiveLLM, which achieves active interaction by leveraging the model's endogenous states to guide interaction decisions. The model first learns to perceive semantic sufficiency from partial inputs through two complementary training mechanisms: mask-based streaming modeling and synchronized privileged self-distillation (SPSD). The former applies monotonic random masking to the input during training, simulating progressively revealed streaming inputs and enabling the model to learn local semantic dependencies from partial-input views. The latter aligns the partial-context student view with a full-context teacher view generated by the same evolving model, allowing privileged full-context evidence to guide the student's understanding under incomplete observations. Together, these mechanisms induce endogenous sufficiency cues without requiring external teachers or annotations, providing a versatile foundation for the plug-and-play integration of diverse decision heads. Extensive evaluation across text and speech streaming tasks confirms that ProactiveLLM significantly reduces interaction latency while maintaining quality, validating its capacity for dynamic and active interaction. Code is publicly available at https://github.com/EIT-NLP/StreamingLLM/tree/main/ProactiveLLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
