arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.03561cs.CLcs.AIcs.LG
收藏

Reactive Transformer (RxT)——面向事件驱动反应式语言模型的有状态实时处理

Reactive Transformer (RxT) -- Stateful Real-Time Processing for Event-Driven Reactive Language Models

Adam Filipek

AI总结:

针对传统Transformer无状态、长对话成本与延迟高的问题,提出事件驱动的Reactive Transformer (RxT)架构,通过固定大小短期记忆和生成-记忆更新解耦,将对话成本从二次降为线性,实现低延迟有状态长对话,实验验证其性能更优、延迟恒定。

摘要:

Transformer架构已成为大语言模型(LLM)的事实标准,在语言理解和生成方面展现出卓越能力。然而,其在对话式AI中的应用从根本上受限于无状态特性,以及相对于序列长度$L$的二次计算复杂度($O(L^2)$)。当前模型通过每轮重新处理不断扩展的对话历史来模拟记忆,导致长对话中成本和延迟高得令人难以承受。\n本文提出Reactive Transformer (RxT),这是一种旨在克服这些局限的新型架构,实现了从数据驱动到事件驱动范式的转变。RxT将每轮对话作为离散事件实时处理,在集成的固定大小短期记忆(STM)系统中维护上下文。该架构具有独特的运行周期:生成器-解码器基于当前查询和先前的记忆状态生成响应,随后记忆编码器与专用的Memory Attention网络用完整交互的表示异步更新STM。这一设计从根本上改变了扩展动态,将对话的总用户侧成本从相对于交互次数$N$的二次方($O(N^2 \u00b7 T)$)降至线性($O(N \u00b7 T)$)。通过将响应生成与记忆更新解耦,RxT实现了低延迟,支持真正实时、有状态且经济可行的长时对话。\n我们在合成数据上开展了一系列概念验证实验以验证该架构,结果表明,与规模相当的基线无状态模型相比,RxT性能更优,且推理延迟保持恒定。

英文摘要:

The Transformer architecture has become the de facto standard for Large Language Models (LLMs), demonstrating remarkable capabilities in language understanding and generation. However, its application in conversational AI is fundamentally constrained by its stateless nature and the quadratic computational complexity ($O(L^2)$) with respect to sequence length $L$. Current models emulate memory by reprocessing an ever-expanding conversation history with each turn, leading to prohibitive costs and latency in long dialogues. This paper introduces the Reactive Transformer (RxT), a novel architecture designed to overcome these limitations by shifting from a data-driven to an event-driven paradigm. RxT processes each conversational turn as a discrete event in real-time, maintaining context in an integrated, fixed-size Short-Term Memory (STM) system. The architecture features a distinct operational cycle where a generator-decoder produces a response based on the current query and the previous memory state, after which a memory-encoder and a dedicated Memory Attention network asynchronously update the STM with a representation of the complete interaction. This design fundamentally alters the scaling dynamics, reducing the total user-facing cost of a conversation from quadratic ($O(N^2 \cdot T)$) to linear ($O(N \cdot T)$) with respect to the number of interactions $N$. By decoupling response generation from memory updates, RxT achieves low latency, enabling truly real-time, stateful, and economically viable long-form conversations. We validated our architecture with a series of proof-of-concept experiments on synthetic data, demonstrating superior performance and constant-time inference latency compared to a baseline stateless model of comparable size.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑