连续提示：基于大语言模型的流处理管道处理
Continuous Prompts: LLM-Augmented Pipeline Processing over Unstructured Streams
AI总结：
本文提出连续提示框架，通过引入大语言模型的持续推理能力，实现对无结构流的持久语义处理，并通过动态优化提升效率与精度的平衡。
摘要：
对无结构流进行监控日益需要持久且具有语义的计算，但目前的大语言模型框架仍然是无状态且一次性使用的，限制了它们在长时间分析中的实用性。我们引入了连续提示（CPs），这是首个将大语言模型推理引入连续流处理的框架。CPs扩展了RAG到流式设置，定义了连续语义运算符，并提供了多种实现，主要侧重于基于大语言模型的方法，但也报告了一种基于嵌入的变体。此外，我们研究了两种大语言模型中心的优化，元组批处理和运算符融合，以显著提高效率，同时管理精度损失。由于这些优化本质上是用速度换取精度，我们提出了一个动态优化框架，使用轻量级阴影执行和成本感知的多目标贝叶斯优化（MOBO）来学习吞吐量-精度前沿，并在探测预算下适应计划。我们将在VectraFlow流处理系统中实现CPs。通过运算符级别的微基准测试和真实数据集上的流式管道，我们展示了VectraFlow能够适应工作负载动态，导航精度-效率权衡，并在演变的无结构流上维持持久的语义查询。
英文摘要：
Monitoring unstructured streams increasingly requires persistent, semantics-aware computation, yet today's LLM frameworks remain stateless and one-shot, limiting their usefulness for long-running analytics. We introduce Continuous Prompts (CPs), the first framework that brings LLM reasoning into continuous stream processing. CPs extend RAG to streaming settings, define continuous semantic operators, and provide multiple implementations, primarily focusing on LLM-based approaches but also reporting one embedding-based variants. Furthermore, we study two LLM-centric optimizations, tuple batching and operator fusion, to significantly improve efficiency while managing accuracy loss. Because these optimizations inherently trade accuracy for speed, we present a dynamic optimization framework that uses lightweight shadow executions and cost-aware multi-objective Bayesian optimization (MOBO) to learn throughput-accuracy frontiers and adapt plans under probing budgets. We implement CPs in the VectraFlow stream processing system. Using operator-level microbenchmarks and streaming pipelines on real datasets, we show that VectraFlow can adapt to workload dynamics, navigate accuracy-efficiency trade-offs, and sustain persistent semantic queries over evolving unstructured streams.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
