Hybrid-RACA：用于实时文本预测的混合检索增强创作辅助
Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Prediction
AI总结：
本文提出Hybrid-RACA系统，通过检索增强记忆结合云端大模型与客户端小模型，并采用异步记忆更新机制，在五个数据集上实现了低延迟的高性能实时文本预测。
摘要：
经检索增强的大型语言模型（LLM）在许多应用中表现出色。然而，这些模型的计算需求在将其应用于实时任务（如创作辅助）时带来了挑战。为此，我们提出了混合检索增强创作辅助（Hybrid-RACA），这是一种用于实时文本预测的新系统，它通过检索增强记忆，将基于云的LLM与较小的客户端模型高效结合。这种整合使客户端模型能够借助LLM的能力和云端数据生成更好的响应。同时，通过一种新颖的异步记忆更新机制，客户端模型无需等待云端的响应，即可为用户输入提供实时补全。我们在五个数据集上的实验表明，Hybrid-RACA在保持低延迟的同时提供了强大的性能。
英文摘要：
Large language models (LLMs) enhanced with retrieval augmentation has shown great performance in many applications. However, the computational demands for these models pose a challenge when applying them to real-time tasks, such as composition assistance. To address this, we propose Hybrid Retrieval-Augmented Composition Assistance (Hybrid-RACA), a novel system for real-time text prediction that efficiently combines a cloud-based LLM with a smaller client-side model through retrieval augmented memory. This integration enables the client model to generate better responses, benefiting from the LLM's capabilities and cloud-based data. Meanwhile, via a novel asynchronous memory update mechanism, the client model can deliver real-time completions to user inputs without the need to wait for responses from the cloud. Our experiments on five datasets demonstrate that Hybrid-RACA offers strong performance while maintaining low latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
