Router-Suggest: 动态路由用于视觉 grounded 对话中的多模态自动补全
Router-Suggest: Dynamic Routing for Multimodal Auto-Completion in Visually-Grounded Dialogs
AI总结：
Router-Suggest 通过动态路由选择文本模型和 VLMs，提升多模态对话中的自动补全效率和用户满意度。
摘要：
实时多模态自动补全对于数字助手、聊天机器人、设计工具和医疗咨询至关重要，其中用户输入依赖于共享的视觉上下文。我们引入了多模态自动补全（MAC）任务，该任务利用部分输入文本和视觉提示预测即将出现的字符。与传统的纯文本自动补全（TAC）不同，MAC 通过多模态上下文来更好地捕捉用户意图。为了实现这一任务，我们调整 MMDialog 和 ImageChat 创建基准数据集。我们评估了领先的视觉语言模型（VLMs）与强大的文本基线模型，突显了准确性和效率之间的权衡。我们提出了 Router-Suggest，一种路由框架，可以根据对话上下文动态选择文本模型和 VLMs，并提供一个轻量级的变体以适应资源受限的环境。Router-Suggest 在最佳 VLM 上实现了 2.3 倍到 10 倍的加速。一项用户研究显示，VLMs 在用户满意度上显著优于文本模型，特别是在节省用户输入努力和改进多轮对话补全质量方面。这些发现突显了多模态上下文在自动补全中的必要性，从而导致更智能、用户感知的助手。
英文摘要：
Real-time multimodal auto-completion is essential for digital assistants, chatbots, design tools, and healthcare consultations, where user inputs rely on shared visual context. We introduce Multimodal Auto-Completion (MAC), a task that predicts upcoming characters in live chats using partially typed text and visual cues. Unlike traditional text-only auto-completion (TAC), MAC grounds predictions in multimodal context to better capture user intent. To enable this task, we adapt MMDialog and ImageChat to create benchmark datasets. We evaluate leading vision-language models (VLMs) against strong textual baselines, highlighting trade-offs in accuracy and efficiency. We present Router-Suggest, a router framework that dynamically selects between textual models and VLMs based on dialog context, along with a lightweight variant for resource-constrained environments. Router-Suggest achieves a 2.3x to 10x speedup over the best-performing VLM. A user study shows that VLMs significantly excel over textual models on user satisfaction, notably saving user typing effort and improving the quality of completions in multi-turn conversations. These findings underscore the need for multimodal context in auto-completions, leading to smarter, user-aware assistants.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
