Talk to Me, Jarvis：一个面向自动驾驶赛车的开源可边缘部署语音助手框架
Talk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecars
- Technical University of Munich（慕尼黑工业大学）
- Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出Jarvis，一个离线边缘部署的语音助手框架，通过微调Mistral 7B实现低延迟命令分类，在自动驾驶赛车中达到97.63%意图识别准确率和1.39秒平均延迟，并开源支持进一步研究。
AI中文摘要：
大型语言模型的最新进展提升了其作为语音助手后端组件的有效性，特别是在意图理解和上下文感知输入分类方面。然而，在线托管的模型引入了网络依赖和可变的推理延迟，限制了它们在时间关键的自动驾驶应用中的适用性。在本工作中，我们通过开发Jarvis来解决这些问题，这是一个用于自动驾驶车辆高级行为命令的离线语音助手。其架构将语音识别与合成以及自然语言命令分类集成到一个轻量级的本地框架中。Jarvis的核心组件是一个文本到命令的分类器，通过对Mistral 7B模型进行领域特定的微调构建，展示了低延迟推理。我们的实验评估表明，我们的解决方案优于更大的在线托管模型，实现了97.63%的意图识别准确率和平均1.39秒的处理延迟，非常适合需要快速响应的操作。为了支持进一步的研究和微调，我们提供了开源实现。
英文摘要：
Recent advances in large language models have improved their effectiveness as back-end components for voice assistants, particularly in intent understanding and context-aware input classification. However, online-hosted models introduce network dependency and variable inference latency, limiting their suitability for time-critical autonomous driving applications. In this work, we address these issues by developing Jarvis, an offline voice assistant for high-level behavioral commands of autonomous vehicles. Its architecture integrates speech recognition and synthesis with natural language command classification into a lightweight, local framework. Jarvis core component is a text-to-command classifier, built using a domain-specific fine-tuning of the Mistral 7B model, demonstrating low-latency inference. Our experimental evaluation demonstrates that our solution outperforms larger online-hosted models, achieving 97.63 % intent recognition accuracy with an average processing latency of 1.39 s, making it well-suited for operations requiring quick response times. To support further research and fine-tuning, we provide an open-source implementation.
