Vinci：基于第一视角视觉语言模型的实时具身智能助手
Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model
AI总结：
本文提出 Vinci，一个基于第一视角视觉语言模型、可部署于便携设备的实时具身智能助手，支持始终在线观察、自然对话、长视频流理解、任务规划及分步视频演示生成。
摘要：
我们介绍了 Vinci，一个基于第一视角视觉语言模型构建的实时具身智能助手。Vinci 专为部署在智能手机和可穿戴相机等便携设备上而设计，以“始终开启”模式运行，持续观察环境以提供无缝交互与协助。用户可以唤醒系统并进行自然对话来提问或寻求帮助，响应通过音频传递，实现免提便利。凭借其实时处理长视频流的能力，Vinci 能够回答用户关于当前观察和历史上下文的查询，同时还能基于过去的交互提供任务规划。为了进一步增强可用性，Vinci 集成了一个视频生成模块，可为需要详细指导的任务创建分步视觉演示。我们希望 Vinci 能够为便携式、实时的第一视角 AI 系统建立一个稳健的框架，为用户提供情境化和可操作的洞察。我们发布了用于开发该设备的完整实现，并配套一个演示网络平台，用于在 https://github.com/OpenGVLab/vinci 上测试上传的视频。
英文摘要：
We introduce Vinci, a real-time embodied smart assistant built upon an egocentric vision-language model. Designed for deployment on portable devices such as smartphones and wearable cameras, Vinci operates in an "always on" mode, continuously observing the environment to deliver seamless interaction and assistance. Users can wake up the system and engage in natural conversations to ask questions or seek assistance, with responses delivered through audio for hands-free convenience. With its ability to process long video streams in real-time, Vinci can answer user queries about current observations and historical context while also providing task planning based on past interactions. To further enhance usability, Vinci integrates a video generation module that creates step-by-step visual demonstrations for tasks that require detailed guidance. We hope that Vinci can establish a robust framework for portable, real-time egocentric AI systems, empowering users with contextual and actionable insights. We release the complete implementation for the development of the device in conjunction with a demo web platform to test uploaded videos at https://github.com/OpenGVLab/vinci.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
