ReVision：一个用于隐私保护任务导向视觉指令重写的数据集和基线VLM
ReVision: A Dataset and Baseline VLM for Privacy-Preserving Task-Oriented Visual Instruction Rewriting
AI总结：
ReVision提出一个数据集和基线VLM，通过将多模态指令转换为纯文本命令，实现轻量级设备端指令重写，提升隐私保护的多模态AI应用能力。
摘要：
高效的、隐私保护的多模态交互对于AR、VR以及现代智能手机等具有强大摄像头的设备成为人类与计算机通信的主要接口至关重要。现有的强大多模态交互大型视觉-语言模型（VLMs）往往依赖于基于云的处理，这引发了两个主要问题：（1）通过传输敏感视觉数据到服务器而泄露视觉隐私；（2）其在设备上的实时可用性有限。本文探讨了视觉指令重写，一种将多模态指令转换为纯文本命令的新方法，允许无缝集成轻量级设备端指令重写VLM（250M参数）与现有对话AI系统，提高视觉数据隐私性。为此，我们提出了一个包含超过39,000个示例的跨14个领域的数据集，并开发了一个紧凑型VLM，预训练在图像描述数据集上，并针对指令重写进行微调。实验结果通过BLEU、METEOR和ROUGE等自然语言生成指标以及语义解析分析评估，证明即使模型的量化版本（<500MB存储足迹）也能实现有效的指令重写，从而实现隐私导向的多模态AI应用。
英文摘要：
Efficient and privacy-preserving multimodal interaction is essential as AR, VR, and modern smartphones with powerful cameras become primary interfaces for human-computer communication. Existing powerful large vision-language models (VLMs) enabling multimodal interaction often rely on cloud-based processing, raising significant concerns about (1) visual privacy by transmitting sensitive vision data to servers, and (2) their limited real-time, on-device usability. This paper explores Visual Instruction Rewriting, a novel approach that transforms multimodal instructions into text-only commands, allowing seamless integration of lightweight on-device instruction rewriter VLMs (250M parameters) with existing conversational AI systems, enhancing vision data privacy. To achieve this, we present a dataset of over 39,000 examples across 14 domains and develop a compact VLM, pretrained on image captioning datasets and fine-tuned for instruction rewriting. Experimental results, evaluated through NLG metrics such as BLEU, METEOR, and ROUGE, along with semantic parsing analysis, demonstrate that even a quantized version of the model (<500MB storage footprint) can achieve effective instruction rewriting, thus enabling privacy-focused, multimodal AI applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
