Small-Large Collaboration：使用元个性化小型 VLM 实现大型 VLM 的高训练效率概念个性化
Small-Large Collaboration: Training-efficient Concept Personalization for Large VLM using a Meta Personalized Small VLM
AI总结：
针对大型 VLM 训练成本高且难以直接个性化的问题，提出 SLC 协作框架，通过训练元个性化小型 VLM 生成信息并结合测试时反思策略，由大型 VLM 整合信息以低成本实现高效个性化。
摘要：
将视觉语言模型（VLM）个性化以将其转化为日常助手已成为一个热门研究方向。然而，OpenAI 等领先公司不断增加模型规模，并开发出思维链（CoT）等复杂设计。尽管大型 VLM 擅长复杂的多模态理解，但其高昂的训练成本和通过付费 API 的有限访问权限限制了直接个性化。相反，小型 VLM 易于个性化且可免费获取，但缺乏足够的推理能力。受此启发，我们提出了一种名为 Small-Large Collaboration（SLC）的新型协作框架，用于大型 VLM 个性化。在该框架中，小型 VLM 负责生成个性化信息，而大型模型则整合这些个性化信息以提供准确响应。为了有效融入个性化信息，我们开发了一种测试时反思策略，以防止小型 VLM 可能产生的幻觉。由于 SLC 只需为大型 VLM 训练一个元个性化小型 VLM，整个过程具有高训练效率。据我们所知，这是首个同时支持开源和闭源大型 VLM 的高训练效率框架，能够实现更广泛的现实世界个性化应用。我们在各种基准和大型 VLM 上进行了全面实验，以证明所提出的 SLC 框架的有效性。代码将发布于 https://github.com/Hhankyangg/SLC。
英文摘要：
Personalizing Vision-Language Models (VLMs) to transform them into daily assistants has emerged as a trending research direction. However, leading companies like OpenAI continue to increase model size and develop complex designs such as the chain of thought (CoT). While large VLMs are proficient in complex multi-modal understanding, their high training costs and limited access via paid APIs restrict direct personalization. Conversely, small VLMs are easily personalized and freely available, but they lack sufficient reasoning capabilities. Inspired by this, we propose a novel collaborative framework named Small-Large Collaboration (SLC) for large VLM personalization, where the small VLM is responsible for generating personalized information, while the large model integrates this personalized information to deliver accurate responses. To effectively incorporate personalized information, we develop a test-time reflection strategy, preventing the potential hallucination of the small VLM. Since SLC only needs to train a meta personalized small VLM for the large VLMs, the overall process is training-efficient. To the best of our knowledge, this is the first training-efficient framework that supports both open-source and closed-source large VLMs, enabling broader real-world personalized applications. We conduct thorough experiments across various benchmarks and large VLMs to demonstrate the effectiveness of the proposed SLC framework. The code will be released at https://github.com/Hhankyangg/SLC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
