概念树：一个可控的合成数据框架助力更强的个性化视觉语言模型
Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs
AI总结：
研究提升VLM个性化能力问题，提出概念树（CaT）框架，以树结构表示概念生成多样正负样本，经数据过滤策略确保质量，实验证明该框架显著增强VLM在个性化基准测试中的能力，是首个可控合成数据管道。
摘要：
视觉语言模型（VLM）在多模态任务中表现出色，近期对提升其个性化能力兴趣渐浓。许多方法用正负样本微调模型，但用户提供的正样本稀缺且检索的负样本质量低。本文系统研究正负样本数量和多样性对VLM个性化任务的影响，基于分析引入概念树（CaT）框架，它以树结构表示概念，能生成不同难度和多样性的正负样本，通过数据过滤策略确保数据质量。实验表明，CaT显著提升了VLM在个性化基准测试中的能力，是首个用于VLM个性化的可控合成数据管道。
英文摘要：
Vision-Language Models (VLMs) have demonstrated exceptional performance in various multi-modal tasks. Recently, there has been an increasing interest in improving the personalization capabilities of VLMs. To better integrate user-provided concepts into VLMs, many methods use positive and negative samples to fine-tune these models. However, the scarcity of user-provided positive samples and the low quality of retrieved negative samples pose challenges for existing techniques. To reveal the relationship between sample and model performance, we systematically investigate the amount and diversity impact of positive and negative samples (easy and hard) on VLM personalization tasks. Based on the detailed analysis, we introduce Concept-as-Tree (CaT), which represents a concept as a tree structure, thereby enabling the data generation of positive and negative samples with varying difficulty and diversity, and can be easily extended to multi-concept scenarios. With a well-designed data filtering strategy, our CaT framework can ensure the quality of generated data, constituting a powerful pipeline. We perform thorough experiments with various VLM personalization baselines to assess the effectiveness of the pipeline, alleviating the lack of positive samples and the low quality of negative samples. Our results demonstrate that CaT equipped with the proposed data filter significantly enhances the capabilities of VLMs across personalization benchmarks. To the best of our knowledge, this work is the first controllable synthetic data pipeline for VLM personalization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
