VIVECaption：一种改进标题质量的分步方法
VIVECaption: A Split Approach to Caption Quality Improvement
AI总结：
VIVECaption通过双面方法改进标题质量，利用分层抽样和模型对齐策略提升图像-标题对齐效果，提供高质量训练数据解决方案。
摘要：
标题质量已成为训练高质量的文本到图像（T2I）和文本到视频（T2V）生成模型的关键瓶颈。尽管视觉语言模型（VLMs）常用于从视觉数据生成标题，但它们存在幻觉、推理能力差和理解不够精细的问题，导致图像-标题对不一致，从而降低下游模型性能。本技术报告介绍了VIVECaption，一种系统性的双面方法来改进标题质量。我们首先建立了全面的标题评估指标分类，区分“通用”和“实例接地”指标，最终目标是展示不同标题质量指标的使用场景和权衡。然后我们使用这种语言描述我们的双面方法来改进标题质量：（1）使用分层抽样创建黄金标准数据集的方法；（2）包含上下文对齐和参数级微调的模型对齐策略，使用SFT。我们在开源模型上展示了我们的方法，重点是结构化的标题格式，以实现更好的解析和下游利用。我们最终证明，在图像标题生成流程中使用微调的字符检测模型可以显著提高整体图像-标题对齐质量。我们的工作回应了企业AI开发中对高质量“素食”训练数据日益增长的需求，为寻求改进标题-图像对齐而无需依赖可能受版权保护的网络抓取内容的团队提供了实用解决方案。
英文摘要：
Caption quality has emerged as a critical bottleneck in training high-quality text-to-image (T2I) and text-to-video (T2V) generative models. While visual language models (VLMs) are commonly deployed to generate captions from visual data, they suffer from hallucinations, poor compositional reasoning, and limited fine-grained understanding, resulting in misaligned image-caption pairs that degrade downstream model performance. This technical report introduces VIVECaption, a systematic two-sided approach to caption quality improvement. We first establish a comprehensive taxonomy of caption evaluation metrics, distinguishing between "universal" and "instance-grounded" metrics, with the ultimate goal of showcasing the use-cases and tradeoffs between different caption quality metrics. We then use this language to describe our two-sided approach to caption quality improvement: (1) a gold-standard dataset creation methodology using stratified sampling and (2) a model alignment strategy encompassing context alignment and parameter-level finetuning using SFT. We demonstrate our methodology on open-source models, focusing on structured caption formats that enable better parsing and downstream utilization. We ultimately show that using a finetuned character detection model in an image captioning pipeline significantly improves holistic image-caption alignment quality. Our work addresses the growing need for high-quality "vegan" training data in enterprise AI development, providing practical solutions for teams seeking to improve caption-image alignment without relying on potentially copyright-protected web-scraped content.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
