VisuCraft：通过结构化信息提取增强大型视觉-语言模型在复杂视觉引导创意内容生成中的能力
VisuCraft: Enhancing Large Vision-Language Models for Complex Visual-Guided Creative Content Generation via Structured Information Extraction
AI总结：
VisuCraft通过结构化信息提取与动态提示生成，增强大型视觉-语言模型在复杂视觉引导创意内容生成中的视觉保真度、创造力与指令遵循能力。
摘要：
本文介绍了VisuCraft，一个旨在显著增强大型视觉-语言模型（LVLMs）在复杂视觉引导创意内容生成中能力的新型框架。现有的LVLMs在生成长文本时，往往在保持高视觉保真度、真正创造力和精确遵循细微用户指令方面表现出局限性。VisuCraft通过集成多模态结构化信息提取器（E）和动态提示生成模块（G）来解决这些挑战。提取器将输入图像中的细粒度视觉属性提炼为丰富的结构化表示，动态提示模块随后将该表示与用户指令结合，为底层LVLMs（如LLaVA、InstructBLIP）生成高度优化的提示。在自构建的ImageStoryGen-500K数据集上，使用VisuGen Metrics（视觉接地、创造力和指令遵循）进行评估，VisuCraft在故事生成和诗歌创作等任务上始终优于基线LVLMs。我们的结果展示了显著的改进，特别是在创造力和指令遵循方面，验证了VisuCraft在生成富有想象力、视觉接地且与用户对齐的长篇创意文本方面的有效性。这项工作为LVLMs在复杂创意AI应用中解锁了新的潜力。
英文摘要：
This paper introduces VisuCraft, a novel framework designed to significantly enhance the capabilities of Large Vision-Language Models (LVLMs) in complex visual-guided creative content generation. Existing LVLMs often exhibit limitations in maintaining high visual fidelity, genuine creativity, and precise adherence to nuanced user instructions when generating long-form texts. VisuCraft addresses these challenges by integrating a multimodal structured information extractor (E) and a dynamic prompt generation module (G). The extractor distills fine-grained visual attributes from input images into a rich, structured representation, which the dynamic prompt module then combines with user instructions to create highly optimized prompts for underlying LVLMs (e.g., LLaVA, InstructBLIP). Evaluated on the self-constructed ImageStoryGen-500K dataset using VisuGen Metrics (Visual Grounding, Creativity, and Instruction Adherence), VisuCraft consistently outperforms baseline LVLMs across tasks like story generation and poetry composition. Our results demonstrate remarkable improvements, particularly in creativity and instruction adherence, validating VisuCraft's effectiveness in producing imaginative, visually grounded, and user-aligned long-form creative text. This work unlocks new potential for LVLMs in sophisticated creative AI applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
