MVLLaVA: 面向统一灵活新视角合成的智能体
MVLLaVA: An Intelligent Agent for Unified and Flexible Novel View Synthesis
AI总结：
本研究提出集成多视角扩散模型与LLaVA的智能体MVLLaVA，通过任务指令模板微调LLaVA，可基于多种输入和语言指令灵活完成新视角合成，经实验验证性能稳健、通用性强。
摘要：
本文提出MVLLaVA，一款专为新视角合成任务设计的智能体。MVLLaVA将多个多视角扩散模型与大型多模态模型LLaVA集成，可高效处理广泛任务。MVLLaVA是一个通用统一平台，可适配多种输入类型，包括单张图像、描述性文本提示、或特定的方位角变化，通过语言指令引导生成视角。我们精心设计了任务专属的指令模板，随后用于微调LLaVA。最终，MVLLaVA具备了根据用户指令生成新视角图像的能力，展现出在多样任务中的灵活性。我们开展了实验验证MVLLaVA的有效性，结果表明其在应对各类新视角合成挑战时性能稳健、通用性强。
英文摘要：
This paper introduces MVLLaVA, an intelligent agent designed for novel view synthesis tasks. MVLLaVA integrates multiple multi-view diffusion models with a large multimodal model, LLaVA, enabling it to handle a wide range of tasks efficiently. MVLLaVA represents a versatile and unified platform that adapts to diverse input types, including a single image, a descriptive caption, or a specific change in viewing azimuth, guided by language instructions for viewpoint generation. We carefully craft task-specific instruction templates, which are subsequently used to fine-tune LLaVA. As a result, MVLLaVA acquires the capability to generate novel view images based on user instructions, demonstrating its flexibility across diverse tasks. Experiments are conducted to validate the effectiveness of MVLLaVA, demonstrating its robust performance and versatility in tackling diverse novel view synthesis challenges.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
