ProductWebGen: 多模态产品网页生成基准测试
ProductWebGen: Benchmarking Multimodal Product Webpage Generation
AI总结：
提出ProductWebGen基准，用于评估多模态生成模型从产品图像和指令生成一致产品展示网页的能力，并比较了基于编辑和基于统一模型两种工作流。
摘要：
从源产品图像以及布局和视觉内容指令中制作产品展示网页，对于营销、广告和电子商务等领域具有重要的实用价值。直观上，该任务要求产品展示之间严格的视觉一致性以及高保真度的指令遵循，以联合生成可渲染的HTML代码。这些对可控性和指令遵循的要求与先进多模态生成模型（如图像编辑模型和统一模型）的核心特征紧密一致。为此，本文引入ProductWebGen来系统性地基准测试这些模型的产品网页生成能力。我们组织了包含500个测试样本的ProductWebGen，涵盖13个产品类别；每个样本由源图像、视觉内容指令和网页指令组成。任务是根据源图像和指令生成包含多个一致图像的产品展示网页。鉴于任务的混合模态输入输出性质，我们设计并系统比较了两种评估工作流——一种使用大语言模型和图像编辑模型分别生成HTML代码和图像（基于编辑），另一种依赖单个统一模型生成两者，其中图像生成依赖于先前的多模态上下文（基于统一模型）。实验结果表明，基于编辑的方法在网页指令遵循和内容吸引力方面取得领先结果，而基于统一模型的方法在满足视觉内容指令方面可能展现出更多优势。我们还构建了一个监督微调数据集ProductWebGen-1k，包含1000组真实产品图像和LLM生成的HTML代码。我们在开源统一模型BAGEL上验证了其有效性。数据和代码可在https://github.com/SJTU-DENG-Lab/ProductWebGen获取。
英文摘要：
Crafting a product display webpage from a source product image, along with layout and visual content instructions, holds significant practical value for domains such as marketing, advertising, and E-commerce. Intuitively, this task demands strict visual consistency across product displays and high-fidelity instruction following to jointly generate renderable HTML code. These requirements on controllability and instruction-following are closely aligned with the core features of advanced multimodal generative models, such as image editing models and unified models. To this end, this paper introduces ProductWebGen to systematically benchmark the product webpage generation capacities of these models. We organize ProductWebGen with 500 test samples covering 13 product categories; each sample consists of a source image, a visual content instruction, and a webpage instruction. The task is to generate a product showcase webpage including multiple consistent images in accordance with the source image and instructions. Given the mixed-modality input-output nature of the task, we design and systematically compare two workflows for evaluation -- one uses large language models and image editing models to separately generate HTML code and images (editing-based), while the other relies on a single UM to generate both, with image generation conditioned on the preceding multimodal context (UM-based). Empirical results show that editing-based approaches achieve leading results in webpage instruction following and content appeal, while UM-based ones may display more advantages in fulfilling visual content instructions. We also construct a supervised fine-tuning dataset, ProductWebGen-1k, with 1,000 groups of real product images and LLM-generated HTML code. We verify its effectiveness on the open-source UM BAGEL. The data and code are available at https://github.com/SJTU-DENG-Lab/ProductWebGen.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
