arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.06082cs.HC
收藏

MemoVis:一款由生成式AI驱动的、用于为3D设计反馈创建配套参考图像的工具

MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback

Chen Chen, Cuong Nguyen, Thibault Groueix, Vladimir G. Kim, Nadir Weibel

AI总结:

针对3D设计反馈提供者缺乏专业技能、难以创建匹配的参考图像的问题,研究推出MemoVis工具,通过实时视角建议和三类图像修改器生成配套参考图,实验验证了其有效性。

摘要:

提供异步反馈是3D设计工作流中的关键步骤。提供反馈的一种常见做法是将文本评论与配套参考图像配对,这有助于阐明文本的核心要点。理想情况下,反馈提供者应具备3D和图像编辑技能,以创建能有效传达自身想法的参考图像。但他们往往缺乏这类技能,因此不得不借助草图或网络图片,而这些内容可能与当前的3D设计匹配度不佳。为解决这一问题,我们推出了MemoVis——一款文本编辑器界面,它能借助由反馈评论驱动的生成式AI,协助反馈提供者创建参考图像。首先,该工具具备一项基于视觉-语言基础模型的新型实时视角建议功能,可帮助反馈提供者将评论与相机视角绑定。其次,在确定相机视角后,我们引入了三种基于预训练2D生成模型的图像修改器,可将文本评论转化为该视角下3D场景的更新版本。我们针对反馈提供者开展了一项被试内研究,验证了MemoVis的有效性。配套图像的质量和明确性则由另外8名具备3D设计经验的参与者进行评估。

英文摘要:

Providing asynchronous feedback is a critical step in the 3D design workflow. A common approach to providing feedback is to pair textual comments with companion reference images, which helps illustrate the gist of text. Ideally, feedback providers should possess 3D and image editing skills to create reference images that can effectively describe what they have in mind. However, they often lack such skills, so they have to resort to sketches or online images which might not match well with the current 3D design. To address this, we introduce MemoVis, a text editor interface that assists feedback providers in creating reference images with generative AI driven by the feedback comments. First, a novel real-time viewpoint suggestion feature, based on a vision-language foundation model, helps feedback providers anchor a comment with a camera viewpoint. Second, given a camera viewpoint, we introduce three types of image modifiers, based on pre-trained 2D generative models, to turn a text comment into an updated version of the 3D scene from that viewpoint. We conducted a within-subjects study with feedback providers, demonstrating the effectiveness of MemoVis. The quality and explicitness of the companion images were evaluated by another eight participants with prior 3D design experience.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑