ScribbleSense: 基于生成的涂鸦纹理编辑与意图预测
ScribbleSense: Generative Scribble-Based Texture Editing with Intent Prediction
AI总结：
ScribbleSense通过结合多模态大语言模型和图像生成模型，实现了基于生成的涂鸦纹理编辑与意图预测，提升交互式编辑性能。
摘要：
交互式3D模型纹理编辑为创建3D资产提供了增强的机会，自由手绘风格提供了最直观的体验。然而，现有方法主要支持基于草图的交互以勾勒轮廓，而基于粗粒度涂鸦交互的利用仍然有限。此外，当前方法往往由于涂鸦指令的抽象性质而遇到挑战，这可能导致编辑意图模糊和目标语义位置不明确。为了解决这些问题，我们提出了ScribbleSense，一种结合多模态大语言模型（MLLMs）和图像生成模型的编辑方法，以有效解决这些挑战。我们利用MLLMs的视觉能力来预测涂鸦背后的编辑意图。一旦确定了涂鸦的语义意图，我们便利用全局生成的图像来提取局部纹理细节，从而锚定局部语义并缓解关于目标语义位置的模糊性。实验结果表明，我们的方法有效利用了MLLMs的优势，实现了对于基于涂鸦的纹理编辑的最先进的交互编辑性能。
英文摘要：
Interactive 3D model texture editing presents enhanced opportunities for creating 3D assets, with freehand drawing style offering the most intuitive experience. However, existing methods primarily support sketch-based interactions for outlining, while the utilization of coarse-grained scribble-based interaction remains limited. Furthermore, current methodologies often encounter challenges due to the abstract nature of scribble instructions, which can result in ambiguous editing intentions and unclear target semantic locations. To address these issues, we propose ScribbleSense, an editing method that combines multimodal large language models (MLLMs) and image generation models to effectively resolve these challenges. We leverage the visual capabilities of MLLMs to predict the editing intent behind the scribbles. Once the semantic intent of the scribble is discerned, we employ globally generated images to extract local texture details, thereby anchoring local semantics and alleviating ambiguities concerning the target semantic locations. Experimental results indicate that our method effectively leverages the strengths of MLLMs, achieving state-of-the-art interactive editing performance for scribble-based texture editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
