AI 中文总结
ES3D是将语义嵌入3D空间的框架，通过多视图语义特征构建3D语义嵌入，实现组件感知的3D资产检索与编辑，可生成几何一致、语义连贯的编辑结果。
AI 中文摘要
现有的3D编辑方法在可控性方面已取得显著进展，但仍存在若干重要局限。多数方法依赖文本驱动的编辑，难以表达用户期望的细粒度视觉变化；此外，许多方法需要手动提供3D掩码，或对应保持不变的区域引入非预期改动。这些局限主要源于缺乏细粒度语义理解，导致现有模型难以检索或修改特定3D组件。本文提出ES3D框架，其将语义直接嵌入3D空间，支持基于多幅局部参考图像和可选文本查询的3D资产组件感知检索与编辑。具体而言，首先通过将多视图语义特征投影到资产的体素化空间，构建3D语义嵌入；随后，通过计算3D语义嵌入与图像或文本查询的语义嵌入之间的特征相似度，执行3D组件检索；编辑阶段则采用带有修复机制的预训练3D生成模型，在用户提供的图像引导下修改检索到的组件，同时保留资产其余部分。总体而言，ES3D是一种基于语义线索检索可编辑区域、以多幅图像为条件的3D编辑框架。大量实验表明，ES3D可生成几何一致、语义连贯的编辑结果，为3D编辑提供鲁棒的基于图像和文本辅助的控制能力。
英文摘要
Existing 3D editing methods have made notable progress in controllability, yet they remain limited in several important ways. Most approaches rely on text-driven editing, which struggles to express fine-grained visual changes intended by the user. Moreover, many methods require manually supplied 3D masks or introduce unintended changes to regions that should remain untouched. These limitations largely arise from the absence of fine-grained semantic understanding, making it difficult for existing models to retrieve or modify specific 3D components. We introduce ES3D, a framework that embeds semantics directly into 3D space, enabling component-aware retrieval and editing of a 3D asset conditioned on multiple local reference images and optional text queries. We first construct a 3D semantic embedding by projecting multi-view semantic features into the voxelized space of the asset. We then perform 3D component retrieval by computing feature similarity between the 3D semantic embedding and the semantic embeddings of image or text queries. For editing, we employ a pretrained 3D generative model with an inpainting mechanism to modify the retrieved components guided by user-provided images while preserving the rest of the asset. Overall, ES3D is a 3D editing framework that retrieves editable regions based on semantic cues and uses multiple images as conditions. Extensive experiments demonstrate that ES3D produces geometrically consistent and semantically coherent edits, enabling robust image-based and text-assisted control for 3D editing.
