通过用户指令进行情境感知的室内点云物体生成
Context-Aware Indoor Point Cloud Object Generation through User Instructions
AI总结：
针对室内场景修改依赖预设物体库与位置的问题，提出文本驱动的端到端多模态点云生成网络，结合 Point-E、量化位置预测和 Top-K 估计实现情境一致的新物体布局。
摘要：
室内场景修改已成为计算机视觉中的一个突出领域，尤其是在增强现实（AR）和虚拟现实（VR）应用方面。传统方法通常依赖预先存在的物体数据库和预先确定的物体位置，这限制了它们在新场景中的灵活性和适应性。为应对这一挑战，我们提出了一种新颖的端到端多模态深度神经网络，能够在文本指令驱动下生成与周围环境无缝融合的点云物体。我们的模型通过支持创建具有此前未见物体布局的新环境，彻底改变了场景修改方式，无需预先存储的 CAD 模型。我们利用 Point-E 作为生成模型，并引入量化位置预测和 Top-K 估计等创新技术，以解决由模糊语言描述导致的假阴性问题。此外，我们进行了全面评估，以展示生成物体的多样性、文本指令的有效性以及定量指标，证实我们的模型在生成室内物体方面的真实性和多功能性。为提供全面评估，我们将视觉定位作为额外指标纳入，以确保模型所生成场景的质量和连贯性。通过这些进展，我们的方法不仅推进了室内场景修改的最先进水平，也为沉浸式计算和数字环境创建方面的未来创新奠定了基础。
英文摘要：
Indoor scene modification has emerged as a prominent area within computer vision, particularly for its applications in Augmented Reality (AR) and Virtual Reality (VR). Traditional methods often rely on pre-existing object databases and predetermined object positions, limiting their flexibility and adaptability to new scenarios. In response to this challenge, we present a novel end-to-end multi-modal deep neural network capable of generating point cloud objects seamlessly integrated with their surroundings, driven by textual instructions. Our model revolutionizes scene modification by enabling the creation of new environments with previously unseen object layouts, eliminating the need for pre-stored CAD models. Leveraging Point-E as our generative model, we introduce innovative techniques such as quantized position prediction and Top-K estimation to address the issue of false negatives resulting from ambiguous language descriptions. Furthermore, we conduct comprehensive evaluations to showcase the diversity of generated objects, the efficacy of textual instructions, and the quantitative metrics, affirming the realism and versatility of our model in generating indoor objects. To provide a holistic assessment, we incorporate visual grounding as an additional metric, ensuring the quality and coherence of the scenes produced by our model. Through these advancements, our approach not only advances the state-of-the-art in indoor scene modification but also lays the foundation for future innovations in immersive computing and digital environment creation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
