arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.03327cs.CVcs.AIcs.CL
收藏

Uni3D-LLM:用大语言模型统一点云感知、生成与编辑

Uni3D-LLM: Unifying Point Cloud Perception, Generation and Editing with Large Language Models

Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Dalian University of Technology(大连理工大学);Shandong University(山东大学)

Dingning Liu, Xiaoshui Huang, Yuenan Hou, Zhihui Wang, Zhenfei Yin, Yongshun Gong, Peng Gao, Wanli Ouyang

AI总结:

Uni3D-LLM提出统一框架,利用大语言模型在点云场景中集成感知、生成与编辑任务,通过自然语言精确控制对象操作,实验验证了其有效性与实际应用潜力。

摘要:

在本文中,我们介绍了Uni3D-LLM,这是一个统一框架,利用大语言模型(LLM)在点云场景中集成3D感知、生成和编辑任务。该框架使用户能够借助自然语言描述的多样性,轻松地在场景中的指定位置生成和修改对象。Uni3D-LLM利用自然语言的表达力,允许对3D对象的生成和编辑进行精确控制,从而显著增强了操作灵活性和可控性。通过将点云映射到统一表示空间,Uni3D-LLM实现了跨应用功能,使得从3D对象的精确实例化到交互设计的多样化需求等一系列广泛任务能够无缝执行。通过一系列全面的严格实验,Uni3D-LLM在点云的理解、生成和编辑方面的有效性得到了验证。此外,我们评估了集成点云感知模块对生成和编辑过程的影响,证实了我们的方法在实际应用中的巨大潜力。

英文摘要:

In this paper, we introduce Uni3D-LLM, a unified framework that leverages a Large Language Model (LLM) to integrate tasks of 3D perception, generation, and editing within point cloud scenes. This framework empowers users to effortlessly generate and modify objects at specified locations within a scene, guided by the versatility of natural language descriptions. Uni3D-LLM harnesses the expressive power of natural language to allow for precise command over the generation and editing of 3D objects, thereby significantly enhancing operational flexibility and controllability. By mapping point cloud into the unified representation space, Uni3D-LLM achieves cross-application functionality, enabling the seamless execution of a wide array of tasks, ranging from the accurate instantiation of 3D objects to the diverse requirements of interactive design. Through a comprehensive suite of rigorous experiments, the efficacy of Uni3D-LLM in the comprehension, generation, and editing of point cloud has been validated. Additionally, we have assessed the impact of integrating a point cloud perception module on the generation and editing processes, confirming the substantial potential of our approach for practical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑