arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.22013cs.CVcs.GRcs.LG
收藏

PointLLM-R: 通过链式推理增强3D点云推理

PointLLM-R: Enhancing 3D Point Cloud Reasoning via Chain-of-Thought

Visual Computing Research Center (VCC), College of Computer Science(视觉计算研究中心(VCC),计算机科学学院);Software Engineering (CSSE) Shenzhen University China(软件工程(CSSE)深圳大学中国);VCC, CSSE Shenzhen University China(VCC,CSSE 深圳大学中国);Shenzhen University(深圳大学)

Chaoqi Chen, Qile Xu, Wenjun Zhou, Hui Huang

AI总结:

本文提出了一种数据驱动的框架,用于构建大规模链式推理监督,以改进3D点云理解。通过两阶段流程优化点文本指令数据,并合成高质量推理路径,构建了包含55K样本的PoCoTI数据集,训练PointLLM-R实现3D多模态语言模型的推理能力,实验表明其在生成3D分类和描述任务中达到最先进的性能。

摘要:

通过语言理解3D点云仍然是计算机图形学和视觉计算中的基本挑战,由于点云数据的不规则结构和现有3D多模态模型中缺乏显式推理。尽管链式推理(CoT)在LLM和基于图像的MLLM中表现出强大的有效性,但其在3D理解中的扩展仍鲜有探索。本文提出了一种数据驱动的框架,用于构建大规模CoT监督,专门针对3D点云理解。我们的框架由一个两阶段流程组成,首先通过基于视觉语言模型的质量评估和参考引导细化点文本指令数据,然后通过人机协同提示优化(HiLPO)合成高质量的推理路径。使用这种方法,我们构建了PoCoTI,一个包含55K样本的CoT增强点文本指令遵循数据集。在PoCoTI上微调PointLLM,得到PointLLM-R,一个具备推理能力的3D多模态语言模型。在生成3D分类和描述任务上的大量实验表明,PointLLM-R在生成3D分类和描述任务中达到了最先进的性能,并且能够稳健地推广到现实世界扫描点云和多轮对话场景中。

英文摘要:

Understanding 3D point clouds through language remains a fundamental challenge in computer graphics and visual computing, due to the irregular structure of point cloud data and the lack of explicit reasoning in existing 3D multimodal models. While Chain-of-Thought (CoT) reasoning has shown strong effectiveness in LLMs and image-based MLLMs, its extension to 3D understanding remains largely underexplored. In this paper, we propose a data-centric framework for constructing large-scale CoT supervision tailored to 3D point cloud understanding. Our framework consists of a two-stage pipeline that first refines point-text instruction data via vision-language-model-based quality evaluation and reference-guided refinement, and then synthesizes high-quality reasoning paths through Human-in-the-Loop Prompt Optimization (HiLPO). Using this approach, we build PoCoTI, a CoT-enhanced point-text instruction-following dataset containing 55K samples with explicit reasoning paths. Fine-tuning PointLLM on PoCoTI yields PointLLM-R, a reasoning-capable 3D multimodal language model. Extensive experiments on generative 3D classification and captioning demonstrate that PointLLM-R achieves state-of-the-art performance and generalizes robustly to real-world scanned point clouds and multi-turn dialogue scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑