arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.08682cs.AIcs.CLcs.CV
收藏

LLM-PCGC:基于大语言模型的点云几何压缩

LLM-PCGC: Large Language Model-based Point Cloud Geometry Compression

Peking University(北京大学)

Yuqi Ye, Wei Gao

AI总结:

针对大语言模型直接应用于点云几何压缩存在的结构理解与模态鸿沟问题,提出LLM-PCGC方法,通过聚类、K树、LoRA等适配技术实现跨模态对齐,首次将LLM用作点云压缩器,性能显著优于G-PCC参考软件及现有最优学习方法。

摘要:

实现高效点云压缩的关键在于获取与复杂三维数据结构相匹配的鲁棒上下文模型。近年来,大语言模型(LLM)的发展凸显了其双重能力:既是具备上下文学习与生成能力的强大生成器,也是高效的压缩器。LLM的这一双重属性使其尤其适合满足数据压缩的需求。因此,本文探索了将LLM用于压缩任务的潜力,重点开展了无损点云几何压缩(PCGC)实验。\n然而,将LLM直接应用于PCGC任务存在一些显著挑战:LLM无法很好地理解点云的结构,且通过文本描述填补文本与点云之间的鸿沟是一项艰巨任务,尤其是对于大型复杂点云和小型无定形点云而言。\n为解决这些问题,本文提出了一种全新架构——基于大语言模型的点云几何压缩(LLM-PCGC)方法,该方法无需任何文本描述或对齐操作,即可利用LLM压缩点云几何信息。通过采用多种适配技术实现跨模态表征对齐与语义一致性,包括聚类、K树、令牌映射不变性以及低秩适配(LoRA),所提方法可将LLM转化为点云的压缩器/生成器。据我们所知,这是首个将LLM用作点云数据压缩器的架构。\n实验结果表明,LLM-PCGC的性能显著优于现有其他方法:与MPEG基于几何的点云压缩(G-PCC)标准的参考软件相比,其码率降低了40.213%;与当前最优的基于学习的方法相比,其码率降低了2.267%。

英文摘要:

The key to effective point cloud compression is to obtain a robust context model consistent with complex 3D data structures. Recently, the advancement of large language models (LLMs) has highlighted their capabilities not only as powerful generators for in-context learning and generation but also as effective compressors. These dual attributes of LLMs make them particularly well-suited to meet the demands of data compression. Therefore, this paper explores the potential of using LLM for compression tasks, focusing on lossless point cloud geometry compression (PCGC) experiments. However, applying LLM directly to PCGC tasks presents some significant challenges, i.e., LLM does not understand the structure of the point cloud well, and it is a difficult task to fill the gap between text and point cloud through text description, especially for large complicated and small shapeless point clouds. To address these problems, we introduce a novel architecture, namely the Large Language Model-based Point Cloud Geometry Compression (LLM-PCGC) method, using LLM to compress point cloud geometry information without any text description or aligning operation. By utilizing different adaptation techniques for cross-modality representation alignment and semantic consistency, including clustering, K-tree, token mapping invariance, and Low Rank Adaptation (LoRA), the proposed method can translate LLM to a compressor/generator for point cloud. To the best of our knowledge, this is the first structure to employ LLM as a compressor for point cloud data. Experiments demonstrate that the LLM-PCGC outperforms the other existing methods significantly, by achieving -40.213% bit rate reduction compared to the reference software of MPEG Geometry-based Point Cloud Compression (G-PCC) standard, and by achieving -2.267% bit rate reduction compared to the state-of-the-art learning-based method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑