The benefit of transformers in large-scale 3D point cloud perception tasks, such as 3D object detection, is limited by their quadratic computation cost when modeling long-range relationships. In contrast, linear RNNs have low computational complexity and are suitable for long-range modeling. Toward this goal, we propose a simple and effective window-based framework built on LInear grOup RNN (i.e., perform linear RNN for grouped features) for accurate 3D object detection, called LION. The key property is to allow sufficient feature interaction in a much larger group than transformer-based methods. However, effectively applying linear group RNN to 3D object detection in highly sparse point clouds is not trivial due to its limitation in handling spatial modeling. To tackle this problem, we simply introduce a 3D spatial feature descriptor and integrate it into the linear group RNN operators to enhance their spatial features rather than blindly increasing the number of scanning orders for voxel features. To further address the challenge in highly sparse point clouds, we propose a 3D voxel generation strategy to densify foreground features thanks to linear group RNN as a natural property of auto-regressive models. Extensive experiments verify the effectiveness of the proposed components and the generalization of our LION on different linear group RNN operators including Mamba, RWKV, and RetNet. Furthermore, it is worth mentioning that our LION-Mamba achieves state-of-the-art on Waymo, nuScenes, Argoverse V2, and ONCE dataset. Last but not least, our method supports kinds of advanced linear RNN operators (e.g., RetNet, RWKV, Mamba, xLSTM and TTT) on small but popular KITTI dataset for a quick experience with our linear RNN-based framework.
3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
Beijing Normal University(北京师范大学);Institute of Geographic Sciences and Natural Resources Research, Chinese Academy of Sciences(中国科学院地理科学与资源研究所);Tsinghua University(清华大学)
Recurrent Slice Networks for 3D Segmentation of Point Clouds
University of Southern California(南加州大学)
点云是3D数据的有效数据格式。然而,现有的点云3D分割方法要么不对局部依赖\cite{pointnet}建模,要么需要增加计算\cite{kd-net,pointnet2}。这项工作提出了一种新的三维分割框架,RSNet \footnote{Codes are released here https://github.com/qianguih/RSNet},有效地模拟点云的局部结构。RSNet的关键组件是一个轻量级的本地依赖模块。它是一个新的切片池层,递归神经网络(RNN)层和切片解池层的组合。切片池层被设计为将无序点的特征投影到有序的特征向量序列上,以便可以应用传统的端到端学习算法(RNN)。通过在S3DIS \cite{stanford}、ScanNet \cite{scannet}和ShapeNet \cite{shapenet}数据集上的综合实验,验证了RSNet的性能。在其最简单的形式中,RSNets超越了这些基准上所有以前的最先进的方法。并与现有的最先进的方法\cite{pointnet, pointnet2}进行了比较,证明了RSNets的有效性。
Enhancing Steering Estimation with Semantic-Aware GNNs
Hybrid Intelligence part of Capgemini Engineering(凯捷工程混合智能部门);Computer Science and Artificial Intelligence Lab, Massachusetts Institute of Technology(麻省理工学院计算机科学与人工智能实验室)
UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs
School of Electronic Information and Communications, Huazhong University of Science and Technology (HUST), Wuhan, China(华中科技大学电子信息与通信学院);School of Software Engineering, Huazhong University of Science and Technology (HUST), Wuhan, China(华中科技大学软件工程学院);Department of Computer Science, The University of Hong Kong, Hong Kong (HKU) SAR, China(香港大学计算机系);Baidu Inc., Beijing, China(百度公司)
Multi-Angle Point Cloud-VAE: Unsupervised Feature Learning for 3D Point Clouds from Multiple Angles by Joint Self-Reconstruction and Half-to-Half Prediction
Tsinghua University(清华大学);University of Maryland, College Park(马里兰大学帕克分校);Beijing National Research Center for Information Science and Technology (BNRist)(北京信息科学与技术国家研究中心(BNRist))
Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);University of Maryland, College Park(马里兰大学帕克分校)
L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology (BNRist)(北京信息科学与技术国家研究中心);University of Maryland(马里兰大学)
Efficient RWKV-based Representation Learning for 3D Point Clouds
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Shenzhen Institute of Research, Nanjing University of Aeronautics and Astronautics(南京航空航天大学深圳研究院);Collaborative Innovation Center of Novel Software Technology and Industrialization(新型软件技术与产业化协同创新中心);Urban Data Science section, Delft University of Technology(代尔夫特理工大学城市数据科学部);Huazhong University of Science and Technology(华中科技大学)
Cloud gap-filling with deep learning for improved grassland monitoring
Operational Unit BEYOND Centre for EO Research and Satellite Remote Sensing, Institute for Astronomy, Astrophysics Space Applications and Remote Sensing (IAASARS), National Observatory of Athens(运营单位BEYOND地球观测研究与卫星遥感中心、天文、天体物理空间应用与遥感研究所(IAASARS)、雅典国家天文台);Image Processing Laboratory (IPL), Parc Científic, Universitat de València(图像处理实验室(IPL)、科学公园、瓦伦西亚大学)