Recent success of vision foundation models have shown promising performance for the 2D perception tasks. However, it is difficult to train a 3D foundation network directly due to the limited dataset and it remains under explored whether existing foundation models can be lifted to 3D space seamlessly. In this paper, we present PointSeg, a novel training-free paradigm that leverages off-the-shelf vision foundation models to address 3D scene perception tasks. PointSeg can segment anything in 3D scene by acquiring accurate 3D prompts to align their corresponding pixels across frames. Concretely, we design a two-branch prompts learning structure to construct the 3D point-box prompts pairs, combining with the bidirectional matching strategy for accurate point and proposal prompts generation. Then, we perform the iterative post-refinement adaptively when cooperated with different vision foundation models. Moreover, we design a affinity-aware merging algorithm to improve the final ensemble masks. PointSeg demonstrates impressive segmentation performance across various datasets, all without training. Specifically, our approach significantly surpasses the state-of-the-art specialist training-free model by 14.1$\%$, 12.3$\%$, and 12.6$\%$ mAP on ScanNet, ScanNet++, and KITTI-360 datasets, respectively. On top of that, PointSeg can incorporate with various foundation models and even surpasses the specialist training-based methods by 3.4$\%$-5.4$\%$ mAP across various datasets, serving as an effective generalist model.
4D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimation
National Key Lab of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(多谱信息智能处理国家实验室,人工智能与自动化学院,华中科技大学);School of Computing, National University of Singapore(计算学院,新加坡国立大学)
Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review
MBZ University of AI(MBZ人工智能大学);Hamad Bin Khalifa University(哈马德·本·卡西姆大学);Birmingham City University(伯明翰城市大学);Qatar University(卡塔尔大学);University Hospitals Birmingham and University of Birmingham(伯明翰大学医院和伯明翰大学)
Technical Report for the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Pretraining-Diverse Ensemble of Foundation Vision Encoders for Robust Outdoor Scene Understanding
LION Lab, Hefei University of Technology(合肥工业大学 LION 实验室);University of Macau(澳门大学)
H2G: Hierarchy-Aware Hyperbolic Grouping for 3D Scenes
Department of Computer Science and Engineering, Korea University(韩国大学计算机科学与工程系);Intelligence and Interaction Research Center, Korea Institute of Science and Technology(韩国科学技术院智能与交互研究中心)
Enhancing Gaze Reasoning in Vision Foundation Models for Gaze Following
Beijing Jiaotong University(北京交通大学);University of Birmingham(英国伯明翰大学);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所MAIS部);Microsoft, UK(微软公司(英国))
Towards Label-free Scene Understanding by Vision Foundation Models
The University of Hong Kong(香港大学);Hochschule Bremerhaven(不来梅港应用科学大学);National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong(香港中文大学);ShanghaiTech University(上海科技大学);The University of Sydney(悉尼大学);Texas A&M University(德克萨斯A&M大学)
EndoSfM3D: Learning to 3D Reconstruct Any Endoscopic Surgery Scene using Self-supervised Foundation Model
UCL Hawkes Institute(UCL哈维斯研究所);Department of Medical Physics(医学物理系);Biomedical Engineering, University College London, UK(生物医学工程,伦敦大学学院,英国);Division of Informatics, Imaging and Data Sciences, The University of Manchester, Manchester, UK(信息学、成像与数据科学部门,曼彻斯特大学,曼彻斯特,英国)
ZeroReg: Zero-Shot Point Cloud Registration with Foundation Models
University of Trento(特伦托大学);Fondazione Bruno Kessler(布鲁诺·凯塞勒基金会);East China University of Science and Technology(华东理工大学);ETH Zurich(苏黎世联邦理工学院);Shanghai AI Laboratory(上海人工智能实验室)
Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model
Beihang University(北京航空航天大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shen Yuan Honors College of Beihang University(北京航空航天大学沈元荣誉学院);National University of Singapore(新加坡国立大学)
From Semantics, Scene to Instance-awareness: Distilling Foundation Model for Grounded Open-vocabulary Situation Recognition
National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);Huazhong University of Science and Technology(华中科技大学);The Hong Kong Polytechnic University(香港理工大学)
Attend to Anything: Foundation Model for Unified Human Attention Modeling
College of Computer Science, Sichuan University, Chengdu, 610065, China(四川大学计算机学院);National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, 610065, China(合成视觉基础科学国家重点实验室)
提出 Attend to Anything Model (AAM),一种多模态基础模型,通过层次化语言提示和双曲空间嵌入统一图像、视频和视听任务中的注意力建模,并在16个基准上平均提升6%,视频推理加速约4倍。
DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Features
NVIDIA Research(英伟达研究院);University of Toronto(多伦多大学);University of Southern California(南加州大学);National University of Singapore(新加坡国立大学);Stanford University(斯坦福大学)
RS-Prune: Training-Free Data Pruning at High Ratios for Efficient Remote Sensing Diffusion Foundation Models
Department of Earth System Science, Tsinghua University(清华大学地球系统科学系);School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院);National Supercomputing Center in Shenzhen(深圳国家超算中心);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生学院);Department of Geography, The University of Hong Kong(香港大学地理系);Wangxuan Institute of Computer Technology, Peking University(北京大学王轩计算机技术研究所)