We developed a robust solution for real-time 6D object detection in industrial applications by integrating FoundationPose, SAM2, and LightGlue, eliminating the need for retraining. Our approach addresses two key challenges: the requirement for an initial object mask in the first frame in FoundationPose and issues with tracking loss and automatic rotation for symmetric objects. The algorithm requires only a CAD model of the target object, with the user clicking on its location in the live feed during the initial setup. Once set, the algorithm automatically saves a reference image of the object and, in subsequent runs, employs LightGlue for feature matching between the object and the real-time scene, providing an initial prompt for detection. Tested on the YCB dataset and industrial components such as bleach cleanser and gears, the algorithm demonstrated reliable 6D detection and tracking. By integrating SAM2 and FoundationPose, we effectively mitigated common limitations such as the problem of tracking loss, ensuring continuous and accurate tracking under challenging conditions like occlusion or rapid movement.
Introducing semantically meaningful objects to visual Simultaneous Localization And Mapping (SLAM) has the potential to improve both the accuracy and reliability of pose estimates, especially in challenging scenarios with significant view-point and appearance changes. However, how semantic objects should be represented for an efficient inclusion in optimization-based SLAM frameworks is still an open question. Superquadrics(SQs) are an efficient and compact object representation, able to represent most common object types to a high degree, and typically retrieved from 3D point-cloud data. Howev…
PLGS: Robust Panoptic Lifting with 3D Gaussian Splatting
School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）；Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences（中国科学院软件研究所）；Peking University（北京大学）
A Good Initialization is All You Need for Faithful Visual Attribution
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Object detection is a central downstream task used to test if pre-trained network parameters confer benefits, such as improved accuracy or training speed. The complexity of object detection methods can make this benchmarking non-trivial when new architectures, such as Vision Transformer (ViT) models, arrive. These difficulties (e.g., architectural incompatibility, slow training, high memory consumption, unknown training formulae, etc.) have prevented recent studies from benchmarking detection transfer learning with standard ViT models. In this paper, we present training techniques that overcom…
Mask Proposal Voting Based on Geodesic Framework for Robust Image Segmentation
Yuanshen Rehabilitation Institute, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金康复医院）；Yueyang Hospital of Integrated Traditional Chinese and Western Medicine, Shanghai University of Traditional Chinese Medicine（上海中医药大学附属岳阳中西医结合医院）；Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University, Shandong Academy of Medical Sciences（山东第一医科大学附属山东省肿瘤医院放疗科）；University Paris Dauphine, PSL Research University, CNRS, UMR 7534, CEREMADE（巴黎多芬纳大学，PSL研究大学，法国国家科学研究中心，UMR 7534，CEREMADE）
ReflexTrack: A Feedback-Driven Agent for Training-Free Referring Video Object Segmentation
School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机科学学院）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音和信号处理中心）
Graph3R: Unifying View Graph Initialization with Feed-Forward Reconstruction
Hong Kong University of Science and Technology（香港科技大学）；Southern University of Science and Technology（南方科技大学）；China Merchants Group（招商局集团）；LionRock AI Lab（狮岩人工智能实验室）；SmartMore Ltd.（思谋科技有限公司）
We report competitive results on object detection and instance segmentation on the COCO dataset using standard models trained from random initialization. The results are no worse than their ImageNet pre-training counterparts even when using the hyper-parameters of the baseline system (Mask R-CNN) that were optimized for fine-tuning pre-trained models, with the sole exception of increasing the number of training iterations so the randomly initialized models may converge. Training from random initialization is surprisingly robust; our results hold even when: (i) using only 10% of the training da…
We present a novel form of interactive video object segmentation where a few clicks by the user helps the system produce a full spatio-temporal segmentation of the object of interest. Whereas conventional interactive pipelines take the user's initialization as a starting point, we show the value in the system taking the lead even in initialization. In particular, for a given video frame, the system precomputes a ranked list of thousands of possible segmentation hypotheses (also referred to as object region proposals) using image and motion cues. Then, the user looks at the top ranked proposals…
The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution
Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）；University of Science and Technology Beijing (USTB)（北京科技大学）
Video object segmentation (VOS) describes the task of segmenting a set of objects in each frame of a video. In the semi-supervised setting, the first mask of each object is provided at test time. Following the one-shot principle, fine-tuning VOS methods train a segmentation model separately on each given object mask. However, recently the VOS community has deemed such a test time optimization and its impact on the test runtime as unfeasible. To mitigate the inefficiencies of previous fine-tuning approaches, we present efficient One-Shot Video Object Segmentation (e-OSVOS). In contrast to most …
MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation
Faculty of Electrical Engineering and Computing, University of Zagreb（萨格勒布大学电气工程与计算学院）；Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）；Fundamental AI Lab, University of Technology Nuremberg（纽伦堡工业大学基础人工智能实验室）
Korea Advanced Institute of Science and Technology（韩国科学技术院）
In recent years, a ton of research has been conducted on real image denoising tasks. However, the efforts are more focused on improving real image denoising through creating a better network architecture. We explore a different direction where we propose to improve real image denoising performance through a better learning strategy that can enable test-time adaptation on the multi-task network. The learning strategy is two stages where the first stage pre-train the network using meta-auxiliary learning to get better meta-initialization. Meanwhile, we use meta-learning for fine-tuning (meta-tra…
Medical image segmentation, or computing voxelwise semantic masks, is a fundamental yet challenging task to compute a voxel-level semantic mask. To increase the ability of encoder-decoder neural networks to perform this task across large clinical cohorts, contrastive learning provides an opportunity to stabilize model initialization and enhance encoders without labels. However, multiple target objects (with different semantic meanings) may exist in a single image, which poses a problem for adapting traditional contrastive learning methods from prevalent 'image-level classification' to 'pixel-l…
Vis2Reg: Visibility-Aware Landmark-Free Geometric 3D--2D Registration for Liver Laparoscopy
AI in Medicine and Surgery Group, School of Computer Science, University of Leeds（利兹大学计算机学院医学与外科人工智能小组）；Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院放射诊断学系）；Department of HPB and Transplant Surgery, St. James’s University Hospital（圣詹姆斯大学医院肝脏胰胆和移植外科）
Vision-Aware Text Features in Referring Image Segmentation: From Object Understanding to Context Understanding
The Hong Kong University of Science and Technology（香港科技大学）；Stony Brook University（石溪大学）；University of Science, VNU-HCM（胡志明市国家大学科学大学）；Vietnam National University, Ho Chi Minh City（胡志明市国家大学）；Trinity College Dublin（都柏林大学学院）
