Enhanced Object Detection: A Study on Vast Vocabulary Object Detection Track for V3Det Challenge 2024
University of Science and Technology of China（中国科学技术大学）；Zhejiang University（浙江大学）；Northwestern Polytechnical University（西北工业大学）；Northwest University（西北大学）
Peixi Wu, Bosong Chai, Xuan Nie, Longquan Yan, Zeyu Wang, Qifan Zhou, Boning Wang, Yansong Peng, Hebei Li
发表信息：
Second Place in CVPR 2024 Vast Vocabulary Visual Detection Challenge
In this technical report, we present our findings from the research conducted on the Vast Vocabulary Visual Detection (V3Det) dataset for Supervised Vast Vocabulary Visual Detection task. How to deal with complex categories and detection boxes has become a difficulty in this track. The original supervised detector is not suitable for this task. We have designed a series of improvements, including adjustments to the network structure, changes to the loss function, and design of training strategies. Our model has shown improvement over the baseline and achieved excellent rankings on the Leaderboard for both the Vast Vocabulary Object Detection (Supervised) track and the Open Vocabulary Object Detection (OVD) track of the V3Det Challenge 2024.
Scaling object taxonomies is one of the important steps toward a robust real-world deployment of recognition systems. We have faced remarkable progress in images since the introduction of the LVIS benchmark. To continue this success in videos, a new video benchmark, TAO, was recently presented. Given the recent encouraging results from both detection and tracking communities, we are interested in marrying those two advances and building a strong large vocabulary video tracker. However, supervisions in LVIS and TAO are inherently sparse or even missing, posing two new challenges for training th…
This report introduces the technical details of the team FuXi-Fresher for LVIS Challenge 2021. Our method focuses on the problem in following two aspects: the long-tail distribution and the segmentation quality of mask and boundary. Based on the advanced HTC instance segmentation algorithm, we connect transformer backbone(Swin-L) through composite connections inspired by CBNetv2 to enhance the baseline results. To alleviate the problem of long-tail distribution, we design a Distribution Balanced method which includes dataset balanced and loss function balaced modules. Further, we use a Mask an…
Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Tracking
Autonomous Intelligent Systems, University of Bonn（博恩大学自主智能系统中心）；Lamarr Institute for Machine Learning and AI（拉马尔人工智能与机器学习研究所）；Center for Robotics, University of Bonn（博恩大学机器人中心）
USE: Universal Segment Embeddings for Open-Vocabulary Image Segmentation
Bosch Research North America（博世北美研究院）；Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）；The Ohio State University（俄亥俄州立大学）；University of California Davis（加州大学戴维斯分校）
SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；School of Information and Communications Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）；Ministry of Education Key Lab of Intelligent Network Security, Xi’an, China（教育部智能网络安全重点实验室）；Pazhou Laboratory (Huangpu), Radarweg 29, Guangzhou, China（琶洲实验室（黄埔），雷达路29号，广州，中国）
Recent open-vocabulary detection methods aim to detect novel objects by distilling knowledge from vision-language models (VLMs) trained on a vast amount of image-text pairs. To improve the effectiveness of these methods, researchers have utilized datasets with a large vocabulary that contains a large number of object classes, under the assumption that such data will enable models to extract comprehensive knowledge on the relationships between various objects and better generalize to unseen object classes. In this study, we argue that more fine-grained labels are necessary to extract richer kno…
CLIP-VIS: Adapting CLIP for Open-Vocabulary Video Instance Segmentation
School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件学院）
Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images
School of Software Engineering, Xi’an Jiaotong University（软件工程学院，西安交通大学）；School of Computer Science and Technology and Ministry of Education Key Laboratory of Intelligent Networks and Network Security, Xi’an Jiaotong University（计算机科学与技术学院和教育部智能网络与网络安全重点实验室，西安交通大学）；School of Automation, Xi’an Jiaotong University（自动化学院，西安交通大学）；College of Artificial Intelligence, Xi’an Jiaotong University（人工智能学院，西安交通大学）；School of Mathematics and Statistics and Ministry of Education Key Laboratory of Intelligent Networks and Network Security, Xi’an Jiaotong University（数学与统计学院和教育部智能网络与网络安全重点实验室，西安交通大学）
Open, or non-laparoscopic surgery, represents the vast majority of all operating room procedures, but few tools exist to objectively evaluate these techniques at scale. Current efforts involve human expert-based visual assessment. We leverage advances in computer vision to introduce an automated approach to video analysis of surgical execution. A state-of-the-art convolutional neural network architecture for object detection was used to detect operating hands in open surgery videos. Automated assessment was expanded by combining model predictions with a fast object tracker to enable surgeon-sp…
OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking
School of Software Technology, Zhejiang University（浙江大学软件学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
VOVTrack: Exploring the Potentiality in Videos for Open-Vocabulary Object Tracking
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；City University of Hong Kong（香港城市大学）
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Honda Research Institute（本田研究院）
3D object trackers usually require training on large amounts of annotated data that is expensive and time-consuming to collect. Instead, we propose leveraging vast unlabeled datasets by self-supervised metric learning of 3D object trackers, with a focus on data association. Large scale annotations for unlabeled data are cheaply obtained by automatic object detection and association across frames. We show how these self-supervised annotations can be used in a principled manner to learn point-cloud embeddings that are effective for 3D tracking. We estimate and incorporate uncertainty in self-sup…
Scene text recognition has attracted increasing interest in recent years due to its wide range of applications in multilingual translation, autonomous driving, etc. In this report, we describe our solution to the Out of Vocabulary Scene Text Understanding (OOV-ST) Challenge, which aims to extract out-of-vocabulary (OOV) words from natural scene images. Our oCLIP-based model achieves 28.59\% in h-mean which ranks 1st in end-to-end OOV word recognition track of OOV Challenge in ECCV2022 TiE Workshop.
Attention to Trajectory: Trajectory-Aware Open-Vocabulary Tracking
Institute of Software Chinese Academy of Science（中国科学院软件研究所）；University of Chinese Academy of Science（中国科学院大学）；University of North Texas（北得克萨斯大学）；OPPO Research Institute（OPPO研究院）
Carnegie Mellon University（卡内基梅隆大学）；Inria（法国国家信息与自动化研究所）；Argo AI
For many years, multi-object tracking benchmarks have focused on a handful of categories. Motivated primarily by surveillance and self-driving applications, these datasets provide tracks for people, vehicles, and animals, ignoring the vast majority of objects in the world. By contrast, in the related field of object detection, the introduction of large-scale, diverse datasets (e.g., COCO) have fostered significant progress in developing highly robust solutions. To bridge this gap, we introduce a similarly diverse dataset for Tracking Any Object (TAO). It consists of 2,907 high resolution video…
In this paper, we address the basic problem of recognizing moving objects in video images using Visual Vocabulary model and Bag of Words and track our object of interest in the subsequent video frames using species inspired PSO. Initially, the shadow free images are obtained by background modelling followed by foreground modeling to extract the blobs of our object of interest. Subsequently, we train a cubic SVM with human body datasets in accordance with our domain of interest for recognition and tracking. During training, using the principle of Bag of Words we extract necessary features of ce…
COVTrack++: Learning Open-Vocabulary Multi-Object Tracking from Continuous Videos via a Synergistic Paradigm
School of Computer Science and Technology, Tianjin University（天津大学计算机科学与技术学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology（深圳先进技术大学计算机科学与人工智能学院）
NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving
College of Mechanical and Vehicle Engineering, Hunan University（湖南大学机械与运载工程学院）；State Key Laboratory of Intelligent Autonomous Systems, Tongji University（同济大学智能自主系统国家重点实验室）
OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
The Chinese University of Hong Kong（香港中文大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）；Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）
Associate Everything Detected: Facilitating Tracking-by-Detection to the Unknown
School of Automation Engineering, University of Electronic Science and Technology of China (UESTC)（电子科技大学自动化工程学院）；Shenzhen Institute of Advanced Study and the School of Automation Engineering, University of Electronic Science and Technology of China (UESTC)（深圳先进研究院和电子科技大学自动化工程学院）；School of Information and Communication Engineering, University of Electronic Science and Technology of China (UESTC)（电子科技大学信息与通信工程学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
Identifying Signatures of Image Phenotypes to Track Treatment Response in Liver Disease
Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Austria（维也纳医科大学生物医学影像与影像引导治疗部门）；Institute of Clinical Epidemiology, Public Health, Health Economics, Medical Statistics and Informatics, Medical University of Innsbruck, Austria（因斯布鲁克医科大学临床流行病学、公共卫生、卫生经济学、医学统计与信息学研究所）；Novartis Pharmaceuticals Corporation, East Hanover, NJ, USA（诺华制药公司，东汉诺威，新泽西州，美国）；Novartis Pharma AG, Basel, Switzerland（诺华制药集团，巴塞尔，瑞士）；Comprehensive Center for Artificial Intelligence in Medicine, Medical University of Vienna, Austria（医学人工智能综合中心，维也纳医科大学，奥地利）；Division of Gastroenterology and Hepatology, Department of Medicine III, Medical University of Vienna, Austria（消化内科与肝病科，医学第三部，维也纳医科大学，奥地利）；Medical University of Vienna, Waehringer Guertel 18–20, 1090 Vienna, Austria（维也纳医科大学，瓦尔希林格特尔18–20号，1090维也纳，奥地利）
The traditional bag-of-words approach has found a wide range of applications in computer vision. The standard pipeline consists of a generation of a visual vocabulary, a quantization of the features into histograms of visual words, and a classification step for which usually a support vector machine in combination with a non-linear kernel is used. Given large amounts of data, however, the model suffers from a lack of discriminative power. This applies particularly for action recognition, where the vast amount of video features needs to be subsampled for unsupervised visual vocabulary generatio…
