Xidian University（西安电子科技大学）；National University of Defense Technology（国防科技大学）；Edith Cowan University（埃迪斯科文大学）；The University of Western Australia（西澳大学）
Deep learning techniques have led to remarkable breakthroughs in the field of generic object detection and have spawned a lot of scene-understanding tasks in recent years. Scene graph has been the focus of research because of its powerful semantic representation and applications to scene understanding. Scene Graph Generation (SGG) refers to the task of automatically mapping an image into a semantic structural scene graph, which requires the correct labeling of detected objects and their relationships. Although this is a challenging task, the community has proposed a lot of SGG approaches and achieved good results. In this paper, we provide a comprehensive survey of recent achievements in this field brought about by deep learning techniques. We review 138 representative works that cover different input modalities, and systematically summarize existing methods of image-based SGG from the perspective of feature extraction and fusion. We attempt to connect and systematize the existing visual relationship detection methods, to summarize, and interpret the mechanisms and the strategies of SGG in a comprehensive way. Finally, we finish this survey with deep discussions about current existing problems and future research directions. This survey will help readers to develop a better understanding of the current research status and ideas.
Australian National University（澳大利亚国立大学）；Data61-CSIRO（数据61-CSIRO（澳大利亚联邦科学与工业研究组织下属数据科学与人工智能研究所））；Inception Institute of AI（创见人工智能研究所（Inception Institute of AI））
With the availability of low-cost and compact 2.5/3D visual sensing devices, computer vision community is experiencing a growing interest in visual scene understanding of indoor environments. This survey paper provides a comprehensive background to this research topic. We begin with a historical perspective, followed by popular 3D data representations and a comparative analysis of available datasets. Before delving into the application specific details, this survey provides a succinct introduction to the core technologies that are the underlying methods extensively used in the literature. Afte…
Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images
The Hong Kong Polytechnic University（香港理工大学）；Eastern Institute of Technology（东方理工大学）；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）；Great Wall Motor（长城汽车）；Tsinghua University（清华大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）
A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospects
College of Software, Nankai University Tianjin China；School of Electrical；Electronic Engineering, NTU \& Institute for Infocomm Research, A STAR Singapore；School of Information；College of Computer Science, Nankai University Tianjin China；VCIP, College of Computer Science, Nankai University Tianjin China；Academy for Advanced Interdisciplinary Studies, Nankai University Tianjin China；Nankai International Advanced Research Institute Shenzhen Futian China；College of Software, Nankai University；Electronic Engineering, NTU \& Institute for Infocomm Research, A STAR；College of Computer Science, Nankai University；VCIP, College of Computer Science, Nankai University；Academy for Advanced Interdisciplinary Studies, Nankai University；Nankai International Advanced Research Institute
Semantic segmentation plays a fundamental role in a broad variety of computer vision applications, providing key information for the global understanding of an image. Yet, the state-of-the-art models rely on large amount of annotated samples, which are more expensive to obtain than in tasks such as image classification. Since unlabelled data is instead significantly cheaper to obtain, it is not surprising that Unsupervised Domain Adaptation reached a broad success within the semantic segmentation community. This survey is an effort to summarize five years of this incredibly rapidly growing fie…
MarkableAI；Nankai University（南开大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Beihang University（北京航空航天大学）
Detecting and segmenting salient objects from natural scenes, often referred to as salient object detection, has attracted great interest in computer vision. While many models have been proposed and several applications have emerged, a deep understanding of achievements and issues remains lacking. We aim to provide a comprehensive review of recent progress in salient object detection and situate this field among other closely related areas such as generic scene segmentation, object proposal generation, and saliency for fixation prediction. Covering 228 publications, we survey i) roots, key con…
This paper is the first to review the scene flow estimation field, which analyzes and compares methods, technical challenges, evaluation methodologies and performance of scene flow estimation. Existing algorithms are categorized in terms of scene representation, data source, and calculation scheme, and the pros and cons in each category are compared briefly. The datasets and evaluation protocols are enumerated, and the performance of the most representative methods is presented. A future vision is illustrated with few questions arisen for discussion. This survey presents a general introduction…
South China University of Technology（华南理工大学）；University of Oulu（奥卢大学）；National University of Defense Technology（国防科技大学）；Chinese University of Hong Kong（香港中文大学）
Scene classification, aiming at classifying a scene image to one of the predefined scene categories by comprehending the entire image, is a longstanding, fundamental and challenging problem in computer vision. The rise of large-scale datasets, which constitute the corresponding dense sampling of diverse real-world scenes, and the renaissance of deep learning techniques, which learn powerful feature representations directly from big raw data, have been bringing remarkable progress in the field of scene representation and classification. To help researchers master needed advances in this field, …
North Carolina State University（北卡罗来纳州立大学）；Mitsubishi Electric Research Labs (MERL)（三菱电机研究实验室）
This survey article summarizes research trends on the topic of anomaly detection in video feeds of a single scene. We discuss the various problem formulations, publicly available datasets and evaluation criteria. We categorize and situate past research into an intuitive taxonomy and provide a comprehensive comparison of the accuracy of many algorithms on standard test sets. Finally, we also provide best practices and suggest some possible directions for future research.
University of Technology Sydney（悉尼科技大学）；Northwest University（西北大学）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））；Shandong Artificial Intelligence Institute（山东省人工智能研究院）；Carnegie Mellon University（卡内基梅隆大学）
Scene graph is a structured representation of a scene that can clearly express the objects, attributes, and relationships between objects in the scene. As computer vision technology continues to develop, people are no longer satisfied with simply detecting and recognizing objects in images; instead, people look forward to a higher level of understanding and reasoning about visual scenes. For example, given an image, we want to not only detect and recognize objects in the image, but also know the relationship between objects (visual relationship detection), and generate a text description (imag…
Anhui University（安徽大学）；Hefei University of Technology（合肥工业大学）；Advanced Digital Sciences Center（先进数字科学中心）；National University of Singapore（新加坡国立大学）
Automated scene analysis has been a topic of great interest in computer vision and cognitive science. Recently, with the growth of crowd phenomena in the real world, crowded scene analysis has attracted much attention. However, the visual occlusions and ambiguities in crowded scenes, as well as the complex behaviors and scene semantics, make the analysis a challenging task. In the past few years, an increasing number of works on crowded scene analysis have been reported, covering different aspects including crowd motion pattern learning, crowd behavior and activity analysis, and anomaly detect…
The use of deep learning techniques has exploded during the last few years, resulting in a direct contribution to the field of artificial intelligence. This work aims to be a review of the state-of-the-art in scene recognition with deep learning models from visual data. Scene recognition is still an emerging field in computer vision, which has been addressed from a single image and dynamic image perspective. We first give an overview of available datasets for image and video scene recognition. Later, we describe ensemble techniques introduced by research papers in the field. Finally, we give s…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Tencent AI Lab（腾讯人工智能实验室）
We address the challenging problem of image captioning by revisiting the representation of image scene graph. At the core of our method lies the decomposition of a scene graph into a set of sub-graphs, with each sub-graph capturing a semantic component of the input image. We design a deep model to select important sub-graphs, and to decode each selected sub-graph into a single target sentence. By using sub-graphs, our model is able to attend to different components of the image. Our method thus accounts for accurate, diverse, grounded and controllable captioning at the same time. We present ex…
Semantic Scene Completion (SSC) aims to jointly estimate the complete geometry and semantics of a scene, assuming partial sparse input. In the last years following the multiplication of large-scale 3D datasets, SSC has gained significant momentum in the research community because it holds unresolved challenges. Specifically, SSC lies in the ambiguous completion of large unobserved areas and the weak supervision signal of the ground truth. This led to a substantially increasing number of papers on the matter. This survey aims to identify, compare and analyze the techniques providing a critical …
Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）
Understanding a scene by decoding the visual relationships depicted in an image has been a long studied problem. While the recent advances in deep learning and the usage of deep neural networks have achieved near human accuracy on many tasks, there still exists a pretty big gap between human and machine level performance when it comes to various visual relationship detection tasks. Developing on earlier tasks like object recognition, segmentation and captioning which focused on a relatively coarser image understanding, newer tasks have been introduced recently to deal with a finer level of ima…
Our daily life is surrounded by textual information. Nowadays, the automatic collection of textual information becomes possible owing to the drastic improvement of scene text detectors and recognizer. The purpose of this paper is to conduct a large-scale survey of co-occurrence between visual objects (such as book and car) and scene texts with a large image dataset and a state-of-the-art scene text detector and recognizer. Especially, we focus on the function of "label" texts, which are attached to objects for detailing the objects. By analyzing co-occurrence between objects and scene texts, i…
Huazhong University of Science and Technology（华中科技大学）
Ethiopic/Amharic script is one of the oldest African writing systems, which serves at least 23 languages (e.g., Amharic, Tigrinya) in East Africa for more than 120 million people. The Amharic writing system, Abugida, has 282 syllables, 15 punctuation marks, and 20 numerals. The Amharic syllabic matrix is derived from 34 base graphemes/consonants by adding up to 12 appropriate diacritics or vocalic markers to the characters. The syllables with a common consonant or vocalic markers are likely to be visually similar and challenge text recognition tasks. In this work, we presented the first compre…
Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation
The Future Network of Intelligence Institute, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）未来智能网络研究院）；School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）理工学院）
In current practice, scene survey is carried out by workers using total stations. The method has high accuracy, but it incurs high costs if continuous monitoring is needed. Techniques based on photogrammetry, with the relatively cheaper digital cameras, have gained wide applications in many fields. Besides point measurement, photogrammetry can also create a three-dimensional (3D) model of the scene. Accurate 3D model reconstruction depends on high quality images. Degraded images will result in large errors in the reconstructed 3D model. In this paper, we propose a method that can be used to im…
SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；ETH Zürich（苏黎世联邦理工学院）；University of Amsterdam（阿姆斯特丹大学）；Johns Hopkins University（约翰·霍普金斯大学）；University of Pisa（比萨大学）；University of Trento（特伦托大学）
GPT4SGG: Synthesizing Scene Graphs from Holistic and Region-specific Narratives
The Hong Kong Polytechnic University（香港理工大学）；Centre for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港智能科学与技术研究院人工智能与机器人中心）；NLPR, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
Text in curve orientation, despite being one of the common text orientations in real world environment, has close to zero existence in well received scene text datasets such as ICDAR2013 and MSRA-TD500. The main motivation of Total-Text is to fill this gap and facilitate a new research direction for the scene text community. On top of the conventional horizontal and multi-oriented texts, it features curved-oriented text. Total-Text is highly diversified in orientations, more than half of its images have a combination of more than two orientations. Recently, a new breed of solutions that casted…
Scene text removal (STR), a task of erasing text from natural scene images, has recently attracted attention as an important component of editing text or concealing private information such as ID, telephone, and license plate numbers. While there are a variety of different methods for STR actively being researched, it is difficult to evaluate superiority because previously proposed methods do not use the same standardized training/evaluation dataset. We use the same standardized training/testing dataset to evaluate the performance of several previous methods after standardized re-implementatio…
Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review
MBZ University of AI（MBZ人工智能大学）；Hamad Bin Khalifa University（哈马德·本·卡西姆大学）；Birmingham City University（伯明翰城市大学）；Qatar University（卡塔尔大学）；University Hospitals Birmingham and University of Birmingham（伯明翰大学医院和伯明翰大学）
From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representations
Advanced Robotics, Istituto Italiano di Tecnologia (IIT)（先进机器人技术，意大利技术研究院）；Istituto Nazionale per l’Assicurazione contro gli Infortuni sul Lavoro (INAIL)（国家职业伤害保险研究所）；School of Computer Science, University of Nottingham（计算机科学学院，诺丁汉大学）
本文综述 3D Gaussian Splatting 在 SLAM、远程操作、机器人操控和三维内容生成中的应用，比较其相对 NeRF 的技术优势、适配方式与局限，并指出其在真实感、几何和效率间的平衡。
The history of text can be traced back over thousands of years. Rich and precise semantic information carried by text is important in a wide range of vision-based application scenarios. Therefore, text recognition in natural scenes has been an active research field in computer vision and pattern recognition. In recent years, with the rise and development of deep learning, numerous methods have shown promising in terms of innovation, practicality, and efficiency. This paper aims to (1) summarize the fundamental problems and the state-of-the-art associated with scene text recognition; (2) introd…
A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
Hangzhou Normal University（杭州师范大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Whole understanding of the surroundings is paramount to autonomous systems. Recent works have shown that deep neural networks can learn geometry (depth) and motion (optical flow) from a monocular video without any explicit supervision from ground truth annotations, particularly hard to source for these two tasks. In this paper, we take an additional step toward holistic scene understanding with monocular cameras by learning depth and motion alongside with semantics, with supervision for the latter provided by a pre-trained network distilling proxy ground truth images. We address the three task…
The driving behavior at urban intersections is very complex. It is thus crucial for autonomous vehicles to comprehensively understand challenging urban traffic scenes in order to navigate intersections and prevent accidents. In this paper, we introduce a stereo vision and 3D digital map based approach to spatially and temporally analyze the traffic situation at urban intersections. Stereo vision is used to detect, classify and track obstacles, while a 3D digital map is used to improve ego-localization and provide context in terms of road-layout information. A probabilistic approach that tempor…
Embodied Intelligence for 3D Understanding: A Survey on 3D Scene Question Answering
organization= College of Electrical；Information Engineering,Hunan University , city= Changsha , postcode= 410082 , state= Hunan , country= China；organization= School of Computing \& Information Systems ,The University of Melbourne , city= Melbourne , postcode= VIC 3053 , state= VIC , country= Australia；organization= School of Computer Science,The University of Sydney , city= Sydney , postcode= NSW 2006 , state= NSW , country= Australia；organization= School of Artificial Intelligence ,Anhui University , city= Hefei , postcode= 230601 , state= Anhui , country= China
The University of Sydney（悉尼大学）；University of Technology Sydney（悉尼科技大学）
Registration is a transformation estimation problem between two point clouds, which has a unique and critical role in numerous computer vision applications. The developments of optimization-based methods and deep learning methods have improved registration robustness and efficiency. Recently, the combinations of optimization-based and deep learning methods have further improved performance. However, the connections between optimization-based and deep learning methods are still unclear. Moreover, with the recent development of 3D sensors and 3D reconstruction techniques, a new research directio…
Beijing University of Posts and Telecommunications（北京邮电大学）
Deep learning applies multiple processing layers to learn representations of data with multiple levels of feature extraction. This emerging technique has reshaped the research landscape of face recognition (FR) since 2014, launched by the breakthroughs of DeepFace and DeepID. Since then, deep learning technique, characterized by the hierarchical architecture to stitch together pixels into invariant face representation, has dramatically improved the state-of-the-art performance and fostered successful real-world applications. In this survey, we provide a comprehensive review of the recent devel…
Tecnologico de Monterrey（蒙特雷科技大学）；University of Leeds（利兹大学）
Minimally invasive surgery is highly operator dependant with a lengthy procedural time causing fatigue to surgeon and risks to patients such as injury to organs, infection, bleeding, and complications of anesthesia. To mitigate such risks, real-time systems are desired to be developed that can provide intra-operative guidance to surgeons. For example, an automated system for tool localization, tool (or tissue) tracking, and depth estimation can enable a clear understanding of surgical scenes preventing miscalculations during surgical procedures. In this work, we present a systematic review of …
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
Remote sensing image scene classification, which aims at labeling remote sensing images with a set of semantic categories based on their contents, has broad applications in a range of fields. Propelled by the powerful feature learning capabilities of deep neural networks, remote sensing image scene classification driven by deep learning has drawn remarkable attention and achieved significant breakthroughs. However, to the best of our knowledge, a comprehensive review of recent achievements regarding deep learning for scene classification of remote sensing images is still lacking. Considering t…
Inspiring the Next Generation of Segment Anything Models: Comprehensively Evaluate SAM and SAM 2 with Diverse Prompts Towards Context-Dependent Concepts under Different Scenes
Yale University, USA（耶鲁大学）；Nanyang Technological University, Singapore（南洋理工大学）；Dalian University of Technology, China（大连理工大学）；X3000 Inspection Co., Ltd, China（X3000检测有限公司）
