The advent of Convolutional Neural Networks (CNNs) has led to their application in several domains. One noteworthy application is the perception system for autonomous driving that relies on the predictions from CNNs. Practitioners evaluate the generalization ability of such CNNs by calculating various metrics on an independent test dataset. A test dataset is often chosen based on only one precondition, i.e., its elements are not a part of the training data. Such a dataset may contain objects that are both similar and novel w.r.t. the training dataset. Nevertheless, existing works do not reckon the novelty of the test samples and treat them all equally for evaluating generalization. Such novelty-based evaluations are of significance to validate the fitness of a CNN in autonomous driving applications. Hence, we propose a CNN generalization scoring framework that considers novelty of objects in the test dataset. We begin with the representation learning technique to reduce the image data into a low-dimensional space. It is on this space we estimate the novelty of the test samples. Finally, we calculate the generalization score as a combination of the test data prediction performance and novelty. We perform an experimental study of the same for our traffic light detection application. In addition, we systematically visualize the results for an interpretable notion of novelty.
Indiana University（印第安纳大学）；University of Michigan（密歇根大学）
Recognizing abnormal events such as traffic violations and accidents in natural driving scenes is essential for successful autonomous driving and advanced driver assistance systems. However, most work on video anomaly detection suffers from two crucial drawbacks. First, they assume cameras are fixed and videos have static backgrounds, which is reasonable for surveillance applications but not for vehicle-mounted cameras. Second, they pose the problem as one-class classification, relying on arduously hand-labeled training datasets that limit recognition to anomaly categories that have been expli…
CARISSMA, Technische Hochschule Ingolstadt（CARISSMA，因戈尔施塔特应用技术大学）；Technical University of Munich（慕尼黑工业大学）
Detecting unknown and untested scenarios is crucial for scenario-based testing. Scenario-based testing is considered to be a possible approach to validate autonomous vehicles. A traffic scenario consists of multiple components, with infrastructure being one of it. In this work, a method to detect novel traffic scenarios based on their infrastructure images is presented. An autoencoder triplet network provides latent representations for infrastructure images which are used for outlier detection. The triplet training of the network is based on the connectivity graphs of the infrastructure. By us…
SenseTime Group Limited（商汤集团有限公司）；Department of Information Engineering, the Chinese University of Hong Kong（香港中文大学信息工程系）
Over the past few years there is a growing interest in the learning-based self driving system. To ensure safety, such systems are first developed and validated in simulators before being deployed in the real world. However, most of the existing driving simulators only contain a fixed set of scenes and a limited number of configurable settings. That might easily cause the overfitting issue for the learning-based driving systems as well as the lack of their generalization ability to unseen scenarios. To better evaluate and improve the generalization of end-to-end driving, we introduce an open-en…
Sun Yat-Sen University（中山大学）；the University of Sydney（悉尼大学）；Beihang University（北京航空航天大学）
As a crucial component in intelligent transportation systems, traffic flow prediction has recently attracted widespread research interest in the field of artificial intelligence (AI) with the increasing availability of massive traffic mobility data. Its key challenge lies in how to integrate diverse factors (such as temporal rules and spatial dependencies) to infer the evolution trend of traffic flow. To address this problem, we propose a unified neural network called Attentive Traffic Flow Machine (ATFM), which can effectively learn the spatial-temporal feature representations of traffic flow…
Unveiling the Inflexibility of Adaptive Embedding in Traffic Forecasting
SUSTech-UTokyo Joint Research Center on Super Smart City（南方科技大学-东京大学超级智慧城市联合研究中心）；Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology (SUSTech)（南方科技大学可信自主系统研究院）；School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Center for Spatial Information Science, University of Tokyo（东京大学空间信息科学中心）
MIETT: Multi-Instance Encrypted Traffic Transformer for Encrypted Traffic Classification
School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）
Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）
When neural networks process images which do not resemble the distribution seen during training, so called out-of-distribution images, they often make wrong predictions, and do so too confidently. The capability to detect out-of-distribution images is therefore crucial for many real-world applications. We divide out-of-distribution detection between novelty detection ---images of classes which are not in the training set but are related to those---, and anomaly detection ---images with classes which are unrelated to the training set. By related we mean they contain the same type of objects, li…
Zhejiang University of Technology（浙江工业大学）；Tianjin University of Technology（天津理工大学）
Novelty detection is a important research area which mainly solves the classification problem of inliers which usually consists of normal samples and outliers composed of abnormal samples. Auto-encoder is often used for novelty detection. However, the generalization ability of the auto-encoder may cause the undesirable reconstruction of abnormal elements and reduce the identification ability of the model. To solve the problem, we focus on the perspective of better reconstructing the normal samples as well as retaining the unique information of normal samples to improve the performance of auto-…
The use of learning-based methods for vehicle behavior prediction is a promising research topic. However, many publicly available data sets suffer from class distribution skews which limits learning performance if not addressed. This paper proposes an interaction-aware prediction model consisting of an LSTM autoencoder and SVM classifier. Additionally, an imbalanced learning technique, the multiclass balancing ensemble is proposed. Evaluations show that the method enhances model performance, resulting in improved classification accuracy. Good generalization properties of learned models are imp…
Advances in traffic forecasting technology can greatly impact urban mobility. In the traffic4cast competition, the task of short-term traffic prediction is tackled in unprecedented detail, with traffic volume and speed information available at 5 minute intervals and high spatial resolution. To improve generalization to unknown cities, as required in the 2021 extended challenge, we propose to predict small quadratic city sections, rather than processing a full-city-raster at once. At test time, breaking down the test data into spatially-cropped overlapping snippets improves stability and robust…
TSceneJAL: Joint Active Learning of Traffic Scenes for 3D Object Detection
Research Institute of Trustworthy Autonomous Systems, and Department of Computer Science and Engineering, Southern University of Science and Technology (SUSTech), China（可信自主系统研究 institute，计算机科学与工程系，南方科技大学，中国）；State Key Lab of Internet of Things for Smart City, University of Macau, China（智慧城市物联网国家重点实验室，澳门大学，中国）；Kuang-Chi Institute of Advanced Technology, China（广驰先进技术研究所，中国）；Shenzhen Deeproute.ai Co.,Ltd, China（深圳DeepRoute.ai有限公司，中国）
SSF-PAN: Semantic Scene Flow-Based Perception for Autonomous Navigation in Traffic Scenarios
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Shenzhen Deeproute.ai Co.,Ltd（深圳元戎启行科技有限公司）
CARISSMA, Technische Hochschule Ingolstadt（因戈尔施塔特应用技术大学 CARISSMA）；Technical University Munich（慕尼黑工业大学）
A novel unsupervised outlier score, which can be embedded into graph based dimensionality reduction techniques, is presented in this work. The score uses the directed nearest neighbor graphs of those techniques. Hence, the same measure of similarity that is used to project the data into lower dimensions, is also utilized to determine the outlier score. The outlier score is realized through a weighted normalized entropy of the similarities. This score is applied to road infrastructure images. The aim is to identify newly observed infrastructures given a pre-collected base dataset. Detecting unk…
AR-GAN: Generative Adversarial Network-Based Defense Method Against Adversarial Attacks on the Traffic Sign Classification System of Autonomous Vehicles
International Institute for Information Technology, Hyderabad（海得拉巴国际信息技术学院）
Multi-view Detection (MVD) is highly effective for occlusion reasoning in a crowded environment. While recent works using deep learning have made significant advances in the field, they have overlooked the generalization aspect, which makes them impractical for real-world deployment. The key novelty of our work is to formalize three critical forms of generalization and propose experiments to evaluate them: generalization with i) a varying number of cameras, ii) varying camera positions, and finally, iii) to new scenes. We find that existing state-of-the-art models show poor generalization by o…
Recent advances in visual recognition show overarching success by virtue of large amounts of supervised data. However,the acquisition of a large supervised dataset is often challenging. This is also true for intelligent transportation applications, i.e., traffic sign recognition. For example, a model trained with data of one country may not be easily generalized to another country without much data. We propose a novel feature embedding scheme for unseen class classification when the representative class template is given. Traffic signs, unlike other objects, have official images. We perform co…
Universidade Federal do Espírito Santo（圣灵联邦大学）；Instituto Federal do Espírito Santo（圣灵联邦研究所）
Autonomous terrestrial vehicles must be capable of perceiving traffic lights and recognizing their current states to share the streets with human drivers. Most of the time, human drivers can easily identify the relevant traffic lights. To deal with this issue, a common solution for autonomous cars is to integrate recognition with prior maps. However, additional solution is required for the detection and recognition of the traffic light. Deep learning techniques have showed great performance and power of generalization including traffic related problems. Motivated by the advances in deep learni…
Urban Traffic Surveillance (UTS) is a surveillance system based on a monocular and calibrated video camera that detects vehicles in an urban traffic scenario with dense traffic on multiple lanes and vehicles performing sharp turning maneuvers. UTS then tracks the vehicles using a 3D bounding box representation and a physically reasonable 3D motion model relying on an unscented Kalman filter based approach. Since UTS recovers positions, shape and motion information in a three-dimensional world coordinate system, it can be employed to recognize diverse traffic violations or to supply intelligent…
Pan-Cancer mitotic figures detection and domain generalization: MIDOG 2025 Challenge
Department of Medical Physics and Biomedical Engineering, UCL, UK（医学物理与生物医学工程系，UCL，英国）；Department of Radiotherapy, University College London Hospitals, London, UK（放射治疗系，University College London Hospitals，伦敦，英国）；Centre for Evolution and Cancer, The Institute of Cancer Research, London, UK（进化与癌症中心，癌症研究所在伦敦，英国）
