Existing datasets for training pedestrian detectors in images suffer from limited appearance and pose variation. The most challenging scenarios are rarely included because they are too difficult to capture due to safety reasons, or they are very unlikely to happen. The strict safety requirements in assisted and autonomous driving applications call for an extra high detection accuracy also in these rare situations. Having the ability to generate people images in arbitrary poses, with arbitrary appearances and embedded in different background scenes with varying illumination and weather conditions, is a crucial component for the development and testing of such applications. The contributions of this paper are three-fold. First, we describe an augmentation method for controlled synthesis of urban scenes containing people, thus producing rare or never-seen situations. This is achieved with a data generator (called DummyNet) with disentangled control of the pose, the appearance, and the target background scene. Second, the proposed generator relies on novel network architecture and associated loss that takes into account the segmentation of the foreground person and its composition into the background scene. Finally, we demonstrate that the data generated by our DummyNet improve performance of several existing person detectors across various datasets as well as in challenging situations, such as night-time conditions, where only a limited amount of training data is available. In the setup with only day-time data available, we improve the night-time detector by $17\%$ log-average miss rate over the detector trained with the day-time data only.
University of Trento（特伦托大学）；National University of Singapore（新加坡国立大学）
In this paper, we consider the problem of domain generalization in semantic segmentation, which aims to learn a robust model using only labeled synthetic (source) data. The model is expected to perform well on unseen real (target) domains. Our study finds that the image style variation can largely influence the model's performance and the style features can be well represented by the channel-wise mean and standard deviation of images. Inspired by this, we propose a novel adversarial style augmentation (AdvStyle) approach, which can dynamically generate hard stylized images during training and …
RLAD: Reinforcement Learning from Pixels for Autonomous Driving in Urban Environments
University of Aveiro（阿威罗大学）；Intelligent System Associate Laboratory (LASI)（智能系统联合实验室（LASI））；Institute of Electronics and Informatics Engineering of Aveiro (IEETA)（阿威罗电子与信息工程研究所（IEETA））
Robotics applications in urban environments are subject to obstacles that exhibit specular reflections hampering autonomous navigation. On the other hand, these reflections are highly polarized and this extra information can successfully be used to segment the specular areas. In nature, polarized light is obtained by reflection or scattering. Deep Convolutional Neural Networks (DCNNs) have shown excellent segmentation results, but require a significant amount of data to achieve best performances. The lack of data is usually overcomed by using augmentation methods. However, unlike RGB images, p…
MPI for Informatics（马克斯·普朗克信息学研究所）；Saarland Informatics Campus（萨尔兰信息学园区）；ETH Zürich（苏黎世联邦理工学院）；CISPA（亥姆霍兹信息安全中心）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Tübingen（蒂宾根大学）
For best performance, today's semantic segmentation methods use large and carefully labeled datasets, requiring expensive annotation budgets. In this work, we show that coarse annotation is a low-cost but highly effective alternative for training semantic segmentation models. Considering the urban scene segmentation scenario, we leverage cheap coarse annotations for real-world captured data, as well as synthetic data to train our model and show competitive performance compared with finely annotated real-world data. Specifically, we propose a coarse-to-fine self-training framework that generate…
CP2M: Clustered-Patch-Mixed Mosaic Augmentation for Aerial Image Segmentation
Northwestern University（西北大学）；Carnegie Mellon University（卡内基梅隆大学）；The University of Hong Kong（香港大学）；Duke University（杜克大学）；University College Dublin（都柏林大学学院）；Beijing University of Technology（北京工业大学）；The ADAPT SFI Research Centre（ADAPT SFI研究中心）
Robotics Institute, University of Michigan（密歇根大学机器人研究所）；Department of Mechanical Engineering, University of Michigan（密歇根大学机械工程系）；Department of Naval Architecture and Marine Engineering, University of Michigan（密歇根大学 naval architecture and marine engineering 系）
Performance on benchmark datasets has drastically improved with advances in deep learning. Still, cross-dataset generalization performance remains relatively low due to the domain shift that can occur between two different datasets. This domain shift is especially exaggerated between synthetic and real datasets. Significant research has been done to reduce this gap, specifically via modeling variation in the spatial layout of a scene, such as occlusions, and scene environmental factors, such as time of day and weather effects. However, few works have addressed modeling the variation in the sen…
Recent developments in the field of deep learning for 3D data have demonstrated promising potential for end-to-end learning directly from point clouds. However, many real-world point clouds contain a large class im-balance due to the natural class im-balance observed in nature. For example, a 3D scan of an urban environment will consist mostly of road and facade, whereas other objects such as poles will be under-represented. In this paper we address this issue by employing a weighted augmentation to increase classes that contain fewer points. By mitigating the class im-balance present in the d…
Research Institute for Infrastructure Paradigm Shift (RIPPS)（基础设施范式转变研究所（RIPPS））；Yachiyo Engineering, Co.,Ltd.（八千代工程有限公司）
Urban rivers provide a water environment that influences residential living. River surface monitoring has become crucial for making decisions about where to prioritize cleaning and when to automatically start the cleaning treatment. We focus on the organic mud, or "scum", that accumulates on the river's surface and contributes to the river's odor and has external economic effects on the landscape. Because of its feature of a sparsely distributed and unstable pattern of organic shape, automating the monitoring process has proved difficult. We propose a patch-wise classification pipeline to dete…
PAUL: Uncertainty-Guided Partition and Augmentation for Robust Cross-View Geo-Localization under Noisy Correspondence
National University of Defense Technology（国防科技大学）；Shenzhen Loop Area Institute（深圳河套学院）；School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen（香港中文大学深圳人工智能学院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳研究院）
University of Science and Technology of China（中国科学技术大学）；Tencent Corporation（腾讯公司）；Georgia Institute of Technology（佐治亚理工学院）；The Chinese University of Hong Kong（香港中文大学）；University of Enna Kore（恩纳科雷大学）
In this paper, we propose two techniques, namely joint modeling and data augmentation, to improve system performances for audio-visual scene classification (AVSC). We employ pre-trained networks trained only on image data sets to extract video embedding; whereas for audio embedding models, we decide to train them from scratch. We explore different neural network architectures for joint modeling to effectively combine the video and audio modalities. Moreover, data augmentation strategies are investigated to increase audio-visual training set size. For the video modality the effectiveness of sev…
Despite the recent progress of generative adversarial networks (GANs) at synthesizing photo-realistic images, producing complex urban scenes remains a challenging problem. Previous works break down scene generation into two consecutive phases: unconditional semantic layout synthesis and image synthesis conditioned on layouts. In this work, we propose to condition layout generation as well for higher semantic control: given a vector of class proportions, we generate layouts with matching composition. To this end, we introduce a conditional framework with novel architecture designs and learning …
Computer Vision Lab, TU Dresden（德累斯顿工业大学计算机视觉实验室）；Autonomous Vision Group, MPI for Intelligent Systems Tübingen（蒂宾根马克斯·普朗克智能系统研究所自主视觉研究组）；Computer Vision and Geometry Group, ETH Zürich（苏黎世联邦理工学院计算机视觉与几何研究组）
The success of deep learning in computer vision is based on availability of large annotated datasets. To lower the need for hand labeled images, virtually rendered 3D worlds have recently gained popularity. Creating realistic 3D content is challenging on its own and requires significant human effort. In this work, we propose an alternative paradigm which combines real and synthetic data for learning semantic instance segmentation and object detection models. Exploiting the fact that not all aspects of the scene are equally important for this task, we propose to augment real-world imagery with …
School of Artificial Intelligence and Robotics, Hunan University（人工智能与机器人学院，湖南大学）；Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（人机学与机器人研究所，卡尔斯鲁厄技术大学）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（机器人视觉感知与控制技术国家工程研究中心，湖南大学）
Background-Mixed Augmentation for Weakly Supervised Change Detection
School of Computer Science and Technology, Civil Aviation University of China（中国民航大学计算机科学与技术学院）；Agency for Science, Technology and Research（新加坡科技研究局）；Zhejiang Sci-Tech University（浙江理工大学）；Nanyang Technological University（南洋理工大学）
Multi-task deep learning for large-scale building detail extraction from high-resolution satellite imagery
Nanjing Normal University（南京师范大学）；Peking University（北京大学）；Technical University of Munich（慕尼黑工业大学）；Shenyang Jianzhu University（沈阳建筑大学）；Nanjing University（南京大学）
Microsoft AI for Good Research Lab（微软人工智能造福社会研究实验室）；Stanford RegLab（斯坦福监管实验室）
Longitudinal studies are vital to understanding dynamic changes of the planet, but labels (e.g., buildings, facilities, roads) are often available only for a single point in time. We propose a general model, Temporal Cluster Matching (TCM), for detecting building changes in time series of remotely sensed imagery when footprint labels are observed only once. The intuition behind the model is that the relationship between spectral values inside and outside of building's footprint will change when a building is constructed (or demolished). For instance, in rural settings, the pre-construction are…
