We address six different classification tasks related to fine-grained building attributes: construction type, number of floors, pitch and geometry of the roof, facade material, and occupancy class. Tackling such a remote building analysis problem became possible only recently due to growing large-scale datasets of urban scenes. To this end, we introduce a new benchmarking dataset, consisting of 49426 images (top-view and street-view) of 9674 buildings. These photos are further assembled, together with the geometric metadata. The dataset showcases various real-world challenges, such as occlusions, blur, partially visible objects, and a broad spectrum of buildings. We propose a new projection pooling layer, creating a unified, top-view representation of the top-view and the side views in a high-dimensional space. It allows us to utilize the building and imagery metadata seamlessly. Introducing this layer improves classification accuracy -- compared to highly tuned baseline models -- indicating its suitability for building analysis.
Beijing University of Posts and Telecommunications（北京邮电大学）；Tencent Inc（腾讯公司）
It has already been observed that audio-visual embedding is more robust than uni-modality embedding for person verification. Here, we proposed a novel audio-visual strategy that considers aggregators from a fusion perspective. First, we introduced weight-enhanced attentive statistics pooling for the first time in face verification. We find that a strong correlation exists between modalities during pooling, so joint attentive pooling is proposed which contains cycle consistency to learn the implicit inter-frame weight. Finally, each modality is fused with a gated attention mechanism to gain rob…
Makeup transfer is the task of applying on a source face the makeup style from a reference image. Real-life makeups are diverse and wild, which cover not only color-changing but also patterns, such as stickers, blushes, and jewelries. However, existing works overlooked the latter components and confined makeup transfer to color manipulation, focusing only on light makeup styles. In this work, we propose a holistic makeup transfer framework that can handle all the mentioned makeup components. It consists of an improved color transfer branch and a novel pattern transfer branch to learn all makeu…
University of Central Florida（中佛罗里达大学）；Mayo Clinic（妙佑医疗国际（梅奥诊所））；University of Catania（卡塔尼亚大学）
Colorectal cancer, largely arising from precursor lesions called polyps, remains one of the leading causes of cancer-related death worldwide. Current clinical standards require the resection and histopathological analysis of polyps due to test accuracy and sensitivity of optical biopsy methods falling substantially below recommended levels. In this study, we design a novel capsule network architecture (D-Caps) to improve the viability of optical biopsy of colorectal polyps. Our proposed method introduces several technical novelties including a novel capsule architecture with a capsule-average …
Voronoi-based Second-order Descriptor with Whitened Metric in LiDAR Place Recognition
Interdisciplinary Program in Neuroscience（神经科学跨学科项目）；Département d’Informatique, École Normale Supérieure (ENS)（高等师范学院计算机系）；Department of Computing, Imperial College London（伦敦帝国理工学院计算系）；Dept. of Computer Science and Engineering, Seoul National University（首尔国立大学计算机科学与工程系）
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；Technical University of Dresden（德累斯顿工业大学）；Papanikolaou Hospital（帕帕尼古拉乌医院）；King’s College Hospital NHS Foundation Trust（国王学院医院国民保健服务基金会信托）
Parkinson's Disease (PD) is a slowly evolving neuro-logical disease that affects about 1% of the population above 60 years old, causing symptoms that are subtle at first, but whose intensity increases as the disease progresses. Automated detection of these symptoms could offer clues as to the early onset of the disease, thus improving the expected clinical outcomes of the patients via appropriately targeted interventions. This potential has led many researchers to develop methods that use widely available sensors to measure and quantify the presence of PD symptoms such as tremor, rigidity and …
Sorbonne University - University Pierre and Marie CURIE（索邦大学 - 皮埃尔和玛丽·居里大学）；Image & Pervasive Access Lab（图像与普适访问实验室）；CNRS（法国国家科学研究中心）；I2R（资讯通信研究院）；Nanyang Technological University（南洋理工大学）；Tencent Co. LTD（腾讯有限公司）；Institut Mines-Telecom（矿业电信学院）
Automatic reading texts in scenes has attracted increasing interest in recent years as texts often carry rich semantic information that is useful for scene understanding. In this paper, we propose a novel scene text proposal technique aiming for accurate reading texts in scenes. Inspired by the pooling layer in the deep neural network architecture, a pooling based scene text proposal technique is developed. A novel score function is designed which exploits the histogram of oriented gradients and is capable of ranking the proposals according to their probabilities of being text. An end-to-end s…
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）
Accurately predicting and detecting interstitial lung disease (ILD) patterns given any computed tomography (CT) slice without any pre-processing prerequisites, such as manually delineated regions of interest (ROIs), is a clinically desirable, yet challenging goal. The majority of existing work relies on manually-provided ILD ROIs to extract sampled 2D image patches from CT slices and, from there, performs patch-based ILD categorization. Acquiring manual ROIs is labor intensive and serves as a bottleneck towards fully-automated CT imaging ILD screening over large-scale populations. Furthermore,…
Dalian University of Technology（大连理工大学）；Tianjin University（天津大学）
Deep Convolutional Networks (ConvNets) are fundamental to, besides large-scale visual recognition, a lot of vision tasks. As the primary goal of the ConvNets is to characterize complex boundaries of thousands of classes in a high-dimensional space, it is critical to learn higher-order representations for enhancing non-linear modeling capability. Recently, Global Second-order Pooling (GSoP), plugged at the end of networks, has attracted increasing attentions, achieving much better performance than classical, first-order networks in a variety of vision tasks. However, how to effectively introduc…
Computer Vision Center（计算机视觉中心）；Universitat Autonoma de Barcelona（巴塞罗那自治大学）
Motivated by the success of powerful while expensive techniques to recognize words in a holistic way, object proposals techniques emerge as an alternative to the traditional text detectors. In this paper we introduce a novel object proposals method that is specifically designed for text. We rely on a similarity based region grouping algorithm that generates a hierarchy of word hypotheses. Over the nodes of this hierarchy it is possible to apply a holistic word recognition method in an efficient way. Our experiments demonstrate that the presented method is superior in its ability of producing g…
We present a framework for learning an efficient holistic representation for handwritten word images. The proposed method uses a deep convolutional neural network with traditional classification loss. The major strengths of our work lie in: (i) the efficient usage of synthetic data to pre-train a deep network, (ii) an adapted version of the ResNet-34 architecture with the region of interest pooling (referred to as HWNet v2) which learns discriminative features for variable sized word images, and (iii) a realistic augmentation of training data with multiple scales and distortions which mimics t…
Multi-Scale Fruit Capsules: Dilated Convolutions and Dynamic Routing for In-the-Wild Explainable Fruit Recognition
University of Maryland Baltimore County（马里兰大学巴尔的摩分校）；Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）；Indian Institute of Science Education and Research Bhopal（印度科学教育与研究学院博帕尔分校）；UJM-Saint-Etienne（圣埃蒂安大学）；CNRS（法国国家科学研究中心）；Laboratoire Hubert Curien（于贝尔·居里安实验室）
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
The Chinese University of Hong Kong（香港中文大学）；Tsinghua University（清华大学）；The University of Hong Kong（香港大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）
Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directions
International Hellenic University（国际希腊大学）；University of Thessaly（色萨利大学）；Democritus University of Thrace（德谟克利特大学）；University of Peloponnese（伯罗奔尼撒大学）；Harokopio University of Athens（哈罗科皮奥大学）
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所复杂系统管理与控制国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；College of Engineering and Computer Science, Australian National University（澳大利亚国立大学工程与计算机科学学院）；School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械与车辆学院）
Monocular 3D object detection encounters occlusion problems in many application scenarios, such as traffic monitoring, pedestrian monitoring, etc., which leads to serious false negative. Multi-view object detection effectively solves this problem by combining data from different perspectives. However, due to label confusion and feature confusion, the orientation estimation of multi-view 3D object detection is intractable, which is important for object tracking and intention prediction. In this paper, we propose a novel multi-view 3D object detection method named MVM3Det which simultaneously es…
MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations
Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；Zhiyuan College, Shanghai Jiao Tong University（上海交通大学紫阳学院）；CPII under InnoHK（创新香港下的CPII）
