Geo-Spatiotemporal Features and Shape-Based Prior Knowledge for Fine-grained Imbalanced Data Classification
MILA, Montreal Institute for Learning Algorithms（MILA蒙特利尔学习算法研究所）；Paris-Saclay University（巴黎萨克雷大学）；CentraleSupélec Paris (ECP)（巴黎中央理工学院）；INRIA Paris（法国国家信息与自动化研究所巴黎分部）；Operations Research Center, MIT（麻省理工学院运筹学中心）；Université de Montréal（蒙特利尔大学）；University of Toronto（多伦多大学）；McGill University（麦吉尔大学）
Charles A. Kantor, Marta Skreta, Brice Rauby, Léonard Boussioux, Emmanuel Jehanno, Alexandra Luccioni, David Rolnick, Hugues Talbot
备注：
Copyright by the authors. All rights reserved to authors only. Correspondence to: ckantor (at) stanford [dot] edu
发表信息：
Proc. IJCAI 2021, Workshop on AI for Social Good, Harvard University (2021)
Fine-grained classification aims at distinguishing between items with similar global perception and patterns, but that differ by minute details. Our primary challenges come from both small inter-class variations and large intra-class variations. In this article, we propose to combine several innovations to improve fine-grained classification within the use-case of wildlife, which is of practical interest for experts. We utilize geo-spatiotemporal data to enrich the picture information and further improve the performance. We also investigate state-of-the-art methods for handling the imbalanced data issue.
Spatiotemporal action recognition deals with locating and classifying actions in videos. Motivated by the latest state-of-the-art real-time object detector You Only Watch Once (YOWO), we aim to modify its structure to increase action detection precision and reduce computational time. Specifically, we propose four novel approaches in attempts to improve YOWO and address the imbalanced class issue in videos by modifying the loss function. We consider two moderate-sized datasets to apply our modification of YOWO - the popular Joint-annotated Human Motion Data Base (J-HMDB-21) and a private datase…
Bangor University（班戈大学）；Universidad de Burgos（布尔戈斯大学）；Middlesex University（密德萨斯大学）
A natural way of handling imbalanced data is to attempt to equalise the class frequencies and train the classifier of choice on balanced data. For two-class imbalanced problems, the classification success is typically measured by the geometric mean (GM) of the true positive and true negative rates. Here we prove that GM can be improved upon by instance selection, and give the theoretical conditions for such an improvement. We demonstrate that GM is non-monotonic with respect to the number of retained instances, which discourages systematic instance selection. We also show that balancing the di…
Northwestern Polytechnical University（西北工业大学）；Chinese Academy of Sciences（中国科学院）；University of Oulu（奥卢大学）
Recently, the recognition task of spontaneous facial micro-expressions has attracted much attention with its various real-world applications. Plenty of handcrafted or learned features have been employed for a variety of classifiers and achieved promising performances for recognizing micro-expressions. However, the micro-expression recognition is still challenging due to the subtle spatiotemporal changes of micro-expressions. To exploit the merits of deep learning, we propose a novel deep recurrent convolutional networks based micro-expression recognition approach, capturing the spatial-tempora…
Modern deep neural networks for classification usually jointly learn a backbone for representation and a linear classifier to output the logit of each class. A recent study has shown a phenomenon called neural collapse that the within-class means of features and the classifier vectors converge to the vertices of a simplex equiangular tight frame (ETF) at the terminal phase of training on a balanced dataset. Since the ETF geometric structure maximally separates the pair-wise angles of all classes in the classifier, it is natural to raise the question, why do we spend an effort to learn a classi…
4D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimation
National Key Lab of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（多谱信息智能处理国家实验室，人工智能与自动化学院，华中科技大学）；School of Computing, National University of Singapore（计算学院，新加坡国立大学）
Geoscience and seismology have utilized the most advanced technologies and equipment to monitor seismic events globally from the past few decades. With the enormous amount of data, modern GPU-powered deep learning presents a promising approach to analyze data and discover patterns. In recent years, there are plenty of successful deep learning models for picking seismic waves. However, forecasting extreme earthquakes, which can cause disasters, is still an underdeveloped topic in history. Relevant research in spatiotemporal dynamics mining and forecasting has revealed some successful prediction…
Huazhong University of Science and Technology（华中科技大学）
Neuromorphic vision sensors (event cameras) are inherently suitable for spiking neural networks (SNNs) and provide novel neuromorphic vision data for this biomimetic model. Due to the spatiotemporal characteristics, novel data augmentations are required to process the unconventional visual signals of these cameras. In this paper, we propose a novel Event SpatioTemporal Fragments (ESTF) augmentation method. It preserves the continuity of neuromorphic data by drifting or inverting fragments of the spatiotemporal event stream to simulate the disturbance of brightness variations, leading to more r…
All-around Neural Collapse for Imbalanced Classification
MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（MIIT模式分析与智能关键实验室）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics (NUAA)（南京航空航天大学计算机科学与技术学院）
The current practice in land cover/land use change analysis relies heavily on the individually classified maps of the multitemporal data set. Due to varying acquisition conditions (e.g., illumination, sensors, seasonal differences), the classification maps yielded are often inconsistent through time for robust statistical analysis. 3D geometric features have been shown to be stable for assessing differences across the temporal data set. Therefore, in this article we investigate he use of a multitemporal orthophoto and digital surface model derived from satellite data for spatiotemporal classif…
