Dynamic Time Warping (DTW) is widely used for temporal data processing. However, existing methods can neither learn the discriminative prototypes of different classes nor exploit such prototypes for further analysis. We propose Discriminative Prototype DTW (DP-DTW), a novel method to learn class-specific discriminative prototypes for temporal recognition tasks. DP-DTW shows superior performance compared to conventional DTWs on time series classification benchmarks. Combined with end-to-end deep learning, DP-DTW can handle challenging weakly supervised action segmentation problems and achieves state of the art results on standard benchmarks. Moreover, detailed reasoning on the input video is enabled by the learned action prototypes. Specifically, an action-based video summarization can be obtained by aligning the input sequence with action prototypes.
We address weakly supervised action alignment and segmentation in videos, where only the order of occurring actions is available during training. We propose Discriminative Differentiable Dynamic Time Warping (D3TW), the first discriminative model using weak ordering supervision. The key technical challenge for discriminative modeling with weak supervision is that the loss function of the ordering supervision is usually formulated using dynamic programming and is thus not differentiable. We address this challenge with a continuous relaxation of the min-operator in dynamic programming and extend…
Many time-series classification problems involve developing metrics that are invariant to temporal misalignment. In human activity analysis, temporal misalignment arises due to various reasons including differing initial phase, sensor sampling rates, and elastic time-warps due to subject-specific biomechanics. Past work in this area has only looked at reducing intra-class variability by elastic temporal alignment. In this paper, we propose a hybrid model-based and data-driven approach to learn warping functions that not just reduce intra-class variability, but also increase inter-class separat…
Deep time series metric learning is challenging due to the difficult trade-off between temporal invariance to nonlinear distortion and discriminative power in identifying non-matching sequences. This paper proposes a novel neural network-based approach for robust yet discriminative time series classification and verification. This approach adapts a parameterized attention model to time warping for greater and more adaptive temporal invariance. It is robust against not only local but also large global distortions, so that even matching pairs that do not satisfy the monotonicity, continuity, and…
Recent efforts towards video anomaly detection (VAD) try to learn a deep autoencoder to describe normal event patterns with small reconstruction errors. The video inputs with large reconstruction errors are regarded as anomalies at the test time. However, these methods sometimes reconstruct abnormal inputs well because of the powerful generalization ability of deep autoencoder. To address this problem, we present a novel approach for anomaly detection, which utilizes discriminative prototypes of normal data to reconstruct video frames. In this way, the model will favor the reconstruction of no…
Open-world object detection (OWOD) is a challenging problem that combines object detection with incremental learning and open-set learning. Compared to standard object detection, the OWOD setting is task to: 1) detect objects seen during training while identifying unseen classes, and 2) incrementally learn the knowledge of the identified unknown objects when the corresponding annotations is available. We propose a novel and efficient OWOD solution from a prototype perspective, which we call OCPL: Open-world object detection via discriminative Class Prototype Learning, which consists of a Propo…
Toward Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion
Computer Vision and Learning Systems, Department of Electrical Engineering, Linköping University（计算机视觉与学习系统，电气工程系，利厄普大学）；Helmholtz-Zentrum Dresden-Rossendorf, Responsible AI Group（海姆霍尔茨·德累斯顿-罗斯托克研究中心，负责任人工智能小组）；University of Iceland, Faculty of Electrical and Computer Engineering（冰岛大学，电气与计算机工程学院）
ShanghaiTech University（上海科技大学）；University of Chinese Academy of Sciences（中国科学院大学）
Zero-shot learning (ZSL) aims to recognize objects of novel classes without any training samples of specific classes, which is achieved by exploiting the semantic information and auxiliary datasets. Recently most ZSL approaches focus on learning visual-semantic embeddings to transfer knowledge from the auxiliary datasets to the novel classes. However, few works study whether the semantic information is discriminative or not for the recognition task. To tackle such problem, we propose a coupled dictionary learning approach to align the visual-semantic structures using the class prototypes, wher…
FLOW: Feature-Level Optimal Warping for Generalized Remote Physiological Measurement
Great Bay University（大湾区大学）；Hefei University of Technology（合肥工业大学）；Southern University of Science and Technology（南方科技大学）；Macao Polytechnic University（澳门理工大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Dongguan Key Laboratory for Intelligence and Information Technology（东莞市智能与信息技术重点实验室）
Multimodal Feature Prototype Learning for Interpretable and Discriminative Cancer Survival Prediction
Hangzhou Dianzi University（杭州电子科技大学）；School of Information and Communication Technology, Griffith University（信息与通信技术学院，格里菲斯大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Laboratory of Advanced Theranostic Materials and Technology, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences（先进诊疗材料与技术实验室，宁波材料技术与工程研究所，中国科学院）；School of Biomedical Engineering, Tsinghua University（生物医学工程学院，清华大学）
Discriminative Image Generation with Diffusion Models for Zero-Shot Learning
Huazhong University of Science and Technology（华中科技大学）；Zhejiang University（浙江大学）；Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）
The Japan Research Institute, Limited（日本研究机构有限公司）；Graduate School of Science and Technology, University of Tsukuba（筑波大学科学技术研究生院）；Tsukuba Institute for Advanced Research, Department of Computer Science, University of Tsukuba（筑波先进研究所计算机科学系，筑波大学）
Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）
We present the discriminative recurrent sparse auto-encoder model, comprising a recurrent encoder of rectified linear units, unrolled for a fixed number of iterations, and connected to two linear decoders that reconstruct the input and predict its supervised classification. Training via backpropagation-through-time initially minimizes an unsupervised sparse reconstruction error; the loss function is then augmented with a discriminative term on the supervised classification. The depth implicit in the temporally-unrolled form allows the system to exhibit all the power of deep networks, while sub…
University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）；Pohang University of Science and Technology (POSTECH)（浦项科技大学）
Most recent studies on detecting and localizing temporal anomalies have mainly employed deep neural networks to learn the normal patterns of temporal data in an unsupervised manner. Unlike them, the goal of our work is to fully utilize instance-level (or weak) anomaly labels, which only indicate whether any anomalous events occurred or not in each instance of temporal data. In this paper, we present WETAS, a novel framework that effectively identifies anomalous temporal segments (i.e., consecutive time points) in an input instance. WETAS learns discriminative features from the instance-level l…
Spatio-temporal Decoupled Knowledge Compensator for Few-Shot Action Recognition
School of Computer Science and Engineering, Nanjing University of Science and Technology（计算机科学与工程学院，南京理工大学）；State Key Lab of Brain-Machine Intelligence, Zhejiang University（脑机智能国家重点实验室，浙江大学）
University of Science and Technology of China（中国科学技术大学）；JD AI Research（京东人工智能研究院）
Human actions are typically of combinatorial structures or patterns, i.e., subjects, objects, plus spatio-temporal interactions in between. Discovering such structures is therefore a rewarding way to reason about the dynamics of interactions and recognize the actions. In this paper, we introduce a new design of sub-graphs to represent and encode the discriminative patterns of each action in the videos. Specifically, we present MUlti-scale Sub-graph LEarning (MUSLE) framework that novelly builds space-time graphs and clusters the graphs into compact sub-graphs on each scale with respect to the …
Scientific and Technological Research Council of Turkey (TÜBİTAK) Space Technologies Research Institute (UZAY)（土耳其科学技术研究理事会（TÜBİTAK）空间技术研究所）；Informatics Institute, Middle East Technical University（中东技术大学信息学研究所）
Recent automated crop mapping via supervised learning-based methods have demonstrated unprecedented improvement over classical techniques. However, most crop mapping studies are limited to same-year crop mapping in which the present year's labeled data is used to predict the same year's crop map. Classification accuracies of these methods degrade considerably in cross-year mapping. Cross-year crop mapping is more useful as it allows the prediction of the following years' crop maps using previously labeled data. We propose Vector Dynamic Time Warping (VDTW), a novel multi-year classification ap…
Semantically Guided Dynamic Visual Prototype Refinement for Compositional Zero-Shot Learning
National Key Laboratory of Radar Signal Processing, Xidian University（雷达信号处理国家级重点实验室，西安电子科技大学）；Research Institute of Systems Engineering, Academy of Military Science（系统工程研究所，军事科学院）
IPEC: Test-Time Incremental Prototype Enhancement Classifier for Few-Shot Learning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）；School of Microelectronics, Fudan University（微电子学院，复旦大学）；Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of Education, East China University of Science and Technology（教育部先进化工过程控制与优化重点实验室，东华大学）
Zero-shot learning (ZSL) aims at recognizing classes for which no visual sample is available at training time. To address this issue, one can rely on a semantic description of each class. A typical ZSL model learns a mapping between the visual samples of seen classes and the corresponding semantic descriptions, in order to do the same on unseen classes at test time. State of the art approaches rely on generative models that synthesize visual features from the prototype of a class, such that a classifier can then be learned in a supervised manner. However, these approaches are usually biased to…
ProtoEFNet: Dynamic Prototype Learning for Inherently Interpretable Ejection Fraction Estimation in Echocardiography
Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kongens Lyngby, Denmark（应用数学与计算机科学系，丹麦技术大学，Kongens Lyngby）；Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC, Canada（电气与计算机工程系，不列颠哥伦比亚大学，温哥华，BC，加拿大）；Vancouver General Hospital, Vancouver, BC, Canada（温哥华总医院，温哥华，BC，加拿大）
