Despite significant progress in human action recognition, generalizing to diverse viewpoints remains a challenge. Most existing datasets are captured from ground-level perspectives, and models trained on them often struggle to transfer to drastically different domains such as aerial views. This paper examines how curriculum-based training strategies can improve generalization to unseen real aerial-view data without using any real aerial data during training.
We explore curriculum learning for cross-view action recognition using two out-of-domain sources: synthetic aerial-view data and real ground-view data. Our results on the evaluation on order of training (fine-tuning on synthetic aerial data vs. real ground data) shows that fine-tuning on real ground data but differ in how they transition from synthetic to real. The first uses a two-stage curriculum with direct fine-tuning, while the second applies a progressive curriculum that expands the dataset in multiple stages before fine-tuning. We evaluate both methods on the REMAG dataset using SlowFast (CNN-based) and MViTv2 (Transformer-based) architectures.
Results show that combining the two out-of-domain datasets clearly outperforms training on a single domain, whether real ground-view or synthetic aerial-view. Both curriculum strategies match the top-1 accuracy of simple dataset combination while offering efficiency gains. With the two-step fine-tuning method, SlowFast achieves up to a 37% reduction in iterations and MViTv2 up to a 30% reduction compared to simple combination. The multi-step progressive approach further reduces iterations, by up to 9% for SlowFast and 30% for MViTv2, relative to the two-step method. These findings demonstrate that curriculum-based training can maintain comparable performance (top-1 accuracy within 3% range) while improving training efficiency in cross-view action recognition.
We present ConCur, a contrastive video representation learning method that uses curriculum learning to impose a dynamic sampling strategy in contrastive training. More specifically, ConCur starts the contrastive training with easy positive samples (temporally close and semantically similar clips), and as the training progresses, it increases the temporal span effectively sampling hard positives (temporally away and semantically dissimilar). To learn better context-aware representations, we also propose an auxiliary task of predicting the temporal distance between a positive pair of clips. We c…
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（测绘遥感信息工程国家重点实验室）；Wuhan University（武汉大学）；Information Systems Technology and Design Pillar（信息系统技术与设计学院）；Singapore University of Technology and Design（新加坡科技与设计大学）；School of Geodesy and Geomatics（测绘学院）；School of Mathematics and Statistics（数学与统计学院）；Wuhan University Shenzhen Research Institute（武汉大学深圳研究院）
Although dense local spatial-temporal features with bag-of-features representation achieve state-of-the-art performance for action recognition, the huge feature number and feature size prevent current methods from scaling up to real size problems. In this work, we investigate different types of feature sampling strategies for action recognition, namely dense sampling, uniformly random sampling and selective sampling. We propose two effective selective sampling methods using object proposal techniques. Experiments conducted on a large video dataset show that we are able to achieve better averag…
CurEvo: Curriculum-Guided Self-Evolution for Video Understanding
Huazhong University of Science and Technology（华中科技大学）；La Trobe University（拉特罗布大学）；Beijing Institute of Computer Technology and Applications（北京计算机技术与应用研究院）
Recognizing the full-page of Japanese historical documents is a challenging problem due to the complex layout/background and difficulty of writing styles, such as cursive and connected characters. Most of the previous methods divided the recognition process into character segmentation and recognition. However, those methods provide only character bounding boxes and classes without text transcription. In this paper, we enlarge our previous humaninspired recognition system from multiple lines to the full-page of Kuzushiji documents. The human-inspired recognition system simulates human eye movem…
Tencent（腾讯）；Zhejiang University（浙江大学）；Michigan State University（密歇根州立大学）
As an emerging topic in face recognition, designing margin-based loss functions can increase the feature margin between different classes for enhanced discriminability. More recently, the idea of mining-based strategies is adopted to emphasize the misclassified samples, achieving promising results. However, during the entire training process, the prior methods either do not explicitly emphasize the sample based on its importance that renders the hard samples not fully exploited; or explicitly emphasize the effects of semi-hard/hard samples even at the early training stage that may lead to conv…
CLRecogEye : Curriculum Learning towards exploiting convolution features for Dynamic Iris Recognition
Indian Institute of Technology Mandi, India（印度理工学院马尼分校）；Directorate of Forensic Services, Shimla, India（刑事司法服务局）；Norwegian University of Science and Technology (NTNU), Norway（挪威科学与技术大学（NTNU））
Universitat Pompeu Fabra（庞培法布拉大学）；Ecole Centrale de Nantes（南特中央理工学院）；LMU München（慕尼黑大学）；Technische Universität München（慕尼黑工业大学）；ICREA（加泰罗尼亚研究与高等教育学院）
Current deep-learning based methods do not easily integrate to clinical protocols, neither take full advantage of medical knowledge. In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracture from X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are derived from knowledge such as medical decision trees and inconsistencies in the annotations of multiple experts, which allows us to assign a degree of difficulty to each training sample. We demonstr…
Escuela Politécnica Superior, Universidad Autónoma de Madrid（马德里自治大学高等理工学院）
State-of-the-art deep learning approaches for skin lesion recognition often require pretraining on larger and more varied datasets, to overcome the generalization limitations derived from the reduced size of the skin lesion imaging datasets. ImageNet is often used as the pretraining dataset, but its transferring potential is hindered by the domain gap between the source dataset and the target dermatoscopic scenario. In this work, we introduce a novel pretraining approach that sequentially trains a series of Self-Supervised Learning pretext tasks and only requires the unlabeled skin lesion imag…
Class imbalance problems frequently occur in real-world tasks, and conventional deep learning algorithms are well known for performance degradation on imbalanced training datasets. To mitigate this problem, many approaches have aimed to balance among given classes by re-weighting or re-sampling training samples. These re-balancing methods increase the impact of minority classes and reduce the influence of majority classes on the output of models. However, the extracted representations may be of poor quality owing to the limited number of minority samples. To handle this restriction, several me…
Universitat Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚高级研究学院）；Ecole Centrale de Nantes（南特中央理工学院）；Technische Universität München（慕尼黑工业大学）；LMU München（慕尼黑大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
An adequate classification of proximal femur fractures from X-ray images is crucial for the treatment choice and the patients' clinical outcome. We rely on the commonly used AO system, which describes a hierarchical knowledge tree classifying the images into types and subtypes according to the fracture's location and complexity. In this paper, we propose a method for the automatic classification of proximal femur fractures into 3 and 7 AO classes based on a Convolutional Neural Network (CNN). As it is known, CNNs need large and representative datasets with reliable labels, which are hard to co…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；TU Kaiserslautern（凯泽斯劳滕工业大学）
Curriculum learning is a bio-inspired training technique that is widely adopted to machine learning for improved optimization and better training of neural networks regarding the convergence rate or obtained accuracy. The main concept in curriculum learning is to start the training with simpler tasks and gradually increase the level of difficulty. Therefore, a natural question is how to determine or generate these simpler tasks. In this work, we take inspiration from Spatial Transformer Networks (STNs) in order to form an easy-to-hard curriculum. As STNs have been proven to be capable of remov…
University of Bucharest（布加勒斯特大学）；Institute of Mathematics of the Romanian Academy（罗马尼亚科学院数学研究所）；University “Politehnica” of Bucharest（布加勒斯特理工大学）
Despite the significant advances in recent years, Generative Adversarial Networks (GANs) are still notoriously hard to train. In this paper, we propose three novel curriculum learning strategies for training GANs. All strategies are first based on ranking the training images by their difficulty scores, which are estimated by a state-of-the-art image difficulty predictor. Our first strategy is to divide images into gradually more difficult batches. Our second strategy introduces a novel curriculum loss function for the discriminator that takes into account the difficulty scores of the real imag…
Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Universitat Oberta de Catalunya（加泰罗尼亚开放大学）
Video object segmentation can be understood as a sequence-to-sequence task that can benefit from the curriculum learning strategies for better and faster training of deep neural networks. This work explores different schedule sampling and frame skipping variations to significantly improve the performance of a recurrent architecture. Our results on the car class of the KITTI-MOTS challenge indicate that, surprisingly, an inverse schedule sampling is a better option than a classic forward one. Also, that a progressive skipping of frames during training is beneficial, but only when training with …
Flatness-aware Curriculum Learning via Adversarial Difficulty
Graduate School of Advanced Science and Engineering Hiroshima University（弘前大学先进科学与工程研究院）；Intelligent Production Technology Research & Development Center for Aerospace Gifu University（宇治大学航空航天智能生产技术研究开发中心）
We address the problem of semantic nighttime image segmentation and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night through progressively darker times of day, exploiting cross-time-of-day correspondences between daytime images from a reference map and dark images to guide the label inference in th…
National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore (Suzhou) Research Institute（新加坡国立大学苏州研究院）；Imperial College London（帝国理工学院）
Curriculum learning and self-paced learning are the training strategies that gradually feed the samples from easy to more complex. They have captivated increasing attention due to their excellent performance in robotic vision. Most recent works focus on designing curricula based on difficulty levels in input samples or smoothing the feature maps. However, smoothing labels to control the learning utility in a curriculum manner is still unexplored. In this work, we design a paced curriculum by label smoothing (P-CBLS) using paced learning with uniform label smoothing (ULS) for classification tas…
Tsinghua University（清华大学）；Huawei（华为）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）
One-shot Neural Architecture Search (NAS) has been widely used to discover architectures due to its efficiency. However, previous studies reveal that one-shot performance estimations of architectures might not be well correlated with their performances in stand-alone training because of the excessive sharing of operation parameters (i.e., large sharing extent) between architectures. Thus, recent methods construct even more over-parameterized supernets to reduce the sharing extent. But these improved methods introduce a large number of extra parameters and thus cause an undesirable trade-off be…
Recognition of Handwritten Mathematical Expressions (HMEs) is a challenging problem because of the ambiguity and complexity of two-dimensional handwriting. Moreover, the lack of large training data is a serious issue, especially for academic recognition systems. In this paper, we propose pattern generation strategies that generate shape and structural variations to improve the performance of recognition systems based on a small training set. For data generation, we employ the public databases: CROHME 2014 and 2016 of online HMEs. The first strategy employs local and global distortions to gener…
Unsupervised domain adaptation (UDA) approaches focus on adapting models trained on a labeled source domain to an unlabeled target domain. UDA methods have a strong assumption that the source data is accessible during adaptation, which may not be feasible in many real-world scenarios due to privacy concerns and resource constraints of devices. In this regard, source-free domain adaptation (SFDA) excels as access to source data is no longer required during adaptation. Recent state-of-the-art (SOTA) methods on SFDA mostly focus on pseudo-label refinement based self-training which generally suffe…
Universitat Pompeu Fabra（庞培法布拉大学）；IT University of Copenhagen（哥本哈根信息技术大学）；ICREA（加泰罗尼亚研究与高级研究院）；Ecole Centrale de Nantes（南特中央理工学院）；Hera-MI SAS（Hera-MI公司）
For early breast cancer detection, regular screening with mammography imaging is recommended. Routinary examinations result in datasets with a predominant amount of negative samples. A potential solution to such class-imbalance is joining forces across multiple institutions. Developing a collaborative computer-aided diagnosis system is challenging in different ways. Patient privacy and regulations need to be carefully respected. Data across institutions may be acquired from different devices or imaging protocols, leading to heterogeneous non-IID data. Also, for learning-based methods, new opti…
University of Bucharest（布加勒斯特大学）；University of Trento（特伦托大学）
Training machine learning models in a meaningful order, from the easy samples to the hard ones, using curriculum learning can provide performance improvements over the standard training approach based on random data shuffling, without any additional computational costs. Curriculum learning strategies have been successfully employed in all areas of machine learning, in a wide range of tasks. However, the necessity of finding a way to rank the samples from easy to hard, as well as the right pacing function for introducing more difficult data can limit the usage of the curriculum approaches. In t…
STCL:Curriculum learning Strategies for deep learning image steganography models
Qianan College, North China University of Science and Technology（华北科技大学钱安学院）；School of Cyberspace Security, Beijing University of Posts and Telecommunications（北京邮电大学网络安全学院）；College of Science, North China University of Science and Technology（华北科技大学理学院）
AdaSin: Enhancing Hard Sample Metrics with Dual Adaptive Penalty for Face Recognition
School of Intelligent Systems Science and Engineering, Jinan University（暨南大学智能系统科学与工程学院）；College of Cyber Security, Jinan University（暨南大学网络空间安全学院）；School of Computer Science and Engineering, Macau University of Science and Technology（澳门科技大学计算机科学与工程学院）
University of Science and Technology of China（中国科学技术大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
The audio-video based emotion recognition aims to classify a given video into basic emotions. In this paper, we describe our approaches in EmotiW 2019, which mainly explores emotion features and feature fusion strategies for audio and visual modality. For emotion features, we explore audio feature with both speech-spectrogram and Log Mel-spectrogram and evaluate several facial features with different CNN models and different emotion pretrained strategies. For fusion strategies, we explore intra-modal and cross-modal fusion methods, such as designing attention mechanisms to highlights important…
Scale-Aware Curriculum Learning for Ddata-Efficient Lung Nodule Detection with YOLOv11
Department of Biomedical Engineering, Johns Hopkins University（生物医学工程系，约翰霍普金斯大学）；Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University（放射肿瘤学与分子放射科学系，约翰霍普金斯大学）
Sri Lanka Institute of Information Technology（斯里兰卡信息技术学院）；University of Sheffield（谢菲尔德大学）
This paper presents an implementation on child activity recognition (CAR) with a graph convolution network (GCN) based deep learning model since prior implementations in this domain have been dominated by CNN, LSTM and other methods despite the superior performance of GCN. To the best of our knowledge, we are the first to use a GCN model in child activity recognition domain. In overcoming the challenges of having small size publicly available child action datasets, several learning methods such as feature extraction, fine-tuning and curriculum learning were implemented to improve the model per…
University of Bucharest（布加勒斯特大学）；Department of Computer Science（计算机科学系）；Romania（罗马尼亚）
Curriculum learning techniques are a viable solution for improving the accuracy of automatic models, by replacing the traditional random training with an easy-to-hard strategy. However, the standard curriculum methodology does not automatically provide improved results, but it is constrained by multiple elements like the data distribution or the proposed model. In this paper, we introduce a novel curriculum sampling strategy which takes into consideration the diversity of the training data together with the difficulty of the inputs. We determine the difficulty using a state-of-the-art estimato…
We present four different robust transfer learning and data augmentation strategies for robust mobile scene recognition. By training three mobile-ready (EfficientNetB0, MobileNetV2, MobileNetV3) and two large-scale baseline (VGG16, ResNet50) convolutional neural network architectures on the widely available Event8, Scene15, Stanford40, and MIT67 datasets, we show the generalization ability of our transfer learning strategies. Furthermore, we tested the robustness of our transfer learning strategies under viewpoint and lighting changes using the KTH-Idol2 database. Also, the impact of inference…
Brown University（布朗大学）；Artificial and Natural Intelligence Toulouse Institute (ANITI)（图卢兹人工智能与自然智能研究所（ANITI））；Carney Institute for Brain Science（卡尼脑科学研究所）
The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant improvements in explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects. Across 84 different DNNs trained on ImageNet and three independe…
University of Oxford（牛津大学）；Oxford University Hospitals Trust（牛津大学医院信托）
Accurate and reliable prediction of hospital admission location is important due to resource-constraints and space availability in a clinical setting, particularly when dealing with patients who come from the emergency department. In this work we propose a student-teacher network via reinforcement learning to deal with this specific problem. A representation of the weights of the student network is treated as the state and is fed as an input to the teacher network. The teacher network's action is to select the most appropriate batch of data to train the student network on from a training set s…
