Task Uncertainty Loss Reduce Negative Transfer in Asymmetric Multi-task Feature Learning
National University of Singapore（新加坡国立大学）；Genome Institute of Singapore, A*STAR（新加坡科技研究局下属基因组研究所）；Yong Loo Lin School of Medicine, National University of Singapore（新加坡国立大学杨潞龄医学院）
Rafael Peres da Silva, Chayaporn Suphavilai, Niranjan Nagarajan
备注：
Accepted in AAAI 2021 Student Abstract and Poster Program
Multi-task learning (MTL) is frequently used in settings where a target task has to be learnt based on limited training data, but knowledge can be leveraged from related auxiliary tasks. While MTL can improve task performance overall relative to single-task learning (STL), these improvements can hide negative transfer (NT), where STL may deliver better performance for many individual tasks. Asymmetric multitask feature learning (AMTFL) is an approach that tries to address this by allowing tasks with higher loss values to have smaller influence on feature representations for learning other tasks. Task loss values do not necessarily indicate reliability of models for a specific task. We present examples of NT in two orthogonal datasets (image recognition and pharmacogenomics) and tackle this challenge by using aleatoric homoscedastic uncertainty to capture the relative confidence between tasks, and set weights for task loss. Our results show that this approach reduces NT providing a new approach to enable robust MTL.
Centre for Mathematical Morphology, Mines ParisTech, PSL Research University（巴黎文理研究大学巴黎高等矿业学院数学形态学中心）；LTCI, Télécom ParisTech, Université Paris-Saclay（巴黎萨克雷大学巴黎高等电信学院LTCI实验室）
This paper addresses the issue of building a part-based representation of a dataset of images. More precisely, we look for a non-negative, sparse decomposition of the images on a reduced set of atoms, in order to unveil a morphological and interpretable structure of the data. Additionally, we want this decomposition to be computed online for any new sample that is not part of the initial dataset. Therefore, our solution relies on a sparse, non-negative auto-encoder where the encoder is deep (for accuracy) and the decoder shallow (for interpretability). This method compares favorably to the sta…
Bottlenecks of binary classification from positive and unlabeled data (PU classification) are the requirements that given unlabeled patterns are drawn from the test marginal distribution, and the penalty of the false positive error is identical to the false negative error. However, such requirements are often not fulfilled in practice. In this paper, we generalize PU classification to the class prior shift and asymmetric error scenarios. Based on the analysis of the Bayes optimal classifier, we show that given a test class prior, PU classification under class prior shift is equivalent to PU cl…
In a typical multi-label setting, a picture contains on average few positive labels, and many negative ones. This positive-negative imbalance dominates the optimization process, and can lead to under-emphasizing gradients from positive labels during training, resulting in poor accuracy. In this paper, we introduce a novel asymmetric loss ("ASL"), which operates differently on positive and negative samples. The loss enables to dynamically down-weights and hard-thresholds easy negative samples, while also discarding possibly mislabeled samples. We demonstrate how ASL can balance the probabilitie…
We propose Deep Asymmetric Multitask Feature Learning (Deep-AMTFL) which can learn deep representations shared across multiple tasks while effectively preventing negative transfer that may happen in the feature sharing process. Specifically, we introduce an asymmetric autoencoder term that allows reliable predictors for the easy tasks to have high contribution to the feature learning while suppressing the influences of unreliable predictors for more difficult tasks. This allows the learning of less noisy representations, and enables unreliable predictors to exploit knowledge from the reliable …
Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layer
National Key Laboratory of Radar Signal Processing, Xidian University（雷达信号处理国家重点实验室，西安电子科技大学）；School of Mathematics and Statistics, Xi’an Jiaotong University（数学与统计学院，西安交通大学）；McCombs School of Business, The University of Texas at Austin（商学院，德克萨斯大学奥斯汀分校）
Affinity Uncertainty-based Hard Negative Mining in Graph Contrastive Learning
Australian Artificial Intelligence Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）
We introduce CAMRL, the first curriculum-based asymmetric multi-task learning (AMTL) algorithm for dealing with multiple reinforcement learning (RL) tasks altogether. To mitigate the negative influence of customizing the one-off training order in curriculum-based AMTL, CAMRL switches its training mode between parallel single-task RL and asymmetric multi-task RL (MTRL), according to an indicator regarding the training time, the overall performance, and the performance gap among tasks. To leverage the multi-sourced prior knowledge flexibly and to reduce negative transfer in AMTL, we customize a …
TAM-Chain: Multi-Scale Thyroid Cytology Classification via Absorbing Markov Chains and Shannon Entropy Uncertainty Quantification for False-Negative Suppression and Domain-Shift Adaptation
Faculty of Mathematics, Mechanics and Informatics, VNU University of Science（越南国立大学理学院数学、力学与信息学系）
Although recent multi-task learning methods have shown to be effective in improving the generalization of deep neural networks, they should be used with caution for safety-critical applications, such as clinical risk prediction. This is because even if they achieve improved task-average performance, they may still yield degraded performance on individual tasks, which may be critical (e.g., prediction of mortality risk). Existing asymmetric multi-task learning methods tackle this negative transfer problem by performing knowledge transfer from tasks with low loss to tasks with high loss. However…
