Prioritized experience replay (PER) samples important transitions, rather than uniformly, to improve the performance of a deep reinforcement learning agent. We claim that such prioritization has to be balanced with sample diversity for making the DQN stabilized and preventing forgetting. Our proposed improvement over PER, called Predictive PER (PPER), takes three countermeasures (TDInit, TDClip, TDPred) to (i) eliminate priority outliers and explosions and (ii) improve the sample diversity and distributions, weighted by priorities, both leading to stabilizing the DQN. The most notable among the three is the introduction of the second DNN called TDPred to generalize the in-distribution priorities. Ablation study and full experiments with Atari games show that each countermeasure by its own way and PPER contribute to successfully enhancing stability and thus performance over PER.
ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing Institute of Software Technology（南京软件技术研究院）
Department of Electrical Engineering and Computer Science, Pennsylvania State University, PA, USA（电气工程与计算机科学系，宾夕法尼亚州立大学）；College of Information Sciences and Technology, Pennsylvania State University, PA, USA（信息科学与技术学院，宾夕法尼亚州立大学）
Design of recommender systems aimed at achieving high prediction accuracy is a widely researched area. However, several studies have suggested the need for diversified recommendations, with acceptable level of accuracy, to avoid monotony and improve customers experience. However, increasing diversity comes with an associated reduction in recommendation accuracy; thereby necessitating an optimum tradeoff between the two. In this work, we attempt to achieve accuracy vs diversity balance, by exploiting available ratings and item metadata, through a single, joint optimization model built over the …
University of New South Wales（新南威尔士大学）；Jiangnan University（江南大学）；University of Technology Sydney（悉尼科技大学）；University of California San Diego（加州大学圣地亚哥分校）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）
Zero-Shot Learning (ZSL) aims to transfer classification capability from seen to unseen classes. Recent methods have proved that generalization and specialization are two essential abilities to achieve good performance in ZSL. However, focusing on only one of the abilities may result in models that are either too general with degraded classification ability or too specialized to generalize to unseen classes. In this paper, we propose an end-to-end network, termed as BGSNet, which equips and balances generalization and specialization abilities at the instance and dataset level. Specifically, BG…
Balancing Optimality and Diversity: Human-Centered Decision Making through Generative Curation
Technology and Operations Management, Harvard Business School（哈佛商学院技术与运营管理系）；Heinz College of Information Systems and Public Policy, Carnegie Mellon University（卡内基梅隆大学海因兹学院）
Deep learning has yielded extraordinary results in vision and natural language processing, but this achievement comes at a cost. Most models require enormous resources during training, both in terms of computation and in human labeling effort. We show that we can identify informative and diverse subsets of data that lead to deep learning models with similar performance as the ones trained with the original dataset. Prior methods have exploited diversity and uncertainty in submodular objective functions for choosing subsets. In addition to these measures, we show that balancing constraints on p…
In many important machine learning applications, the training distribution used to learn a probabilistic classifier differs from the testing distribution on which the classifier will be used to make predictions. Traditional methods correct the distribution shift by reweighting the training data with the ratio of the density between test and training data. In many applications training takes place without prior knowledge of the testing distribution on which the algorithm will be applied in the future. Recently, methods have been proposed to address the shift by learning causal structure, but th…
