We propose a reinforcement learning based approach to tackle the cost-sensitive learning problem where each input feature has a specific cost. The acquisition process is handled through a stochastic policy which allows features to be acquired in an adaptive way. The general architecture of our approach relies on representation learning to enable performing prediction on any partially observed sample, whatever the set of its observed features are. The resulting model is an original mix of representation learning and of reinforcement learning ideas. It is learned with policy gradient techniques to minimize a budgeted inference cost. We demonstrate the effectiveness of our proposed method with several experiments on a variety of datasets for the sparse prediction problem where all features have the same cost, but also for some cost-sensitive settings.
We develop novel methodology for active feature acquisition (AFA), the study of how to sequentially acquire a dynamic (on a per instance basis) subset of features that minimizes acquisition costs whilst still yielding accurate predictions. The AFA framework can be useful in a myriad of domains, including health care applications where the cost of acquiring additional features for a patient (in terms of time, money, risk, etc.) can be weighed against the expected improvement to diagnostic performance. Previous approaches for AFA have employed either: deep learning RL techniques, which have diff…
Nanyang Technological University（南洋理工大学）；Microsoft Research（微软研究院）；University of Vienna（维也纳大学）
Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem. To be successful, an agent needs to efficiently gather valuable information about the state of the world for making rewarding decisions. However, in real-life, acquiring valuable information is often highly costly, e.g., in the medical domain, information acquisition might correspond to performing a medical test on a patient. This poses a significant challenge for the agent to perform optimally for the task while reducing the cost for information acquisition. In this …
We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task. While our goal could in principle be tackled by general reinforcement learning, our particular setting is constrained enough to allow more efficient algorithms. In this paper, we work under the Learning to Search framework and show how to formulate the goal of finding a dynamic information acquisition policy in that framework. We apply our formulation on two tasks, sentiment analysis and image recognition, and show that the learned policies exhibit good statistical p…
We consider the problem of active feature acquisition, where we sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way. In this work, we formulate this active feature acquisition problem as a reinforcement learning problem, and provide a novel framework for jointly learning both the RL agent and the classifier (environment). We also introduce a more systematic way of encoding subsets of features that can properly handle innate challenge with missing entries in active feature acquisition problems, that uses the orderless …
Stochastic Encodings for Active Feature Acquisition
Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom（计算机科学与技术系，剑桥大学，剑桥，英国）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom（应用数学与理论物理系，剑桥大学，剑桥，英国）；Department of Statistics, University of Oxford, Oxford, United Kingdom（统计学系，牛津大学，牛津，英国）；Department of Artificial Intelligence, Korea University, Seoul, Korea（人工智能系，韩国大学，首尔，韩国）
Reinforcement Learning of Adaptive Acquisition Policies for Inverse Problems
Donders Institute for Brain, Cognition and Behaviour, Radboud University（拉德堡德大学唐德斯脑、认知与行为研究所）；OnePlanet Research Center, imec-the Netherlands（荷兰OnePlanet研究中心、imec荷兰）；Qualcomm AI Research（高通人工智能研究）；Qualcomm Wireless GmbH（高通无线有限公司）；Qualcomm Technologies, Inc.（高通技术公司）
University of California, Los Angeles (UCLA)（加利福尼亚大学洛杉矶分校（UCLA））
Traditionally, machine learning algorithms rely on the assumption that all features of a given dataset are available for free. However, there are many concerns such as monetary data collection costs, patient discomfort in medical procedures, and privacy impacts of data collection that require careful consideration in any real-world health analytics system. An efficient solution would only acquire a subset of features based on the value it provides while considering acquisition costs. Moreover, datasets that provide feature costs are very limited, especially in healthcare. In this paper, we pro…
Hyperparameter tuning is an omnipresent problem in machine learning as it is an integral aspect of obtaining the state-of-the-art performance for any model. Most often, hyperparameters are optimized just by training a model on a grid of possible hyperparameter values and taking the one that performs best on a validation sample (grid search). More recently, methods have been introduced that build a so-called surrogate model that predicts the validation loss for a specific hyperparameter setting, model and dataset and then sequentially select the next hyperparameter to test, based on a heuristic…
CRPS-Based Targeted Sequential Design with Application in Chemical Space
Institute of Mathematical Statistics and Actuarial Science, University of Bern, Switzerland（数学统计与精算科学研究所，伯尔尼大学，瑞士）；DTIS, ONERA, Paris-Saclay University（DTIS，ONERA，巴黎-萨克雷大学）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Truly intelligent systems are expected to make critical decisions with incomplete and uncertain data. Active feature acquisition (AFA), where features are sequentially acquired to improve the prediction, is a step towards this goal. However, current AFA models all deal with a small set of candidate features and have difficulty scaling to a large feature space. Moreover, they are ignorant about the valid domains where they can predict confidently, thus they can be vulnerable to out-of-distribution (OOD) inputs. In order to remedy these deficiencies and bring AFA models closer to practical use, …
In many real-world machine learning problems, feature values are not readily available. To make predictions, some of the missing features have to be acquired, which can incur a cost in money, computational time, or human time, depending on the problem domain. This leads us to the problem of choosing which features to use at the prediction time. The chosen features should increase the prediction accuracy for a low cost, but determining which features will do that is challenging. The choice should take into account the previously acquired feature values as well as the feature costs. This paper p…
HPOQuest: A Rare-Disease Diagnostic Agent Using Active Phenotype Acquisition
Technical University of Munich（慕尼黑工业大学）；Helmholtz Center Munich（亥姆霍兹慕尼黑中心）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）；Aalto University（阿尔托大学）
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Collaborative Innovation Center of Novel Software Technology and Industrialization（新型软件技术与产业化协同创新中心）；State Key Laboratory of Aerodynamics（空气动力学国家重点实验室）
In Bayesian optimization (BO) for expensive black-box optimization tasks, acquisition function (AF) guides sequential sampling and plays a pivotal role for efficient convergence to better optima. Prevailing AFs usually rely on artificial experiences in terms of preferences for exploration or exploitation, which runs a risk of a computational waste or traps in local optima and resultant re-optimization. To address the crux, the idea of data-driven AF selection is proposed, and the sequential AF selection task is further formalized as a Markov decision process (MDP) and resort to powerful reinfo…
Centre Suisse d’Electronique et de Microtechnique (CSEM)（瑞士电子与微技术中心（CSEM））；École polytechnique fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
Preserving privacy is a growing concern in our society where sensors and cameras are ubiquitous. In this work, for the first time, we propose a trainable image acquisition method that removes the sensitive identity revealing information in the optical domain before it reaches the image sensor. The method benefits from a trainable optical convolution kernel which transmits the desired information while filters out the sensitive content. As the sensitive content is suppressed before it reaches the image sensor, it does not enter the digital domain therefore is unretrievable by any sort of privac…
Amortized Safe Active Learning for Real-Time Data Acquisition: Pretrained Neural Policies From Simulated Nonparametric Functions
Technical University of Berlin（柏林工业大学）；University of Helsinki（赫尔辛基大学）；Robotics Institute Germany（德国机器人研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）；Baden-Württemberg Cooperative State University, Mannheim（巴登-符腾堡州立合作大学曼海姆校区）
Active Learning Using Aggregated Acquisition Functions: Accuracy and Sustainability Analysis
Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University（信息理论与数据 analytics 教授会（INDA），亚琛工业大学）；Automation and Control Institute, Technische Universität Wien (TUW)（自动化与控制研究所，维也纳技术大学）；AIT Austrian Institute of Technology GmbH（奥地利技术研究所（AIT））
NOCTA: Non-Greedy Objective Cost-Tradeoff Acquisition for Longitudinal Data
Department of Computer Science, UNC Chapel Hill（UNC夏洛特山分校计算机科学系）；Department of Computer Science（计算机科学系）；Engineering, University of California San Diego（工程系，加州大学圣地亚哥分校）
Millimeter wave (mmWave) communication with large antenna arrays is a promising technique to enable extremely high data rates due to the large available bandwidth in mmWave frequency bands. In addition, given the knowledge of an optimal directional beamforming vector, large antenna arrays have been shown to overcome both the severe signal attenuation in mmWave as well as the interference problem. However, fundamental limits on achievable learning rate of an optimal beamforming vector remain. This paper considers the problem of adaptive and sequential optimization of the beamforming vectors dur…
University of Tübingen（蒂宾根大学）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Amazon Research（亚马逊研究院）
Bayesian quadrature (BQ) is a sample-efficient probabilistic numerical method to solve integrals of expensive-to-evaluate black-box functions, yet so far,active BQ learning schemes focus merely on the integrand itself as information source, and do not allow for information transfer from cheaper, related functions. Here, we set the scene for active learning in BQ when multiple related information sources of variable cost (in input and source) are accessible. This setting arises for example when evaluating the integrand requires a complex simulation to be run that can be approximated by simulati…
Stanford University（斯坦福大学）；Toyota Research Institute（丰田研究所）
This paper presents a novel online framework for safe crowd-robot interaction based on risk-sensitive stochastic optimal control, wherein the risk is modeled by the entropic risk measure. The sampling-based model predictive control relies on mode insertion gradient optimization for this risk measure as well as Trajectron++, a state-of-the-art generative model that produces multimodal probabilistic trajectory forecasts for multiple interacting agents. Our modular approach decouples the crowd-robot interaction into learning-based prediction and model-based control, which is advantageous compared…
Relevant and high-quality data are critical to successful development of machine learning applications. For machine learning applications on dynamic systems equipped with a large number of sensors, such as connected vehicles and robots, how to find relevant and high-quality data features in an efficient way is a challenging problem. In this work, we address the problem of feature selection in constrained continuous data acquisition. We propose a feedback-based dynamic feature selection algorithm that efficiently decides on the feature set for data collection from a dynamic system in a step-wis…
In supervised learning, acquiring labeled training data for a predictive model can be very costly, but acquiring a large amount of unlabeled data is often quite easy. Active learning is a method of obtaining predictive models with high precision at a limited cost through the adaptive selection of samples for labeling. This paper explains the basic problem settings of active learning and recent research trends. In particular, research on learning acquisition functions to select samples from the data for labeling, theoretical work on active learning algorithms, and stopping criteria for sequenti…
Fudan University（复旦大学）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
Bayesian optimization is a promising methodology for analog circuit synthesis. However, the sequential nature of the Bayesian optimization framework significantly limits its ability to fully utilize real-world computational resources. In this paper, we propose an efficient parallelizable Bayesian optimization algorithm via Multi-objective ACquisition function Ensemble (MACE) to further accelerate the optimization procedure. By sampling query points from the Pareto front of the probability of improvement (PI), expected improvement (EI) and lower confidence bound (LCB), we combine the benefits o…
ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosis
Washington University in St. Louis（圣路易斯华盛顿大学）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Arizona State University（亚利桑那州立大学）；Yale University（耶鲁大学）；Amazon（亚马逊公司）
