Active feature acquisition (AFA) studies how to sequentially acquire features for each data instance to trade off predictive performance against acquisition cost. This survey offers the first unified treatment of AFA via an explicit partially observable Markov decision process (POMDP) formulation. We place this formulation in the broader literature on optimal information acquisition and, more specifically, in a family of structured POMDPs (for example, information-gathering and sensing POMDPs) whose assumptions and algorithmic tools directly apply to AFA. This connection provides a common language for comparing problem settings and methods, and it highlights where AFA can leverage established results in structured POMDP planning and approximation. Building on this perspective, we present an up-to-date taxonomy of AFA methods that (roughly) mirrors standard approaches to solving POMDPs: (i) embedded cost-aware predictors (notably cost-sensitive decision trees and ensembles), (ii) model-based methods that plan using learned probabilistic components, (iii) model-free methods that learn acquisition policies from simulated episodes, and (iv) hybrid methods that combine the strengths of model-based and model-free approaches. We argue that this POMDP-centric view clarifies connections among existing methods and motivates more principled algorithm design. Since much prior work is heuristic and lacks formal guarantees, we also outline routes to guarantees by connecting AFA to adaptive stochastic optimization. We conclude by highlighting open challenges and promising directions for future research.
We develop novel methodology for active feature acquisition (AFA), the study of how to sequentially acquire a dynamic (on a per instance basis) subset of features that minimizes acquisition costs whilst still yielding accurate predictions. The AFA framework can be useful in a myriad of domains, including health care applications where the cost of acquiring additional features for a patient (in terms of time, money, risk, etc.) can be weighed against the expected improvement to diagnostic performance. Previous approaches for AFA have employed either: deep learning RL techniques, which have diff…
We consider the problem of active feature acquisition, where we sequentially select the subset of features in order to achieve the maximum prediction performance in the most cost-effective way. In this work, we formulate this active feature acquisition problem as a reinforcement learning problem, and provide a novel framework for jointly learning both the RL agent and the classifier (environment). We also introduce a more systematic way of encoding subsets of features that can properly handle innate challenge with missing entries in active feature acquisition problems, that uses the orderless …
Non-Myopic Active Feature Acquisition via Pathwise Policy Gradients
Department of Computer Science and Department of Computer Science and Engineering（计算机科学系和计算机科学与工程系）；Chalmers University of Technology（楚姆勒斯技术大学）；University of Gothenburg（哥德堡大学）
We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task. While our goal could in principle be tackled by general reinforcement learning, our particular setting is constrained enough to allow more efficient algorithms. In this paper, we work under the Learning to Search framework and show how to formulate the goal of finding a dynamic information acquisition policy in that framework. We apply our formulation on two tasks, sentiment analysis and image recognition, and show that the learned policies exhibit good statistical p…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Many real-world situations allow for the acquisition of additional relevant information when making an assessment with limited or uncertain data. However, traditional ML approaches either require all features to be acquired beforehand or regard part of them as missing data that cannot be acquired. In this work, we consider models that perform active feature acquisition (AFA) and query the environment for unobserved features to improve the prediction assessments at evaluation time. Our work reformulates the Markov decision process (MDP) that underlies the AFA problem as a generative modeling ta…
Stochastic Encodings for Active Feature Acquisition
Department of Computer Science and Technology, University of Cambridge, Cambridge, United Kingdom（计算机科学与技术系，剑桥大学，剑桥，英国）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom（应用数学与理论物理系，剑桥大学，剑桥，英国）；Department of Statistics, University of Oxford, Oxford, United Kingdom（统计学系，牛津大学，牛津，英国）；Department of Artificial Intelligence, Korea University, Seoul, Korea（人工智能系，韩国大学，首尔，韩国）
Nanyang Technological University（南洋理工大学）；Microsoft Research（微软研究院）；University of Vienna（维也纳大学）
Solving real-life sequential decision making problems under partial observability involves an exploration-exploitation problem. To be successful, an agent needs to efficiently gather valuable information about the state of the world for making rewarding decisions. However, in real-life, acquiring valuable information is often highly costly, e.g., in the medical domain, information acquisition might correspond to performing a medical test on a patient. This poses a significant challenge for the agent to perform optimally for the task while reducing the cost for information acquisition. In this …
Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings
Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心——德国环境健康研究中心）；Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Fraunhofer Institute for Cognitive Systems IKS（弗劳恩霍夫认知系统研究所IKS）
Evaluation of Active Feature Acquisition Methods for Static Feature Settings
Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心——德国环境健康研究中心）；Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Fraunhofer Institute for Cognitive Systems IKS（弗劳恩霍夫认知系统研究所IKS）
University of Pennsylvania（宾夕法尼亚大学）；Cornell University（康奈尔大学）
In this paper, we propose a novel Reinforcement Learning approach for solving the Active Information Acquisition problem, which requires an agent to choose a sequence of actions in order to acquire information about a process of interest using on-board sensors. The classic challenges in the information acquisition problem are the dependence of a planning algorithm on known models and the difficulty of computing information-theoretic cost functions over arbitrary distributions. In contrast, the proposed framework of reinforcement learning does not require any knowledge on models and alleviates …
BRiG-AFA: Bellman Risk-to-Go Learning for Non-Myopic Active Feature Acquisition
Curtin Business School, Curtin University（科廷大学科廷商学院）；School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University（科廷大学电气、计算与数学科学学院）
We propose a general-purpose approach to discovering active learning (AL) strategies from data. These strategies are transferable from one domain to another and can be used in conjunction with many machine learning models. To this end, we formalize the annotation process as a Markov decision process, design universal state and action spaces and introduce a new reward function that precisely model the AL objective of minimizing the annotation cost. We seek to find an optimal (non-myopic) AL strategy using reinforcement learning. We evaluate the learned strategies on multiple unrelated domains a…
Active Jammer Localization via Acquisition-Aware Path Planning
1Dept. of Computer Science, University of Massachusetts Boston, Boston, MA, USA 2Dept. of Electrical \& Computer Engineering, Northeastern University, Boston, MA, USA
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Truly intelligent systems are expected to make critical decisions with incomplete and uncertain data. Active feature acquisition (AFA), where features are sequentially acquired to improve the prediction, is a step towards this goal. However, current AFA models all deal with a small set of candidate features and have difficulty scaling to a large feature space. Moreover, they are ignorant about the valid domains where they can predict confidently, thus they can be vulnerable to out-of-distribution (OOD) inputs. In order to remedy these deficiencies and bring AFA models closer to practical use, …
Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high. Besides active learning, data augmentation is also an effective technique to enlarge the limited amount of labeled instances. However, the potential gain from virtual instances generated by data augmentation has not been considered in the acquisition process of active learning yet. Looking ahead the effect of data augmentation in the process of acquisition would select and generate the data instances that are informative for training the mod…
University of Oxford（牛津大学）；OATML；Department of Computer Science（计算机科学系）
We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian active learning, as an expansion of BatchBALD that takes into account an evaluation set of unlabeled data, for example, the pool set. We also develop a variant for the non-Bayesian setting, which we call Evaluation Information Gain. To reduce computational requirements and allow these methods to scale to larger acquisition batch sizes, we introduce stochastic acquisition functions that use importance sampling of tempered acquisition scores. We call this method PowerEvaluationBALD. We show in a few initial experiments …
We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning. BatchBALD is a greedy linear-time $1 - \frac{1}{e}$-approximate algorithm amenable to dynamic programming and efficient caching. We compare BatchBALD to the commonly used approach for batch data acquisition and find that the current approach acquires similar and redundant points, sometimes performing worse than randomly acquiring data. We fini…
Amortized Safe Active Learning for Real-Time Data Acquisition: Pretrained Neural Policies From Simulated Nonparametric Functions
Technical University of Berlin（柏林工业大学）；University of Helsinki（赫尔辛基大学）；Robotics Institute Germany（德国机器人研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）；Baden-Württemberg Cooperative State University, Mannheim（巴登-符腾堡州立合作大学曼海姆校区）
Active Learning Using Aggregated Acquisition Functions: Accuracy and Sustainability Analysis
Chair of Information Theory and Data Analytics (INDA), RWTH Aachen University（信息理论与数据 analytics 教授会（INDA），亚琛工业大学）；Automation and Control Institute, Technische Universität Wien (TUW)（自动化与控制研究所，维也纳技术大学）；AIT Austrian Institute of Technology GmbH（奥地利技术研究所（AIT））
Berlin Institute of Health, Charité - Universitätsmedizin Berlin（柏林健康研究所，柏林查理大学）；Intelligent Medicine Institute, Fudan University（智能医学研究院，复旦大学）；Department of Mathematics and Computer Science, Freie Universität Berlin（数学与计算机科学系，柏林自由大学）
Active learning is an established technique to reduce the labeling cost to build high-quality machine learning models. A core component of active learning is the acquisition function that determines which data should be selected to annotate. State-of-the-art acquisition functions -- and more largely, active learning techniques -- have been designed to maximize the clean performance (e.g. accuracy) and have disregarded robustness, an important quality property that has received increasing attention. Active learning, therefore, produces models that are accurate but not robust. In this paper, we …
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；RIKEN Center for AIP（理化学研究所人工智能项目中心）；University of Tokyo（东京大学）
Feature missing is a serious problem in many applications, which may lead to low quality of training data and further significantly degrade the learning performance. While feature acquisition usually involves special devices or complex process, it is expensive to acquire all feature values for the whole dataset. On the other hand, features may be correlated with each other, and some values may be recovered from the others. It is thus important to decide which features are most informative for recovering the other features as well as improving the learning performance. In this paper, we try to …
Heidelberg University（海德堡大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）
Model selection is treated as a standard performance boosting step in many machine learning applications. Once all other properties of a learning problem are fixed, the model is selected by grid search on a held-out validation set. This is strictly inapplicable to active learning. Within the standardized workflow, the acquisition function is chosen among available heuristics a priori, and its success is observed only after the labeling budget is already exhausted. More importantly, none of the earlier studies report a unique consistently successful acquisition heuristic to the extent to stand …
Imaging in clinical routine is subject to changing scanner protocols, hardware, or policies in a typically heterogeneous set of acquisition hardware. Accuracy and reliability of deep learning models suffer from those changes as data and targets become inconsistent with their initial static training set. Continual learning can adapt to a continuous data stream of a changing imaging environment. Here, we propose a method for continual active learning on a data stream of medical images. It recognizes shifts or additions of new imaging sources - domains -, adapts training accordingly, and selects …
Active Transfer Bagging: A New Approach for Accelerated Active Learning Acquisition of Data by Combined Transfer Learning and Bagging Based Models
Chemical Engineering School for the Engineering of Matter, Transport, & Energy Arizona State University（材料与运输工程学院，亚利桑那州立大学）；Materials Science & Engineering School for the Engineering of Matter, Transport, & Energy Arizona State University（材料科学与工程学院，亚利桑那州立大学）
Ordering the selection of training data using active learning can lead to improvements in learning efficiently from smaller corpora. We present an exploration of active learning approaches applied to three grounded language problems of varying complexity in order to analyze what methods are suitable for improving data efficiency in learning. We present a method for analyzing the complexity of data in this joint problem space, and report on how characteristics of the underlying task, along with design decisions such as feature selection and classification model, drive the results. We observe th…
We present a novel adaptation of active learning to graph-based semi-supervised learning (SSL) under non-Gaussian Bayesian models. We present an approximation of non-Gaussian distributions to adapt previously Gaussian-based acquisition functions to these more general cases. We develop an efficient rank-one update for applying "look-ahead" based methods as well as model retraining. We also introduce a novel "model change" acquisition function based on these approximations that further expands the available collection of active learning acquisition functions for such methods.
Active learning typically focuses on training a model on few labeled examples alone, while unlabeled ones are only used for acquisition. In this work we depart from this setting by using both labeled and unlabeled data during model training across active learning cycles. We do so by using unsupervised feature learning at the beginning of the active learning pipeline and semi-supervised learning at every active learning cycle, on all available data. The former has not been investigated before in active learning, while the study of latter in the context of deep learning is scarce and recent find…
Modern deep learning has achieved great success in various fields. However, it requires the labeling of huge amounts of data, which is expensive and labor-intensive. Active learning (AL), which identifies the most informative samples to be labeled, is becoming increasingly important to maximize the efficiency of the training process. The existing AL methods mostly use only a single final fixed model for acquiring the samples to be labeled. This strategy may not be good enough in that the structural uncertainty of a model for given training data is not considered to acquire the samples. In this…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Technology Innovation Institute（技术创新研究院）
In deep active learning, it is especially important to choose multiple examples to markup at each step to work efficiently, especially on large datasets. At the same time, existing solutions to this problem in the Bayesian setup, such as BatchBALD, have significant limitations in selecting a large number of examples, associated with the exponential complexity of computing mutual information for joint random variables. We, therefore, present the Large BatchBALD algorithm, which gives a well-grounded approximation to the BatchBALD method that aims to achieve comparable quality while being more c…
Machine learning in medical imaging during clinical routine is impaired by changes in scanner protocols, hardware, or policies resulting in a heterogeneous set of acquisition settings. When training a deep learning model on an initial static training set, model performance and reliability suffer from changes of acquisition characteristics as data and targets may become inconsistent. Continual learning can help to adapt models to the changing environment by training on a continuous data stream. However, continual manual expert labelling of medical imaging requires substantial effort. Thus, ways…
In this paper, we consider active information acquisition when the prediction model is meant to be applied on a targeted subset of the population. The goal is to label a pre-specified fraction of customers in the target or test set by iteratively querying for information from the non-target or training set. The number of queries is limited by an overall budget. Arising in the context of two rather disparate applications- banking and medical diagnosis, we pose the active information acquisition problem as a constrained optimization problem. We propose two greedy iterative algorithms for solving…
AFABench: A Generic Framework for Benchmarking Active Feature Acquisition
Chalmers University of Technology \& University of Gothenburg Sweden；Chalmers University of Technology Sweden；Chalmers University of Technology \& University of Gothenburg；Chalmers University of Technology
OATML（牛津大学机器学习与应用实验室）；University of Oxford（牛津大学）；Machine Learning and Causal Inference in Healthcare Lab（医疗保健机器学习与因果推理实验室）；Technion（以色列理工学院）
Estimating personalized treatment effects from high-dimensional observational data is essential in situations where experimental designs are infeasible, unethical, or expensive. Existing approaches rely on fitting deep models on outcomes observed for treated and control populations. However, when measuring individual outcomes is costly, as is the case of a tumor biopsy, a sample-efficient strategy for acquiring each result is required. Deep Bayesian active learning provides a framework for efficient data acquisition by selecting points with high uncertainty. However, existing methods bias trai…
