通过OPAL进行预测辅助推断的优化标注资源分配
Optimized Labeling Resource Allocation for Prediction-Assisted Inference via OPAL
AI总结：
提出OPAL方法，通过可学习的平滑策略分配标注资源，以最小化估计方差，实现预测辅助推断中的高效标注和统计推断。
摘要：
主动统计推断是一个新框架，能够对总体参数做出具有可证明统计保证的精确声明。它利用预测性“黑箱”机器学习模型策略性地决定标注哪些数据点，大致优先考虑ML模型对其标签值不确定的样本。一个主要问题是，当不确定性估计存在噪声时，该框架可能变得脆弱。本文介绍了OPAL（标注分配优化策略），它在可处理的平滑策略类中学习标注策略，以产生方差最小的估计量。实际上，OPAL是一个端到端的流程，将黑箱模型的不确定性得分转化为数据自适应的标注策略，然后对收集的样本进行推断。我们在涵盖医学影像数据、计算社会科学和蛋白质组学的真实数据集上评估了OPAL。作为一个具体例子，我们考虑从组织病理学图像预测乳腺癌亚型，并使用OPAL为不同人口统计组的比值比形成有效的置信区间。我们表明，OPAL在有限样本中实现了名义覆盖，并具有人们期望从拥有更多标注样本的方法中获得的准确性。
英文摘要：
Active Statistical Inference is a new framework to make precise claims about population parameters with provable statistical guarantees. It uses a predictive "black-box" machine learning (ML) model to strategically decide which data points to label, roughly prioritizing samples for which the ML model is unsure about their label values. A major issue is that the framework can be brittle when uncertainty estimates are noisy. This paper introduces OPAL (Optimized Policy for Allocation of Labels), which learns a labeling strategy within a tractable class of smooth policies to yield estimators with the lowest variance. In effect, OPAL is an end-to-end pipeline that turns a black-box model's uncertainty scores into a data-adaptive labeling strategy and then performs inference on the collected samples. We evaluate OPAL on real datasets spanning medical imaging data, computational social science, and proteomics. As a concrete example, we consider predicting breast cancer subtype from histopathology images and using OPAL to form valid confidence intervals for odds ratios for different demographic groups. We show that OPAL achieves nominal coverage in finite samples and has the accuracy one expects from methods which have far more labeled samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
