arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.27892stat.MLcs.LGstat.AP
收藏

通过专家混合的预测推理

Prediction-powered Inference by Mixture of Experts

Department of Mathematics, Hong Kong University of Science and Technology(香港理工大学数学系);Department of Mathematical and Statistical Sciences, University of Alberta(阿尔伯塔大学数学与统计学系)

Yanwu Gu, Linglong Kong, Dong Xia

AI总结:

本文提出基于预测的半监督推理框架,利用专家混合方法降低方差,提升预测精度,适用于多种统计估计任务。

摘要:

人工智能行业迅速发展,产生了多种强大的预测工具,每种工具都有独特的网络架构、训练策略和领域优势。这些工具为半监督推理提供了新机会,其中标记数据稀缺且昂贵,而未标记数据丰富易得。本文将预测器视为专家混合(MOE),提出基于预测的半监督推理框架。受PPI的方差减少原理启发,该框架寻找方差最小的专家混合。相比标准PPI,MOE框架能适应单个预测器的未知性能,利用集体预测能力,并提供最佳专家保证。框架适用于均值估计、线性回归、分位数估计和一般M估计。本文为MOE框架建立了非渐近理论,建立了置信区间覆盖误差的上界。数值实验验证了MOE推理的实用性,并支持了理论发现。

英文摘要:

The rapidly expanding artificial intelligence (AI) industry has produced diverse yet powerful prediction tools, each with its own network architecture, training strategy, data-processing pipeline, and domain-specific strengths. These tools create new opportunities for semi-supervised inference, in which labeled data are limited and expensive to obtain, whereas unlabeled data are abundant and widely available. Given a collection of predictors, we treat them as a mixture of experts (MOE) and introduce an MOE-powered semi-supervised inference framework built upon prediction-powered inference (PPI). Motivated by the variance reduction principle underlying PPI, the proposed framework seeks the mixture of experts that achieves the smallest possible variance. Compared with standard PPI, the MOE-powered inference framework adapts to the unknown performance of individual predictors, benefits from their collective predictive power, and enjoys a best-expert guarantee. The framework is flexible and applies to mean estimation, linear regression, quantile estimation, and general M-estimation. We develop non-asymptotic theory for the MOE-powered inference framework and establish upper bounds on the coverage error of the resulting confidence intervals. Numerical experiments demonstrate the practical effectiveness of MOE-powered inference and corroborate our theoretical findings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑