发表机构
University of Wisconsin–Madison（威斯康星大学麦迪逊分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出Frontier Learning框架，将预训练模型库视为互补信息源，通过拼接白盒内部表示与黑盒预测输出构建特征，在分布偏移场景中提升预测性能，表现优于或相当于最强的单个复用策略。
AI 中文摘要
现代机器学习流程越来越依赖在下游任务中复用预训练模型和基础模型。这些预训练模型不仅性能存在差异，使用方式也不同：一些仅提供黑盒预测，另一些则允许白盒访问可探测或微调的内部表示。当部署到存在分布偏移的目标域时，没有单一策略（包括零样本应用、微调或直接训练目标特定模型）始终最优。本研究提出了Frontier Learning框架，该框架将跨越不同训练历史和访问机制的候选模型库视为互补信息源，而非互斥选项。Frontier Learning通过将白盒候选模型的内部表示与黑盒候选模型的预测输出拼接，构建统一的目标域特征，随后使用带标签的目标数据在该拼接表示上拟合轻量级正则化监督学习器。由于所得假设类包含通过零样本复用、微调及直接训练得到的预测器作为特例，对前沿学习器的经验风险最小化在训练样本上的表现保证不逊于任何单个基线。我们在涵盖不同源-目标兼容性的模拟中，以及两个真实世界分布偏移场景（DomainNet/VisDA上的视觉域适应、使用MIMIC-IV-Notes的重症监护室域间临床死亡率预测）中评估了该框架。在所有场景中，Frontier Learning的表现与最强的单个复用策略相当或更优，最大增益恰好出现在所考虑的偏移范围内无单个基线可靠时。
英文摘要
Modern machine learning pipelines increasingly rely on reusing pretrained and foundation models across downstream tasks. These pretrained models can differ not only in performance but also in how they can be used: some only provide black-box predictions, while others may permit white-box access to internal representations that can be probed or fine-tuned. When deployed to the target domain in the presence of distribution shift, no single strategy, including zero-shot application, fine-tuning, or directly training a target-specific model, is uniformly the best. In this work, we propose Frontier Learning, a framework that treats a library of candidate models spanning different training histories and access regimes as complementary sources of information rather than mutually exclusive alternatives. Frontier Learning constructs a unified target-domain feature by concatenating internal representations from white-box candidates as well as prediction outputs from black-box candidates, then fits a lightweight, regularized supervised learner on this concatenated representation using labeled target data. Because the resulting hypothesis class contains predictors obtained by zero-shot reuse, fine-tuning, and direct training as special cases, empirical risk minimization over the frontier learner is guaranteed to be no worse, on the training sample, than any individual baseline. We evaluate the framework in simulations spanning varying degrees of source-target compatibility and in two real-world distribution-shift settings: visual domain adaptation on DomainNet/VisDA and clinical mortality prediction across intensive care unit domains using MIMIC-IV-Notes. Across all settings, Frontier Learning matches or outperforms the strongest individual reuse strategy, with the largest gains arising precisely when no single baseline is reliable across the range of shift considered.
Comments28 pages, 4 figures, 3 tables
