arXiv 2509.005092025-08-30cs.CV

让我成为专家：面向语义分割的通用黑盒模型到专用模型的蒸馏

Make me an Expert: Distilling from Generalist Black-Box Models into Specialized Models for Semantic Segmentation

LTCI, Télécom-Paris, Institut Polytechnique de Paris（LTCI， Télécom-Paris， Institut Polytechnique de Paris）；LIX, Ecole Polytechnique, CNRS, Institut Polytechnique de Paris（LIX， Ecole Polytechnique， CNRS， Institut Polytechnique de Paris）；University of Bergamo（博洛尼亚大学）；NVIDIA；Inria at University Grenoble Alpes, LJK（Inria 于格勒堡阿尔卑斯大学，LJK）

Yasser Benigmim, Subhankar Roy, Khalid Oublal, Imad Eddine Marouf, Slim Essid, Vicky Kalogeiton, Stéphane Lathuilière

备注：: Github repo : https://github.com/yasserben/ATGC

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对AIaaS下黑盒API模型仅输出one-hot预测、无法用传统域适应训练本地专用模型的问题，提出B2D设置与ATGC方法，利用DINOv2注意力图动态选最优尺度蒸馏，在多数据集上实现显著提升。

摘要：

人工智能即服务（AIaaS）的兴起通过应用程序编程接口（API）降低了预训练模型的使用门槛，但也提出了一个根本问题：如何利用不公开权重、训练数据或logits的黑盒模型有效训练本地模型？在这一约束下，现有域适应范式均不适用。为应对这一挑战，我们提出黑盒蒸馏（Black-Box Distillation，B2D）设置，可在现实约束下实现本地模型适配：（1）API模型为开放词汇模型，基于大规模通用数据训练；（2）仅能获取one-hot预测结果。我们发现开放词汇模型对输入分辨率存在显著敏感性，不同物体类别在不同尺度下可获得最优分割效果，这一局限被称为“分辨率诅咒”。我们提出的方法——注意力引导尺度选择器（ATtention-Guided sCaler，ATGC），利用DINOv2注意力图动态选择黑盒模型推理的最优尺度，以此解决上述问题。ATGC通过熵对注意力图打分，识别适合生成伪标签的高信息量尺度，从而实现有效蒸馏。实验表明，在仅依赖one-hot API预测的黑盒监督下，该方法在多个数据集上取得了显著性能提升。我们的代码可在https://github.com/yasserben/ATGC 获取。

英文摘要：

The rise of Artificial Intelligence as a Service (AIaaS) democratizes access to pre-trained models via Application Programming Interfaces (APIs), but also raises a fundamental question: how can local models be effectively trained using black-box models that do not expose their weights, training data, or logits, a constraint in which current domain adaptation paradigms are impractical ? To address this challenge, we introduce the Black-Box Distillation (B2D) setting, which enables local model adaptation under realistic constraints: (1) the API model is open-vocabulary and trained on large-scale general-purpose data, and (2) access is limited to one-hot predictions only. We identify that open-vocabulary models exhibit significant sensitivity to input resolution, with different object classes being segmented optimally at different scales, a limitation termed the "curse of resolution". Our method, ATtention-Guided sCaler (ATGC), addresses this challenge by leveraging DINOv2 attention maps to dynamically select optimal scales for black-box model inference. ATGC scores the attention maps with entropy to identify informative scales for pseudo-labelling, enabling effective distillation. Experiments demonstrate substantial improvements under black-box supervision across multiple datasets while requiring only one-hot API predictions. Our code is available at https://github.com/yasserben/ATGC.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。