ELODI：用于正向一致训练的集成 Logit 差异抑制
ELODI: Ensemble Logit Difference Inhibition for Positive-Congruent Training
AI总结：
针对模型更新引入负翻转的问题，提出ELODI方法，通过将同质集成蒸馏为单模型并引入LDI目标仅惩罚高logit类别差异，在单模型推理成本下实现高准确率与低负翻转率。
摘要：
负翻转是指旧模型更新时分类系统引入的错误。现有的降低负翻转率（NFR）的方法，要么以牺牲整体准确率为代价，通过强制新模型模仿旧模型来实现，要么使用集成方法，这会导致推理成本呈难以承受的倍数增加。我们分析了集成方法在降低NFR中的作用，并观察到它们能够消除那些通常不靠近决策边界，但在其logit距离中往往表现出较大偏差的负翻转。基于这一观察，我们提出了一种名为Ensemble Logit Difference Inhibition（ELODI）的方法，用于训练分类系统，该系统在错误率和NFR方面均达到卓越性能，而推理成本仅为单模型。该方法将同质集成蒸馏为用于更新分类系统的单一学生模型。ELODI还引入了一种广义的蒸馏目标，即Logit Difference Inhibition（LDI），它仅对具有最高logit值的子集类别的logit差异进行惩罚。在多个图像分类基准上，使用ELODI进行模型更新展现出卓越的准确率保持和NFR降低效果。
英文摘要：
Negative flips are errors introduced in a classification system when a legacy model is updated. Existing methods to reduce the negative flip rate (NFR) either do so at the expense of overall accuracy by forcing a new model to imitate the old models, or use ensembles, which multiply inference cost prohibitively. We analyze the role of ensembles in reducing NFR and observe that they remove negative flips that are typically not close to the decision boundary, but often exhibit large deviations in the distance among their logits. Based on the observation, we present a method, called Ensemble Logit Difference Inhibition (ELODI), to train a classification system that achieves paragon performance in both error rate and NFR, at the inference cost of a single model. The method distills a homogeneous ensemble to a single student model which is used to update the classification system. ELODI also introduces a generalized distillation objective, Logit Difference Inhibition (LDI), which only penalizes the logit difference of a subset of classes with the highest logit values. On multiple image classification benchmarks, model updates with ELODI demonstrate superior accuracy retention and NFR reduction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
