arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2404.06972cs.LGcs.AI

迈向持续学习的工业应用:面向类别增量学习的新指标提案

Toward industrial use of continual learning : new metrics proposal for class incremental learning

Michelin(米其林);Université Clermont Auvergne(克莱蒙奥弗涅大学);Clermont Auvergne INP(克莱蒙奥弗涅国立综合理工学院);CNRS(法国国家科学研究中心);Institut Pascal(帕斯卡尔研究所);Laboratoire de Mathématiques Blaise Pascal(布莱斯帕斯卡数学实验室)

Konaté Mohamed Abbas, Anne-Françoise Yao, Thierry Chateau, Pierre Bouges

AI总结:

本文针对类别增量学习中平均任务准确率过于乐观的问题,提出MICA及融合性能变化的标量指标,以更公平地评估持续学习方法并支持工业应用。

摘要:

在本文中,我们使用若干高性能方法,研究了用于持续学习(CL)的类别增量学习策略中所采用的持续学习性能指标。我们尤其考察了平均任务准确率。首先,我们通过若干简单实验表明,该指标在捕捉性能方面缺乏表达能力。我们证明,监控平均任务性能过于乐观,并可能对未来真实工业应用产生误导性结论。随后,我们首先提出一个简单指标,即最小增量类别准确率(Minimal Incremental Class Accuracy,MICA),它能够对不同持续学习方法给出更公平、更有用的评估。此外,为了提供一种可轻松比较持续学习中不同方法性能的简单方式,我们推导了另一个单一标量指标,该指标同时考虑学习性能变化以及我们新引入的指标。

英文摘要:

In this paper, we investigate continual learning performance metrics used in class incremental learning strategies for continual learning (CL) using some high performing methods. We investigate especially mean task accuracy. First, we show that it lacks of expressiveness through some simple experiments to capture performance. We show that monitoring average tasks performance is over optimistic and can lead to misleading conclusions for future real life industrial uses. Then, we propose first a simple metric, Minimal Incremental Class Accuracy (MICA) which gives a fair and more useful evaluation of different continual learning methods. Moreover, in order to provide a simple way to easily compare different methods performance in continual learning, we derive another single scalar metric that take into account the learning performance variation as well as our newly introduced metric.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑