arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.14026cs.CV
收藏

AutoLoss-Zero:从头开始搜索损失函数以查找一般任务

AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks

The Chinese University of Hong Kong(香港中文大学);SenseTime Research(商汤科技研究院);Qing Yuan Research Institute, Shanghai Jiao Tong University(上海交通大学清源研究院);CUHK-SenseTime Joint Laboratory, The Chinese University of Hong Kong(香港中文大学-商汤科技联合实验室);Tsinghua University(清华大学)

Hao Li, Tianwen Fu, Jifeng Dai, Hongsheng Li, Gao Huang, Xizhou Zhu

摘要:

在深度网络中各种组件的自动化设计方面取得了重大进展。然而,具有各种评估指标的通用任务的损失函数的自动设计仍然没有得到充分研究。以前手工制作损失函数的工作严重依赖于人类的专业知识,这限制了它们的可扩展性。同时,现有的损失函数的搜索主要集中在特定的任务和特定的度量,与特定任务的算法。这些工作是否可以扩展到一般的任务是未经验证和怀疑。在本文中,我们提出了AutoLoss-Zero,第一个通用的框架,从头开始搜索损失函数的通用任务。具体来说,我们设计了一个基本的搜索空间组成的原始数学运算符,以适应异构的任务和评价指标。一个变种的进化算法,发现损失函数在基本搜索空间。为了提高搜索效率,提出了适用于一般任务的丢失拒绝协议和梯度等价性检查策略。对各种计算机视觉任务的广泛实验表明,我们搜索的损失函数与现有的损失函数相当或优于现有的损失函数,这些损失函数可以很好地推广到不同的数据集和网络。代码将被释放。

英文摘要:

Significant progress has been achieved in automating the design of various components in deep networks. However, the automatic design of loss functions for generic tasks with various evaluation metrics remains under-investigated. Previous works on handcrafting loss functions heavily rely on human expertise, which limits their extendibility. Meanwhile, existing efforts on searching loss functions mainly focus on specific tasks and particular metrics, with task-specific heuristics. Whether such works can be extended to generic tasks is not verified and questionable. In this paper, we propose AutoLoss-Zero, the first general framework for searching loss functions from scratch for generic tasks. Specifically, we design an elementary search space composed only of primitive mathematical operators to accommodate the heterogeneous tasks and evaluation metrics. A variant of the evolutionary algorithm is employed to discover loss functions in the elementary search space. A loss-rejection protocol and a gradient-equivalence-check strategy are developed so as to improve the search efficiency, which are applicable to generic tasks. Extensive experiments on various computer vision tasks demonstrate that our searched loss functions are on par with or superior to existing loss functions, which generalize well to different datasets and networks. Code shall be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑