arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.01384cs.CVcs.AIcs.LG
收藏

使用协同多任务神经架构搜索实现边缘有效的密集预测

Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search

UNC at Chapel Hill(北卡罗来纳大学教堂山分校);Google Research(谷歌研究院);X, The Moonshot Factory(X,登月工厂)

Thanh Vu, Yanqi Zhou, Chunfeng Wen, Yueqi Li, Jan-Michael Frahm

摘要:

在这项工作中,我们提出了一种新颖且可扩展的解决方案,以解决在边缘平台上开发高效密集预测的挑战。我们的第一个关键见解是,多任务学习(MTL)和硬件感知神经架构搜索(NAS)可以协同工作,从而极大地受益于设备上的密集预测(DP)。实验结果表明,这两种范式的联合学习在提高DP准确性方面令人惊讶地有效,与单任务NAS的迁移学习和MTL中的现有最先进方法相比,都具有优异的性能,所有这些都只有十分之一的计算量。据我们所知,我们的框架,名为EDNAS,是第一个成功地利用NAS和MTL的协同关系的DP。我们的第二个关键见解是,多任务DP的标准深度训练可能会对MTL评估造成显著的不稳定性和噪声。相反,我们提出了JAReD,这是一种改进的,易于采用的联合绝对-相对深度损失,可减少高达88%的不需要的噪声,同时提高精度。我们对标准数据集进行广泛的评估,对强基线和最先进的方法进行基准测试,并对发现的最佳架构进行分析。

英文摘要:

In this work, we propose a novel and scalable solution to address the challenges of developing efficient dense predictions on edge platforms. Our first key insight is that MultiTask Learning (MTL) and hardware-aware Neural Architecture Search (NAS) can work in synergy to greatly benefit on-device Dense Predictions (DP). Empirical results reveal that the joint learning of the two paradigms is surprisingly effective at improving DP accuracy, achieving superior performance over both the transfer learning of single-task NAS and prior state-of-the-art approaches in MTL, all with just 1/10th of the computation. To the best of our knowledge, our framework, named EDNAS, is the first to successfully leverage the synergistic relationship of NAS and MTL for DP. Our second key insight is that the standard depth training for multi-task DP can cause significant instability and noise to MTL evaluation. Instead, we propose JAReD, an improved, easy-to-adopt Joint Absolute-Relative Depth loss, that reduces up to 88% of the undesired noise while simultaneously boosting accuracy. We conduct extensive evaluations on standard datasets, benchmark against strong baselines and state-of-the-art approaches, as well as provide an analysis of the discovered optimal architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑