arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.04000cs.AIcs.LGcs.RO
收藏

自主系统中多通道多任务学习的软硬件协同设计

Software/Hardware Co-design for Multi-modal Multi-task Learning in Autonomous Systems

Georgia Institute of Technology(佐治亚理工学院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Cong Hao, Deming Chen

摘要:

同时优化AI授权的自治系统的结果质量(QoR)和服务质量(QoS)是非常具有挑战性的。首先,存在多个输入源,例如,来自不同传感器的多模态数据,需要不同的数据预处理、传感器融合和特征聚合。其次,有多个任务需要各种AI模型同时运行,例如,感知定位和控制第三,计算和控制系统是异构的,由具有不同特征的硬件组件组成,例如嵌入式CPU、GPU、FPGA和专用加速器。因此,自治系统本质上需要多模态多任务(MMMT)学习,必须了解硬件性能和实现策略。虽然MMMT学习已经引起了广泛的研究兴趣,但其在自治系统中的应用仍然有待探索。在本文中,我们首先讨论了应用MMMT技术在自治系统中的机会,然后讨论了必须解决的独特挑战。此外,我们讨论了MMMT模型和硬件协同设计的必要性和机会,这是自治系统,特别是与功率/资源有限或异构平台的关键。我们制定的MMMT模型和异构硬件实现协同设计作为一个可微优化问题,与提高解决方案的质量,降低整体功耗和关键路线延迟的目标。我们主张在自主系统和软件/硬件协同设计解决方案中进一步探索MMMT。

英文摘要:

Optimizing the quality of result (QoR) and the quality of service (QoS) of AI-empowered autonomous systems simultaneously is very challenging. First, there are multiple input sources, e.g., multi-modal data from different sensors, requiring diverse data preprocessing, sensor fusion, and feature aggregation. Second, there are multiple tasks that require various AI models to run simultaneously, e.g., perception, localization, and control. Third, the computing and control system is heterogeneous, composed of hardware components with varied features, such as embedded CPUs, GPUs, FPGAs, and dedicated accelerators. Therefore, autonomous systems essentially require multi-modal multi-task (MMMT) learning which must be aware of hardware performance and implementation strategies. While MMMT learning has been attracting intensive research interests, its applications in autonomous systems are still underexplored. In this paper, we first discuss the opportunities of applying MMMT techniques in autonomous systems and then discuss the unique challenges that must be solved. In addition, we discuss the necessity and opportunities of MMMT model and hardware co-design, which is critical for autonomous systems especially with power/resource-limited or heterogeneous platforms. We formulate the MMMT model and heterogeneous hardware implementation co-design as a differentiable optimization problem, with the objective of improving the solution quality and reducing the overall power consumption and critical path latency. We advocate for further explorations of MMMT in autonomous systems and software/hardware co-design solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑