arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.00011cs.LGcs.AI
收藏

一种有效的多任务时间序列分类联邦蒸馏学习系统

An Efficient Federated Distillation Learning System for Multi-task Time Series Classification

Southwest Jiaotong University(西南交通大学);Chengdu University of Information Technology(成都信息工程大学);University of Nottingham(诺丁汉大学)

Huanlai Xing, Zhiwen Xiao, Rong Qu, Zonghai Zhu, Bowen Zhao

摘要:

提出了一种用于多任务时间序列分类的高效联邦蒸馏学习系统(EFDLS)。EFDLS由一个中央服务器和多个移动用户组成,其中不同的用户可以运行不同的TSC任务。EFDLS有两个新的组件,即基于特征的学生-教师(FBST)框架和基于距离的权重匹配(DBWM)计划。在每个用户中,FBST框架通过知识蒸馏将知识从教师的隐藏层转移到学生的隐藏层,教师和学生具有相同的网络结构。对于每个连接的用户,其学生模型的隐藏层权重会定期上传到EFDLS服务器。DBWM方案部署在服务器上,使用最小二乘距离来度量两个给定模型的权重之间的相似性。该方案为每个连接的用户找到一个合作伙伴,使得用户及其合作伙伴的权重在所有上传的权重中最接近。服务器交换并将用户及其伙伴的权重发送回这两个用户,然后这两个用户将接收到的权重加载到他们的教师的隐藏层。实验结果表明,所提出的EFDLS在一组选定的UCR 2018数据集上实现了优异的性能,就top-1精度而言。

英文摘要:

This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components, namely a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. Within each user, the FBST framework transfers knowledge from its teacher's hidden layers to its student's hidden layers via knowledge distillation, with the teacher and student having identical network structure. For each connected user, its student model's hidden layers' weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user's and its partner's weights are the closest among all the weights uploaded. The server exchanges and sends back the user's and its partner's weights to these two users which then load the received weights to their teachers' hidden layers. Experimental results show that the proposed EFDLS achieves excellent performance on a set of selected UCR2018 datasets regarding top-1 accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑