DiTMoS：深入探索微控制器上的多样微型模型选择
DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers
AI总结：
DiTMoS提出一种选择器-分类器架构的DNN训练推理框架，通过多样数据划分、对抗训练和特征聚合提升弱模型组合准确率，在微控制器上实现高达13.4%的精度提升。
摘要：
在微控制器上实现高效且准确的深度神经网络（DNN）推理并非易事，原因在于其片上资源受限。当前的方法主要侧重于压缩较大的模型，但这以牺牲模型准确性为代价。在本文中，我们从相反的角度重新思考这个问题，即直接构建小型/弱模型并提高其准确性。为此，我们引入了DiTMoS，一种新颖的DNN训练与推理框架，采用选择器-分类器架构，其中选择器将每个输入样本路由到适当的分类器进行分类。DiTMoS基于一个关键洞察：弱模型的组合可以展现出高度的多样性，并且它们的联合可以显著提高准确率上限。为了接近这一上限，DiTMoS引入了三种策略，包括多样化的训练数据划分以增加分类器的多样性，对抗性选择器-分类器训练以确保协同交互从而最大化它们的互补性，以及异构特征聚合以提高分类器的能力。我们进一步提出了一种网络切片技术，以减轻特征聚合带来的额外内存开销。我们将DiTMoS部署在Neucleo STM32F767ZI板上，并分别基于三个用于人类活动识别、关键词识别和情感识别的时间序列数据集对其进行评估。实验结果表明：（a）与最佳基线相比，DiTMoS实现了高达13.4%的准确率提升；（b）网络切片几乎完全消除了特征聚合带来的内存开销，仅带来轻微延迟增加。
英文摘要：
Enabling efficient and accurate deep neural network (DNN) inference on microcontrollers is non-trivial due to the constrained on-chip resources. Current methodologies primarily focus on compressing larger models yet at the expense of model accuracy. In this paper, we rethink the problem from the inverse perspective by constructing small/weak models directly and improving their accuracy. Thus, we introduce DiTMoS, a novel DNN training and inference framework with a selector-classifiers architecture, where the selector routes each input sample to the appropriate classifier for classification. DiTMoS is grounded on a key insight: a composition of weak models can exhibit high diversity and the union of them can significantly boost the accuracy upper bound. To approach the upper bound, DiTMoS introduces three strategies including diverse training data splitting to increase the classifiers' diversity, adversarial selector-classifiers training to ensure synergistic interactions thereby maximizing their complementarity, and heterogeneous feature aggregation to improve the capacity of classifiers. We further propose a network slicing technique to alleviate the extra memory overhead incurred by feature aggregation. We deploy DiTMoS on the Neucleo STM32F767ZI board and evaluate it based on three time-series datasets for human activity recognition, keywords spotting, and emotion recognition, respectively. The experiment results manifest that: (a) DiTMoS achieves up to 13.4% accuracy improvement compared to the best baseline; (b) network slicing almost completely eliminates the memory overhead incurred by feature aggregation with a marginal increase of latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
