POET：在集成再物质化和寻呼的微型设备上训练神经网络
POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging
摘要：
在手机等边缘设备上微调模型，可以实现对敏感数据的隐私保护个性化。然而，边缘训练在历史上仅限于具有简单架构的相对较小的模型，因为训练是内存和能量密集型的。我们提出了POET，这是一种能够在内存不足的电池供电边缘设备上训练大型神经网络的算法。POET联合优化了重物化和分页两种算法的综合搜索搜索空间，以减少反向传播算法的内存消耗。给定内存预算和运行时间约束，我们制定了一个混合整数线性规划（MILP）的能量最优训练。我们的方法可以在嵌入式设备上训练更大的模型，同时降低能耗，同时不修改反向传播的数学正确性。我们证明了在Cortex-M类嵌入式设备的内存限制内微调ResNet-18和BERT是可能的，同时在能源效率方面优于当前的边缘训练方法。POET是一个开源项目，可在https://github.com/ShishirPatil/poet上获取
英文摘要：
Fine-tuning models on edge devices like mobile phones would enable privacy-preserving personalization over sensitive data. However, edge training has historically been limited to relatively small models with simple architectures because training is both memory and energy intensive. We present POET, an algorithm to enable training large neural networks on memory-scarce battery-operated edge devices. POET jointly optimizes the integrated search search spaces of rematerialization and paging, two algorithms to reduce the memory consumption of backpropagation. Given a memory budget and a run-time constraint, we formulate a mixed-integer linear program (MILP) for energy-optimal training. Our approach enables training significantly larger models on embedded devices while reducing energy consumption while not modifying mathematical correctness of backpropagation. We demonstrate that it is possible to fine-tune both ResNet-18 and BERT within the memory constraints of a Cortex-M class embedded device while outperforming current edge training methods in energy efficiency. POET is an open-source project available at https://github.com/ShishirPatil/poet
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
