时间剪枝(PIT):一种用于时间卷积网络的轻量级网络结构优化器
Pruning In Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks
摘要:
时间卷积网络(TCN)是用于时间序列处理任务的有前途的深度学习模型。TCN的一个关键特征是时间膨胀卷积,其优化需要大量的实验。我们提出了一个自动膨胀优化器,它解决了时间轴上的权重修剪问题,并在单个训练中学习膨胀因子和权重。我们的方法将真实SoC硬件目标上的模型大小和推理延迟分别减少了7.4倍和3倍,与没有膨胀的网络相比,没有准确性下降。它还从单个模型开始产生丰富的帕累托最优TCN集,在大小和准确性方面都优于手工设计的解决方案。
英文摘要:
Temporal Convolutional Networks (TCNs) are promising Deep Learning models for time-series processing tasks. One key feature of TCNs is time-dilated convolution, whose optimization requires extensive experimentation. We propose an automatic dilation optimizer, which tackles the problem as a weight pruning on the time-axis, and learns dilation factors together with weights, in a single training. Our method reduces the model size and inference latency on a real SoC hardware target by up to 7.4x and 3x, respectively with no accuracy drop compared to a network without dilation. It also yields a rich set of Pareto-optimal TCNs starting from a single model, outperforming hand-designed solutions in both size and accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。