饮食深度生成音频模型与结构化彩票
Diet deep generative audio models with structured lottery
摘要：
深度学习模型在大多数音频应用领域提供了非常成功的解决方案。然而，这些模型的高精度是以巨大的计算成本为代价的。在评估所提出的模型的质量时，这方面几乎总是被忽视。然而，在评价模型时不应不考虑其复杂性。这一点在音频应用中尤为重要，因为音频应用严重依赖于具有实时约束的专用嵌入式硬件。在本文中，我们通过研究深度生成音频模型的彩票假设，建立在最近的观察结果基础上，即深度模型是高度过度参数化的。这个假设指出，深度模型中存在非常高效的小型子网络，如果孤立地训练，它将提供比大型模型更高的准确性。然而，彩票是通过依赖非结构化掩蔽来发现的，这意味着所得到的模型在磁盘大小或推理时间方面都没有提供任何增益。相反，我们在这里开发一种方法，旨在执行结构化修剪。我们表明，这需要依赖于全球选择，并引入一个特定的标准的基础上互信息。首先，我们确认了一个令人惊讶的结果，即较小的模型比大型模型提供更高的准确性。我们进一步表明，我们可以删除高达95%的模型权重，而不会显着降低准确性。因此，我们可以通过流行的方法（如Wavenet，SING或DDSP）获得非常轻的生成音频模型，这些方法的精度高达100倍。我们研究了在Raspberry Pi和Arduino上嵌入这些模型的理论界限，并表明我们可以在CPU上获得与大型GPU模型质量相当的生成模型。最后，我们讨论了在嵌入式平台上实现深度生成音频模型的可能性。
英文摘要：
Deep learning models have provided extremely successful solutions in most audio application fields. However, the high accuracy of these models comes at the expense of a tremendous computation cost. This aspect is almost always overlooked in evaluating the quality of proposed models. However, models should not be evaluated without taking into account their complexity. This aspect is especially critical in audio applications, which heavily relies on specialized embedded hardware with real-time constraints. In this paper, we build on recent observations that deep models are highly overparameterized, by studying the lottery ticket hypothesis on deep generative audio models. This hypothesis states that extremely efficient small sub-networks exist in deep models and would provide higher accuracy than larger models if trained in isolation. However, lottery tickets are found by relying on unstructured masking, which means that resulting models do not provide any gain in either disk size or inference time. Instead, we develop here a method aimed at performing structured trimming. We show that this requires to rely on global selection and introduce a specific criterion based on mutual information. First, we confirm the surprising result that smaller models provide higher accuracy than their large counterparts. We further show that we can remove up to 95% of the model weights without significant degradation in accuracy. Hence, we can obtain very light models for generative audio across popular methods such as Wavenet, SING or DDSP, that are up to 100 times smaller with commensurate accuracy. We study the theoretical bounds for embedding these models on Raspberry Pi and Arduino, and show that we can obtain generative models on CPU with equivalent quality as large GPU models. Finally, we discuss the possibility of implementing deep generative audio models on embedded platforms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
