arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.10608cs.LGcs.AIeess.SP
收藏

一个模型抵得上几万个例子

A model is worth tens of thousands of examples

Technion Israel Institute of Technology(以色列理工学院);University Paris Dauphine, PSL Research University(巴黎第九大学,PSL研究大学);CNRS(法国国家科学研究中心)

Thomas Dagès, Laurent D. Cohen, Alfred M. Bruckstein

摘要:

依赖于数学数据生成模型的传统信号处理方法已经被抛弃,而支持需要大量数据的深度神经网络。由于理论上的样本复杂度几乎不可能评估,这些数量的例子通常是用粗略的经验法则来估计的。然而,这些规则仅建议网络何时应该工作,而不涉及传统方法。特别是,一个有趣的问题是:如果可能的话,神经网络需要多少数据才能与传统的基于模型的方法相媲美或优于传统的基于模型的方法?在这项工作中,我们在两个简单的例子中实证研究了这个问题,其中数据是根据精确定义的数学模型生成的,并且众所周知的最佳或最先进的数学数据不可知的解决方案。第一个问题是解卷积一维高斯信号和第二个是估计一个圆的半径和位置在随机灰度图像的磁盘。通过使用不同数量的训练数据训练各种网络,无论是简单的定制设计还是成熟的网络,我们发现与传统方法相比,网络需要数万个示例,无论网络是从头开始训练还是使用迁移学习或微调。

英文摘要:

Traditional signal processing methods relying on mathematical data generation models have been cast aside in favour of deep neural networks, which require vast amounts of data. Since the theoretical sample complexity is nearly impossible to evaluate, these amounts of examples are usually estimated with crude rules of thumb. However, these rules only suggest when the networks should work, but do not relate to the traditional methods. In particular, an interesting question is: how much data is required for neural networks to be on par or outperform, if possible, the traditional model-based methods? In this work, we empirically investigate this question in two simple examples, where the data is generated according to precisely defined mathematical models, and where well-understood optimal or state-of-the-art mathematical data-agnostic solutions are known. A first problem is deconvolving one-dimensional Gaussian signals and a second one is estimating a circle's radius and location in random grayscale images of disks. By training various networks, either naive custom designed or well-established ones, with various amounts of training data, we find that networks require tens of thousands of examples in comparison to the traditional methods, whether the networks are trained from scratch or even with transfer-learning or finetuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑