Traditional signal processing methods relying on mathematical data generation models have been cast aside in favour of deep neural networks, which require vast amounts of data. Since the theoretical sample complexity is nearly impossible to evaluate, these amounts of examples are usually estimated with crude rules of thumb. However, these rules only suggest when the networks should work, but do not relate to the traditional methods. In particular, an interesting question is: how much data is required for neural networks to be on par or outperform, if possible, the traditional model-based methods? In this work, we empirically investigate this question in two simple examples, where the data is generated according to precisely defined mathematical models, and where well-understood optimal or state-of-the-art mathematical data-agnostic solutions are known. A first problem is deconvolving one-dimensional Gaussian signals and a second one is estimating a circle's radius and location in random grayscale images of disks. By training various networks, either naive custom designed or well-established ones, with various amounts of training data, we find that networks require tens of thousands of examples in comparison to the traditional methods, whether the networks are trained from scratch or even with transfer-learning or finetuning.
Not Every Image is Worth a Thousand Words: Quantifying Originality in Stable Diffusion
School of Computer Science, Tel Aviv University(特拉维夫大学计算机科学学院);Faculty of Law, Tel Aviv University(特拉维夫大学法学院);School of Electrical Engineering, Tel Aviv University(特拉维夫大学电气工程学院)
Therapist-Robot-Patient Physical Interaction is Worth a Thousand Words: Enabling Intuitive Therapist Guidance via Remote Haptic Control
Department of Cognitive Robotics, Delft University of Technology(认知机器人系,代尔夫特理工大学);Department of Rehabilitation Medicine, Erasmus Medical Center(康复医学系,埃拉斯姆斯医疗中心)
Universal Differential Equations for Scientific Machine Learning
Massachusetts Institute of Technology(麻省理工学院);University of Maryland, Baltimore(马里兰大学巴尔的摩分校);Julia Computing(Julia计算公司);Otto von Guericke University, Magdeburg(马格德堡奥托·冯·格里克大学);Saint Petersburg State University(圣彼得堡国立大学)