利用多层感知器训练时间中的重尾:以UCI甲状腺疾病数据库为例
Exploiting Heavy Tails in Training Times of Multilayer Perceptrons: A Case Study with the UCI Thyroid Disease Database
摘要:
多层感知器的权重的随机初始化使得可以将其训练过程建模为Las Vegas算法,即当获得某些所需的训练误差时停止并且其执行时间是随机变量的随机算法。这个模型是用来执行一个著名的模式识别基准的案例研究:UCI甲状腺疾病数据库。经验证据表明,训练时间的概率分布表现出重尾行为,这意味着大概率质量的长时间执行。利用这一事实,通过应用两个简单的重启策略来减少训练时间成本。第一个假设完全了解分布,相对于没有重新启动的训练,预期时间减少了40%。第二,假设零知识,产生从9%到23%的减少。
英文摘要:
The random initialization of weights of a multilayer perceptron makes it possible to model its training process as a Las Vegas algorithm, i.e. a randomized algorithm which stops when some required training error is obtained, and whose execution time is a random variable. This modeling is used to perform a case study on a well-known pattern recognition benchmark: the UCI Thyroid Disease Database. Empirical evidence is presented of the training time probability distribution exhibiting a heavy tail behavior, meaning a big probability mass of long executions. This fact is exploited to reduce the training time cost by applying two simple restart strategies. The first assumes full knowledge of the distribution yielding a 40% cut down in expected time with respect to the training without restarts. The second, assumes null knowledge, yielding a reduction ranging from 9% to 23%.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。