arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1907.07904cs.LGstat.ML
收藏

关于损失函数与T-范数的关系

On the relation between Loss Functions and T-Norms

University of Siena(锡耶纳大学);University of Florence(佛罗伦萨大学)

Francesco Giannini, Giuseppe Marra, Michelangelo Diligenti, Marco Maggini, Marco Gori

摘要:

深度学习已被证明在计算机视觉和自然语言处理等多个领域取得了令人印象深刻的成果。这种成功的一个关键因素是新的损失函数的发展,如流行的交叉熵损失,它已被证明提供更快的收敛速度,并减少在非常深的结构消失梯度的问题。虽然交叉熵损失通常是合理的,从概率的角度来看,本文显示了一种替代和更直接的解释,这种损失的t-范数和相关的生成函数,并得出损失函数和t-范数之间的一般关系。特别是,所提出的工作显示出有趣的结果,导致一类新的损失函数的发展。这些损失可以在任何监督学习任务中被利用,并且可以导致比常用的交叉熵损失更快的收敛速度。

英文摘要:

Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. While the cross-entropy loss is usually justified from a probabilistic perspective, this paper shows an alternative and more direct interpretation of this loss in terms of t-norms and their associated generator functions, and derives a general relation between loss functions and t-norms. In particular, the presented work shows intriguing results leading to the development of a novel class of loss functions. These losses can be exploited in any supervised learning task and which could lead to faster convergence rates that the commonly employed cross-entropy loss.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑