arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.02694cs.LGcs.AIcs.CV
收藏

深度学习中的损失函数与指标

Loss Functions and Metrics in Deep Learning

CICATA-Qro Instituto Politecnico Nacional(墨西哥国立理工学院CICATA-Qro分中心);Facultad de Informática Universidad Autónoma de Querétaro(克雷塔罗自治大学信息学院);Centro de Investigaciones en Óptica A.C.(光学研究中心(AC))

Juan Terven, Diana M. Cordova-Esparza, Alfonso Ramirez-Pedraza, Edgar A. Chavez-Urbiola, Julio A. Romero-Gonzalez

AI总结:

本文全面综述深度学习中的损失函数与性能指标,构建统一理解框架,探讨多损失设置与专用指标,为不同任务提供选择指导,助力有效训练与模型评估。

摘要:

本文全面综述了深度学习中的损失函数和性能指标,强调了不同应用领域的关键进展和实践见解。我们首先概述回归和分类等经典任务中的基本考虑因素,然后将分析扩展到计算机视觉和自然语言处理(包括检索增强生成)等专业领域。在每个场景中,我们系统地研究了如何将不同的损失函数和评估指标配对,以应对类不平衡、异常值和序列级优化等任务特定挑战。本工作的主要贡献包括:(1)一个理解损失和指标如何与不同学习目标对齐的统一框架;(2)对平衡竞争目标的多损失设置的深入讨论;(3)对评估检索增强生成等现代应用(其中忠实度和上下文相关性至关重要)所使用的专用指标的新见解。在此过程中,我们强调了基于经验行为和领域约束选择或组合损失和指标的最佳实践。最后,我们确定了开放问题和有前景的方向,包括损失函数搜索的自动化以及为日益复杂的深度学习任务开发稳健、可解释的评估措施。我们的综述旨在为研究人员和从业者提供更清晰的指导,以设计有效的训练管道和可靠的模型评估,用于广泛的现实世界应用。

英文摘要:

This paper presents a comprehensive review of loss functions and performance metrics in deep learning, highlighting key developments and practical insights across diverse application areas. We begin by outlining fundamental considerations in classic tasks such as regression and classification, then extend our analysis to specialized domains like computer vision and natural language processing including retrieval-augmented generation. In each setting, we systematically examine how different loss functions and evaluation metrics can be paired to address task-specific challenges such as class imbalance, outliers, and sequence-level optimization. Key contributions of this work include: (1) a unified framework for understanding how losses and metrics align with different learning objectives, (2) an in-depth discussion of multi-loss setups that balance competing goals, and (3) new insights into specialized metrics used to evaluate modern applications like retrieval-augmented generation, where faithfulness and context relevance are pivotal. Along the way, we highlight best practices for selecting or combining losses and metrics based on empirical behaviors and domain constraints. Finally, we identify open problems and promising directions, including the automation of loss-function search and the development of robust, interpretable evaluation measures for increasingly complex deep learning tasks. Our review aims to equip researchers and practitioners with clearer guidance in designing effective training pipelines and reliable model assessments for a wide spectrum of real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑