LibAUC：一个用于 X-Risk 优化的深度学习库
LibAUC: A Deep Learning Library for X-Risk Optimization
AI总结：
本文提出开源深度学习库 LibAUC，通过受控采样与动态小批量损失的 DXO 流水线解决 X-risk 优化收敛难题，并在 CID、LTR 和 CLR 任务上验证其性能优势。
摘要：
本文介绍了名为 LibAUC 的获奖深度学习（DL）库，用于实现面向优化一类被称为 X-risks 的风险函数的前沿算法。X-risks 是指一类复合函数，其中每个数据点的损失函数以将该数据点与大量其他数据点进行对比的方式定义。它们在人工智能中具有广泛应用，可用于解决经典和新兴问题，包括但不限于不平衡数据分类（CID）、排序学习（LTR）以及表示的对比学习（CLR）。开发 LibAUC 的动机是解决现有库在求解这些问题时的收敛问题。具体而言，由于经验风险最小化（ERM）框架中使用了标准小批量技术，现有库在处理这些问题时可能无法收敛，或者需要非常大的小批量尺寸才能获得良好性能。我们的库面向深度 X-risk 优化（DXO），并已在解决 CID、LTR 和 CLR 的多种任务方面取得巨大成功。本文的贡献包括：（1）介绍了一种用于实现 DXO 算法的新型基于小批量的流水线，其在受控数据采样器和动态小批量损失的设计上不同于现有深度学习流水线；（2）提供了广泛的基准实验，用于消融研究以及与现有库的比较。LibAUC 库具备可扩展至数百万个待对比项目的性能，在优化 X-risks 时比现有库收敛更快、效果更好，并支持无缝 PyTorch 部署以及适用于各类损失优化的通用 API。我们的库已在 https://github.com/Optimization-AI/LibAUC 向开源社区开放，以促进进一步的学术研究和工业应用。
英文摘要：
This paper introduces the award-winning deep learning (DL) library called LibAUC for implementing state-of-the-art algorithms towards optimizing a family of risk functions named X-risks. X-risks refer to a family of compositional functions in which the loss function of each data point is defined in a way that contrasts the data point with a large number of others. They have broad applications in AI for solving classical and emerging problems, including but not limited to classification for imbalanced data (CID), learning to rank (LTR), and contrastive learning of representations (CLR). The motivation of developing LibAUC is to address the convergence issues of existing libraries for solving these problems. In particular, existing libraries may not converge or require very large mini-batch sizes in order to attain good performance for these problems, due to the usage of the standard mini-batch technique in the empirical risk minimization (ERM) framework. Our library is for deep X-risk optimization (DXO) that has achieved great success in solving a variety of tasks for CID, LTR and CLR. The contributions of this paper include: (1) It introduces a new mini-batch based pipeline for implementing DXO algorithms, which differs from existing DL pipeline in the design of controlled data samplers and dynamic mini-batch losses; (2) It provides extensive benchmarking experiments for ablation studies and comparison with existing libraries. The LibAUC library features scalable performance for millions of items to be contrasted, faster and better convergence than existing libraries for optimizing X-risks, seamless PyTorch deployment and versatile APIs for various loss optimization. Our library is available to the open source community at https://github.com/Optimization-AI/LibAUC, to facilitate further academic research and industrial applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
