arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.11316cs.LGcs.AIcs.CVcs.HCcs.IR
收藏

应用深度学习中的开放问题

Open Problems in Applied Deep Learning

University of Colorado Boulder(科罗拉多大学博尔德分校)

Maziar Raissi

摘要:

这项工作制定了机器学习机制作为一个双层优化问题。内部级优化循环需要最小化在训练数据上评估的适当选择的损失函数。这只不过是为了追求最佳模型参数而进行的经过充分研究的训练过程。外层优化循环研究较少,涉及最大化在验证数据上评估的适当选择的性能度量。这就是我们所说的“迭代过程”,追求最优的模型超参数。在许多其他自由度中,该过程需要模型工程(例如,神经网络架构设计)和管理、实验跟踪、数据集版本化和增强。迭代过程可以通过自动机器学习(AutoML)自动化,也可以留给机器学习的学生、工程师和研究人员凭直觉完成。无论我们采取哪种路线,都需要减少迭代步骤的计算成本,并直接减少开发人工智能算法的碳足迹。尽管迭代步骤作为双层优化问题的清晰和统一的数学公式,但其解决方案是特定情况和复杂的。这项工作将考虑这些情况,同时增加从监督学习到半监督,自我监督,无监督,Few-Shot,联邦,强化和物理信息学习的复杂性。这项工作的结果是,这一提议暴露了外地的许多未决问题,其中许多问题可以同时解决。

英文摘要:

This work formulates the machine learning mechanism as a bi-level optimization problem. The inner level optimization loop entails minimizing a properly chosen loss function evaluated on the training data. This is nothing but the well-studied training process in pursuit of optimal model parameters. The outer level optimization loop is less well-studied and involves maximizing a properly chosen performance metric evaluated on the validation data. This is what we call the "iteration process", pursuing optimal model hyper-parameters. Among many other degrees of freedom, this process entails model engineering (e.g., neural network architecture design) and management, experiment tracking, dataset versioning and augmentation. The iteration process could be automated via Automatic Machine Learning (AutoML) or left to the intuitions of machine learning students, engineers, and researchers. Regardless of the route we take, there is a need to reduce the computational cost of the iteration step and as a direct consequence reduce the carbon footprint of developing artificial intelligence algorithms. Despite the clean and unified mathematical formulation of the iteration step as a bi-level optimization problem, its solutions are case specific and complex. This work will consider such cases while increasing the level of complexity from supervised learning to semi-supervised, self-supervised, unsupervised, few-shot, federated, reinforcement, and physics-informed learning. As a consequence of this exercise, this proposal surfaces a plethora of open problems in the field, many of which can be addressed in parallel.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑