Cockpit：一个用于训练深度神经网络的实用工具
Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks
摘要：
当工程师训练深度学习模型时，他们是非常“盲目”的。用于实时训练诊断的常用方法（例如监控训练/测试损失）是有限的。仅通过这些性能指标评估网络的训练过程类似于调试软件而不通过调试器访问内部状态。为了解决这个问题，我们提出了Cockpit，这是一个工具集，可以更仔细地了解学习机器的内部工作原理，并为从业者提供更翔实和更有意义的状态报告。它有助于识别学习阶段和故障模式，如选择不当的超参数。这些仪器利用了关于梯度分布和曲率的新的高阶信息，这些信息直到最近才变得有效。我们相信，我们为PyTorch开源的这样一个调试工具，对于解决训练过程中的问题是一个有价值的帮助。通过揭示新的见解，它也更普遍地有助于深度网络的可解释性和可解释性。
英文摘要：
When engineers train deep learning models, they are very much 'flying blind'. Commonly used methods for real-time training diagnostics, such as monitoring the train/test loss, are limited. Assessing a network's training process solely through these performance indicators is akin to debugging software without access to internal states through a debugger. To address this, we present Cockpit, a collection of instruments that enable a closer look into the inner workings of a learning machine, and a more informative and meaningful status report for practitioners. It facilitates the identification of learning phases and failure modes, like ill-chosen hyperparameters. These instruments leverage novel higher-order information about the gradient distribution and curvature, which has only recently become efficiently accessible. We believe that such a debugging tool, which we open-source for PyTorch, is a valuable help in troubleshooting the training process. By revealing new insights, it also more generally contributes to explainability and interpretability of deep nets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
