稀有事件支配下变分目标的主动重要抽样：优化和推广的结果
Active Importance Sampling for Variational Objectives Dominated by Rare Events: Consequences for Optimization and Generalization
摘要：
深度神经网络在使用足够的数据进行优化时，可以提供高维函数的准确表示;相比之下，在科学计算中占主导地位的函数近似技术不能很好地扩展维度。因此，许多曾经被认为难以解决的高维采样和近似问题正在通过机器学习的镜头重新审视。虽然无与伦比的准确性的承诺可能意味着需要参数化复杂系统表示的应用程序的复兴，但在许多应用程序中，收集足够的数据来开发这样的表示仍然是一个重大的挑战。在这里，我们介绍了一种方法，结合罕见事件采样技术与神经网络优化，以优化目标函数，由罕见事件占主导地位。我们发现，重要性抽样减少了学习问题的解决方案的渐近方差，这表明有利于推广。我们研究我们的算法的背景下学习动态的两个状态之间的过渡路径的系统，在统计物理和机器学习理论的影响的应用程序的问题。我们的数值实验表明，我们可以成功地学习，即使是复杂的困难，高维和罕见的数据。
英文摘要：
Deep neural networks, when optimized with sufficient data, provide accurate representations of high-dimensional functions; in contrast, function approximation techniques that have predominated in scientific computing do not scale well with dimensionality. As a result, many high-dimensional sampling and approximation problems once thought intractable are being revisited through the lens of machine learning. While the promise of unparalleled accuracy may suggest a renaissance for applications that require parameterizing representations of complex systems, in many applications gathering sufficient data to develop such a representation remains a significant challenge. Here we introduce an approach that combines rare events sampling techniques with neural network optimization to optimize objective functions that are dominated by rare events. We show that importance sampling reduces the asymptotic variance of the solution to a learning problem, suggesting benefits for generalization. We study our algorithm in the context of learning dynamical transition pathways between two states of a system, a problem with applications in statistical physics and implications in machine learning theory. Our numerical experiments demonstrate that we can successfully learn even with the compounding difficulties of high-dimension and rare data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
