如何填充最佳集？无害多样性种群梯度下降
How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
摘要：
虽然传统的优化方法专注于寻找单个最优解，但现代机器学习问题中的大多数目标函数，特别是深度学习中的目标函数，往往具有多个或无穷多个最优解。因此，考虑在目标函数的最优集合中找到一组不同点的问题是有用的。在这项工作中，我们将这个问题框架为最大化主损失函数的最佳集合内的多样性得分的双层优化问题，并使用简单的人口梯度下降框架来解决它，该框架迭代更新点以最大化多样性得分，从而不会损害主损失的优化。我们证明，我们的方法可以有效地产生不同的解决方案在各种应用，包括文本到图像生成，文本到网格生成，分子构象生成和集成神经网络训练。
英文摘要：
Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima. Therefore, it is useful to consider the problem of finding a set of diverse points in the optimum set of an objective function. In this work, we frame this problem as a bi-level optimization problem of maximizing a diversity score inside the optimum set of the main loss function, and solve it with a simple population gradient descent framework that iteratively updates the points to maximize the diversity score in a fashion that does not hurt the optimization of the main loss. We demonstrate that our method can efficiently generate diverse solutions on a variety of applications, including text-to-image generation, text-to-mesh generation, molecular conformation generation and ensemble neural network training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
