自动机器学习中的统一函数散列
Unified Functional Hashing in Automatic Machine Learning
摘要：
自动机器学习（AutoML）领域最近取得了令人印象深刻的成果，包括发现最先进的机器学习解决方案，如神经图像分类器。这通常是通过应用进化搜索方法来完成的，该方法从大空间中采样多个候选解决方案，并通过长时间的训练过程评估每个候选解决方案的质量。因此，搜索往往很慢。在本文中，我们表明，可以通过采用快速统一的函数哈希，特别是通过功能等价缓存技术，我们也提出了大的效率增益。其核心思想是通过散列检测搜索方法何时产生等价的候选项，这种情况经常发生，并且这种方式避免了昂贵的重新评估。我们的散列是“功能性的”，因为它识别等价的候选者，即使它们被不同地表示或编码，并且它是“统一的”，因为相同的算法可以散列任意表示;例如计算图，命令式代码或lambda函数。作为证据，我们在多个AutoML领域显示了显着的改进，包括神经架构搜索和算法发现。最后，我们通过实证分析，考虑哈希冲突，评估噪声和搜索分布的影响。总之，我们希望本文可以作为AutoML中哈希技术的指导。
英文摘要：
The field of Automatic Machine Learning (AutoML) has recently attained impressive results, including the discovery of state-of-the-art machine learning solutions, such as neural image classifiers. This is often done by applying an evolutionary search method, which samples multiple candidate solutions from a large space and evaluates the quality of each candidate through a long training process. As a result, the search tends to be slow. In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present. The central idea is to detect by hashing when the search method produces equivalent candidates, which occurs very frequently, and this way avoid their costly re-evaluation. Our hash is "functional" in that it identifies equivalent candidates even if they were represented or coded differently, and it is "unified" in that the same algorithm can hash arbitrary representations; e.g. compute graphs, imperative code, or lambda functions. As evidence, we show dramatic improvements on multiple AutoML domains, including neural architecture search and algorithm discovery. Finally, we consider the effect of hash collisions, evaluation noise, and search distribution through empirical analysis. Altogether, we hope this paper may serve as a guide to hashing techniques in AutoML.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
