GENO -- 为经典机器学习设计的通用优化
GENO -- GENeric Optimization for Classical Machine Learning
AI总结:
本文提出GENO框架,通过结合建模语言和通用求解器,实现了对大多数经典机器学习问题的高效自动求解,展示了其在效率上的优势。
摘要:
尽管优化是机器学习的长期算法核心,但新模型仍需要耗时实现新求解器。因此,有成千上万种针对机器学习问题的优化算法实现。一个自然的问题是,是否总需要实现新求解器,或者是否存在一个适用于大多数模型的算法。普遍认为这种“万能算法”无法工作,因为该算法无法利用模型特定的结构,因此无法在广泛的问题上高效且稳健。本文挑战这一普遍观点。我们设计并实现了优化框架GENO(GENeric Optimization),它结合了建模语言和通用求解器。GENO从优化问题类的声明性规范中生成求解器。该框架足够灵活,可以涵盖大多数经典机器学习问题。我们在广泛的经典问题以及一些最近提出的问题上展示了自动生成的求解器的性能:(1) 与精心设计的专用求解器一样高效,(2) 比最近的最先进求解器有相当大的优势,(3) 比传统建模语言加求解器方法快多个数量级。
英文摘要:
Although optimization is the longstanding algorithmic backbone of machine learning, new models still require the time-consuming implementation of new solvers. As a result, there are thousands of implementations of optimization algorithms for machine learning problems. A natural question is, if it is always necessary to implement a new solver, or if there is one algorithm that is sufficient for most models. Common belief suggests that such a one-algorithm-fits-all approach cannot work, because this algorithm cannot exploit model specific structure and thus cannot be efficient and robust on a wide variety of problems. Here, we challenge this common belief. We have designed and implemented the optimization framework GENO (GENeric Optimization) that combines a modeling language with a generic solver. GENO generates a solver from the declarative specification of an optimization problem class. The framework is flexible enough to encompass most of the classical machine learning problems. We show on a wide variety of classical but also some recently suggested problems that the automatically generated solvers are (1) as efficient as well-engineered specialized solvers, (2) more efficient by a decent margin than recent state-of-the-art solvers, and (3) orders of magnitude more efficient than classical modeling language plus solver approaches.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。