arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1908.02894cs.LGstat.ML
收藏

多少数据足以学习高性能算法?数据驱动算法设计的泛化保证

How much data is sufficient to learn high-performing algorithms? Generalization guarantees for data-driven algorithm design

Carnegie Mellon University(卡内基梅隆大学);University of Texas at El Paso(德克萨斯大学埃尔帕索分校);University of Pennsylvania(宾夕法尼亚大学);Ocean Genomics, Inc.(海洋基因组学公司);Optimized Markets, Inc.(优化市场公司);Strategic Machine, Inc.(战略机器公司);Strategy Robot, Inc.(战略机器人公司)

Maria-Florina Balcan, Dan DeBlasio, Travis Dick, Carl Kingsford, Tuomas Sandholm, Ellen Vitercik

摘要:

算法通常具有可调参数,这些参数会影响性能指标,例如运行时和解决方案质量。对于实际使用的许多算法,没有参数设置允许有意义的最坏情况界限,因此参数可供用户调整。或者,参数可以在最坏情况近似比或运行时界限的证明内隐式地调整。然而,最坏的情况在实践中可能很少或根本不存在。越来越多的研究表明,数据驱动的算法设计可以显著提高性能。这种方法使用从未知的、特定于应用程序的分布中采样的问题实例训练集,并返回在训练集中具有较强平均性能的参数设置。我们提供了一个广泛适用的理论推导泛化保证,约束算法的平均性能之间的差异,在训练集和其预期的性能。无论参数如何调整,无论是通过自动化还是手动方法,我们的结果都适用。挑战在于,对于许多类型的算法,性能是参数的不稳定函数:稍微扰动参数可能会导致行为发生很大变化。先前的研究已经证明了推广的界限,采用逐案分析的贪心算法,聚类算法,整数规划算法,和销售机制。我们发现了一个统一的结构,我们用来证明非常普遍的保证,但我们恢复的界限从以前的研究。我们的保证适用于算法的性能是一个分段常数,线性,或-更一般的-分段结构的功能,其参数。我们的理论也暗示了新的边界投票机制和动态规划算法从计算生物学。

英文摘要:

Algorithms often have tunable parameters that impact performance metrics such as runtime and solution quality. For many algorithms used in practice, no parameter settings admit meaningful worst-case bounds, so the parameters are made available for the user to tune. Alternatively, parameters may be tuned implicitly within the proof of a worst-case approximation ratio or runtime bound. Worst-case instances, however, may be rare or nonexistent in practice. A growing body of research has demonstrated that data-driven algorithm design can lead to significant improvements in performance. This approach uses a training set of problem instances sampled from an unknown, application-specific distribution and returns a parameter setting with strong average performance on the training set. We provide a broadly applicable theory for deriving generalization guarantees that bound the difference between the algorithm's average performance over the training set and its expected performance. Our results apply no matter how the parameters are tuned, be it via an automated or manual approach. The challenge is that for many types of algorithms, performance is a volatile function of the parameters: slightly perturbing the parameters can cause large changes in behavior. Prior research has proved generalization bounds by employing case-by-case analyses of greedy algorithms, clustering algorithms, integer programming algorithms, and selling mechanisms. We uncover a unifying structure which we use to prove extremely general guarantees, yet we recover the bounds from prior research. Our guarantees apply whenever an algorithm's performance is a piecewise-constant, -linear, or -- more generally -- piecewise-structured function of its parameters. Our theory also implies novel bounds for voting mechanisms and dynamic programming algorithms from computational biology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑