SmartChoices:混合编程和机器学习
SmartChoices: Hybridizing Programming and Machine Learning
摘要:
我们提出了SmartChoices,这是一种使机器学习(ML)成为程序设计语言中一等公民的方法,我们认为这是降低将ML应用于新领域问题的入口成本的一种方法。在构建系统的方法上存在着越来越大的分歧:一方面,编程利用人类专家来定义系统,而另一方面,行为是从机器学习中的数据中学习的。我们建议通过提供一个3-call API来混合这两种方法,我们通过一个名为SmartChoice的对象公开该API。我们描述了SmartChoices接口,它如何可以在编程中使用最少的代码更改,并证明它是一个易于使用,但仍然强大的工具,通过演示改进不使用ML在所有三个算法问题:二进制搜索,快速排序和缓存。在这三个例子中,我们用完全封装在SmartChoice中的ML模型替换了常用的机器学习,因此只需要最少的代码更改。与以前将ML应用于算法问题的工作相反,我们提出的方法不需要放弃现有的实现,而是无缝集成到标准的软件开发工作流程中,并完全控制软件开发人员如何应用ML方法。我们的实现依赖于标准的强化学习(RL)方法。为了更快地学习,我们使用启发式函数作为初始函数。我们展示了如何使用这个初始函数来加速和稳定学习,同时提供一个安全网,防止性能大幅下降-允许在现实生活中的关键应用程序中安全部署。
英文摘要:
We present SmartChoices, an approach to making machine learning (ML) a first class citizen in programming languages which we see as one way to lower the entrance cost to applying ML to problems in new domains. There is a growing divide in approaches to building systems: on the one hand, programming leverages human experts to define a system while on the other hand behavior is learned from data in machine learning. We propose to hybridize these two by providing a 3-call API which we expose through an object called SmartChoice. We describe the SmartChoices-interface, how it can be used in programming with minimal code changes, and demonstrate that it is an easy to use but still powerful tool by demonstrating improvements over not using ML at all on three algorithmic problems: binary search, QuickSort, and caches. In these three examples, we replace the commonly used heuristics with an ML model entirely encapsulated within a SmartChoice and thus requiring minimal code changes. As opposed to previous work applying ML to algorithmic problems, our proposed approach does not require to drop existing implementations but seamlessly integrates into the standard software development workflow and gives full control to the software developer over how ML methods are applied. Our implementation relies on standard Reinforcement Learning (RL) methods. To learn faster, we use the heuristic function, which they are replacing, as an initial function. We show how this initial function can be used to speed up and stabilize learning while providing a safety net that prevents performance to become substantially worse -- allowing for a safe deployment in critical applications in real life.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。