基于多基线的离线模仿学习及其在编译器优化中的应用
Offline Imitation Learning from Multiple Baselines with Applications to Compiler Optimization
AI总结：
本文针对多个互补基线策略的离线模仿学习问题，提出一种极小极大最优的简单算法，并将其应用于编译器内联优化，以生成更小二进制文件，实验表明该策略优于标准强化学习初始策略。
摘要：
本文研究一个强化学习（RL）问题：给定一组由 K 个基线策略收集的轨迹，每个策略单独可能相当次优，但在状态空间的不同互补部分具有较强性能。目标是学习一个策略，使其在整个状态空间上的表现与基线策略的最佳组合相当。我们提出一种简单的基于模仿学习的算法，给出其准确性的样本复杂度上界，并通过匹配的下界证明该算法是极小极大最优的。进一步，我们将该算法应用于机器学习引导的编译器优化场景，学习用于程序内联的策略，目标是生成更小的二进制文件。我们证明，通过本方法的几次迭代，可以学习到一个优于通过标准 RL 学到的初始策略的策略。
英文摘要：
This work studies a Reinforcement Learning (RL) problem in which we are given a set of trajectories collected with K baseline policies. Each of these policies can be quite suboptimal in isolation, and have strong performance in complementary parts of the state space. The goal is to learn a policy which performs as well as the best combination of baselines on the entire state space. We propose a simple imitation learning based algorithm, show a sample complexity bound on its accuracy and prove that the the algorithm is minimax optimal by showing a matching lower bound. Further, we apply the algorithm in the setting of machine learning guided compiler optimization to learn policies for inlining programs with the objective of creating a small binary. We demonstrate that we can learn a policy that outperforms an initial policy learned via standard RL through a few iterations of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
