跨学徒制学习框架：性质与解决方法
Cross apprenticeship learning framework: Properties and solution approaches
摘要：
学徒制学习是一个框架，在这个框架中，智能体使用专家提供的示例轨迹来学习策略以在环境中执行给定的任务。在现实世界中，人们可能会在不同的环境中访问专家轨迹，其中系统动力学不同，而学习任务相同。对于这样的场景，可以定义两种类型的学习目标。一种是学习的策略在一个特定环境中表现良好，另一种是在所有环境中表现良好。为了在原则上平衡这两个目标，我们的工作提出了跨学徒学习（CAL）框架。这包括一个最优化问题，其中寻求每个环境的最优策略，同时确保所有策略保持彼此接近。通过最优化问题中的一个调谐参数来促进这种接近。我们得到的优化问题的调整参数的变化的性质。由于问题是非凸的，我们提供了一个凸外近似。最后，我们展示了我们的框架的属性在一个有风的网格世界环境中的导航任务的上下文中。
英文摘要：
Apprenticeship learning is a framework in which an agent learns a policy to perform a given task in an environment using example trajectories provided by an expert. In the real world, one might have access to expert trajectories in different environments where the system dynamics is different while the learning task is the same. For such scenarios, two types of learning objectives can be defined. One where the learned policy performs very well in one specific environment and another when it performs well across all environments. To balance these two objectives in a principled way, our work presents the cross apprenticeship learning (CAL) framework. This consists of an optimization problem where an optimal policy for each environment is sought while ensuring that all policies remain close to each other. This nearness is facilitated by one tuning parameter in the optimization problem. We derive properties of the optimizers of the problem as the tuning parameter varies. Since the problem is nonconvex, we provide a convex outer approximation. Finally, we demonstrate the attributes of our framework in the context of a navigation task in a windy gridworld environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
