基于结构风险最小化的逆强化学习模型选择
Model Selection for Inverse Reinforcement Learning via Structural Risk Minimization
AI总结：
本文通过引入结构风险最小化（SRM）框架解决逆强化学习中的模型选择权衡问题，利用策略梯度估计经验风险并以Rademacher复杂度上界作为模型惩罚，实现了最优奖励函数类的选择。
摘要：
逆强化学习（IRL）通常假设奖励函数模型被预先指定为特征的加权和，并仅估计权重参数。然而，如何选择特征并确定合适的奖励模型并非易事且依赖经验。过于简单的模型不太可能包含理想的奖励函数，而高复杂度的模型会导致巨大的计算成本和潜在过拟合。本文通过引入统计学习中的结构风险最小化（SRM）框架，来解决IRL问题模型选择中的这一权衡。SRM从假设集中选择最优奖励函数类，使估计误差和模型复杂度最小化。为构建IRL的SRM方案，我们从给定演示中估计策略梯度作为经验风险，并建立Rademacher复杂度的上界作为假设函数类的模型惩罚。进一步给出了SRM的学习保证。特别地，我们提供了线性加权和设置下的显式形式。仿真实验证明了我们算法的性能和效率。
英文摘要：
Inverse reinforcement learning (IRL) usually assumes the reward function model is pre-specified as a weighted sum of features and estimates the weighting parameters only. However, how to select features and determine a proper reward model is nontrivial and experience-dependent. A simplistic model is less likely to contain the ideal reward function, while a model with high complexity leads to substantial computation cost and potential overfitting. This paper addresses this trade-off in the model selection for IRL problems by introducing the structural risk minimization (SRM) framework from statistical learning. SRM selects an optimal reward function class from a hypothesis set minimizing both estimation error and model complexity. To formulate an SRM scheme for IRL, we estimate the policy gradient from given demonstration as the empirical risk, and establish the upper bound of Rademacher complexity as the model penalty of hypothesis function classes. The SRM learning guarantee is further presented. In particular, we provide the explicit form for the linear weighted sum setting. Simulations demonstrate the performance and efficiency of our algorithm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
