基于带记忆重放的正例-无标注学习从演示中学习约束网络
Learning Constraint Network from Demonstrations via Positive-Unlabeled Learning with Memory Replay
AI总结：
针对约束未知或难指定的规划问题，提出带记忆重放的PU学习方法，通过迭代交替更新策略与约束模型，从演示中推断非线性连续约束，在Mujoco环境中验证了其优于基线的约束精度与策略安全性。
摘要：
针对大量真实世界任务的规划需要知晓并编写所有约束的问题，但存在约束未知或难以精确指定的情况，一种可行的解决方案是从专家演示中推断未知约束。现有多数工作局限于学习简单的线性约束，或是需要对真实约束的参数化形式或环境模型有充分的先验知识。为缓解这些问题，本文提出一种正例-无标注（PU）学习方法，可从演示中推断连续、任意形式且可能为非线性的约束。从PU学习的视角，我们将演示中的所有数据视为正例（可行）数据，并学习一个（次）最优策略以生成高回报但可能不可行的轨迹，这些轨迹作为同时包含可行与不可行状态的无标注数据。在数据分布的假设下，通过后处理PU学习技术，从这两个数据集学习可行-不可行分类器（即约束模型）。整个方法采用迭代框架，在更新策略（生成并选择更高回报的策略）与更新约束模型之间交替进行。此外，引入记忆缓冲区记录并复用前序迭代的样本以防止遗忘。该方法的有效性在两个Mujoco环境中得到验证，成功推断出连续非线性约束，且在约束精度和策略安全性方面优于基线方法。
英文摘要：
Planning for a wide range of real-world tasks necessitates to know and write all constraints. However, instances exist where these constraints are either unknown or challenging to specify accurately. A possible solution is to infer the unknown constraints from expert demonstration. The majority of prior works limit themselves to learning simple linear constraints, or require strong knowledge of the true constraint parameterization or environmental model. To mitigate these problems, this paper presents a positive-unlabeled (PU) learning approach to infer a continuous, arbitrary and possibly nonlinear, constraint from demonstration. From a PU learning view, We treat all data in demonstrations as positive (feasible) data, and learn a (sub)-optimal policy to generate high-reward-winning but potentially infeasible trajectories, which serve as unlabeled data containing both feasible and infeasible states. Under an assumption on data distribution, a feasible-infeasible classifier (i.e., constraint model) is learned from the two datasets through a postprocessing PU learning technique. The entire method employs an iterative framework alternating between updating the policy, which generates and selects higher-reward policies, and updating the constraint model. Additionally, a memory buffer is introduced to record and reuse samples from previous iterations to prevent forgetting. The effectiveness of the proposed method is validated in two Mujoco environments, successfully inferring continuous nonlinear constraints and outperforming a baseline method in terms of constraint accuracy and policy safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
