In recent years, reinforcement learning (RL) has gained increasing attention in control engineering. Especially, policy gradient methods are widely used. In this work, we improve the tracking performance of proximal policy optimization (PPO) for arbitrary reference signals by incorporating information about future reference values. Two variants of extending the argument of the actor and the critic taking future reference values into account are presented. In the first variant, global future reference values are added to the argument. For the second variant, a novel kind of residual space with future reference values applicable to model-free reinforcement learning is introduced. Our approach is evaluated against a PI controller on a simple drive train model. We expect our method to generalize to arbitrary references better than previous approaches, pointing towards the applicability of RL to control real systems.
Anticipatory Reinforcement Learning for Trajectory Tracking
Josef Ressel Centre for Intelligent and Secure Industrial Automation, Salzburg University of Applied Sciences（约瑟夫·雷塞尔智能与安全工业自动化中心，萨尔茨堡应用科学大学）；Salzburg University of Applied Sciences（萨尔茨堡应用科学大学）；Paris Lodron University of Salzburg（萨尔茨堡巴黎洛德伦大学）
Tracking the Median of Gradients with a Stochastic Proximal Point Method
Technical University of Munich and Inria Paris, ENS, PSL Research University（慕尼黑技术大学和巴黎Inria、ENS、PSL研究大学）；Université Paris Cité and Sorbonne Université, CNRS（巴黎国立大学和索邦大学、CNRS）；Inria, CNRS, ENS, PSL Research University Paris（Inria、CNRS、ENS、PSL研究大学巴黎）；CCM, Flatiron Institute, Simons Foundation（CCM、Flatiron研究所、Simons基金会）
Exploiting Longitudinal Context in Clinician-Verified Interactive Lesion Tracking
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Germany（德国癌症研究中心（DKFZ）海德堡，医学图像计算部，德国）；Faculty of Mathematics and Computer Science, Heidelberg University, Germany（海德堡大学数学与计算机科学学院，德国）；HIDSS4Health -- Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany（HIDSS4Health——海德堡信息与数据科学健康学校，卡尔斯鲁厄/海德堡，德国）；Medical Faculty, Heidelberg University, Germany（海德堡大学医学学院，德国）；University Hospital Brandenburg an der Havel, Brandenburg Medical School Theodor Fontane, Germany（勃兰登堡运河大学医院，布兰登堡泰奥多尔·冯·_fontane医学学校，德国）；Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Germany（放射肿瘤科模式分析与学习组，海德堡大学医院，德国）
