模糊规则表示策略的策略梯度强化学习：在汽车速度控制仿真中的应用
Policy Gradient Reinforcement Learning for Policy Represented by Fuzzy Rules: Application to Simulations of Speed Control of an Automobile
摘要：
模糊推理和策略梯度强化学习的融合方法已经被提出，其直接学习由具有权重的模糊规则表示的策略函数中的参数，作为最大化每集奖励的期望值。一项研究已将该方法应用于汽车的速度控制任务，并已获得正确的策略，其中一些策略适当地控制汽车的速度，但许多其他策略产生不适当的速度振动。通常，不期望导致输出值的突然时间变化或振动的策略，并且将存在许多情况，其中期望给出输出值的平滑时间变化的策略。在本文中，我们提出了一种融合方法，使用的目标函数，引入去模糊化与重心模型加权随机和约束项的平滑时间变化，作为一种改进措施，以抑制突然变化的模糊控制器的输出值。然后给出了融合过程中的学习规则，并考虑了奖励函数对输出值波动的影响。作为我们的方法在汽车速度控制上的应用的实验结果，证实了所提出的方法具有抑制输出值的时间序列中的不期望的波动的效果。此外，它也表明，奖励功能之间的差异可能会对学习的结果产生不利影响。
英文摘要：
A method of a fusion of fuzzy inference and policy gradient reinforcement learning has been proposed that directly learns, as maximizes the expected value of the reward per episode, parameters in a policy function represented by fuzzy rules with weights. A study has applied this method to a task of speed control of an automobile and has obtained correct policies, some of which control speed of the automobile appropriately but many others generate inappropriate vibration of speed. In general, the policy is not desirable that causes sudden time change or vibration in the output value, and there would be many cases where the policy giving smooth time change in the output value is desirable. In this paper, we propose a fusion method using the objective function, that introduces defuzzification with the center of gravity model weighted stochastically and a constraint term for smoothness of time change, as an improvement measure in order to suppress sudden change of the output value of the fuzzy controller. Then we show the learning rule in the fusion, and also consider the effect by reward functions on the fluctuation of the output value. As experimental results of an application of our method on speed control of an automobile, it was confirmed that the proposed method has the effect of suppressing the undesirable fluctuation in time-series of the output value. Moreover, it was also showed that the difference between reward functions might adversely affect the results of learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
