arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.04205cs.MAcs.AIcs.LG
收藏

强化学习过程中相互影响的检测与考虑

On the Detection of Mutual Influences and Their Consideration in Reinforcement Learning Processes

University of Augsburg(奥格斯堡大学);University of Kassel(卡塞尔大学)

Stefan Rudolph, Sven Tomforde, Jörg Hähner

摘要:

自适应已被提出作为一种机制,以应对复杂的技术系统的控制问题。自适应背后的一个主要驱动力是将传统的设计时决策转移到运行时并由系统本身负责的想法。为了应对不可预见的事件和条件,系统需要创造力-通常通过机器学习能力来实现。这种学习机制基于不同的知识来源。用于强化目的的环境反馈可能是自适应和自组织(SASO)系统社区中最突出的一个。然而,在这种情况下,其他(子)系统对单个系统的学习性能的成功的影响大多被忽视。在这篇文章中,我们提出了一种新的方法来确定其他系统在共享环境中的效用实现的自治系统上执行的动作的影响。考虑智能相机(SC)作为说明性示例:对于诸如对象的3D重建的目标,一个SC在摇摄/倾斜/变焦参数方面的最有希望的配置很大程度上取决于附近的其他SC的配置。由于这种相互影响不能为动态系统预先定义,它们必须在运行时学习。此外,当基于反馈回路概念自改进自己的配置决策时,必须考虑它们,例如,从SASO域或自主和有机计算倡议中已知。我们定义了一种在运行时检测这种影响的方法,提出了一种在强化学习技术中考虑这些信息的方法,并分析了人工和真实世界SASO系统设置中的行为。

英文摘要:

Self-adaptation has been proposed as a mechanism to counter complexity in control problems of technical systems. A major driver behind self-adaptation is the idea to transfer traditional design-time decisions to runtime and into the responsibility of systems themselves. In order to deal with unforeseen events and conditions, systems need creativity -- typically realized by means of machine learning capabilities. Such learning mechanisms are based on different sources of knowledge. Feedback from the environment used for reinforcement purposes is probably the most prominent one within the self-adapting and self-organizing (SASO) systems community. However, the impact of other (sub-)systems on the success of the individual system's learning performance has mostly been neglected in this context. In this article, we propose a novel methodology to identify effects of actions performed by other systems in a shared environment on the utility achievement of an autonomous system. Consider smart cameras (SC) as illustrating example: For goals such as 3D reconstruction of objects, the most promising configuration of one SC in terms of pan/tilt/zoom parameters depends largely on the configuration of other SCs in the vicinity. Since such mutual influences cannot be pre-defined for dynamic systems, they have to be learned at runtime. Furthermore, they have to be taken into consideration when self-improving the own configuration decisions based on a feedback loop concept, e.g., known from the SASO domain or the Autonomic and Organic Computing initiatives. We define a methodology to detect such influences at runtime, present an approach to consider this information in a reinforcement learning technique, and analyze the behavior in artificial as well as real-world SASO system settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑