arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1211.0424cs.NEcs.AI
收藏

具有记忆条件的学习分类器系统求解非马尔可夫问题

Learning classifier systems with memory condition to solve non-Markov problems

Huazhong University of Science and Technology(华中科技大学);China Three Gorges University(三峡大学)

Zhaoxiang Zang, Dehua Li, Junying Wang

摘要:

在学习分类器系统家族中,分类器系统XCS已经成功地用于许多应用。然而,标准的XCS没有记忆机制,只能在马尔可夫环境中学习最优策略,其中最优动作仅由当前感觉输入的状态确定。在实际应用中,大多数环境都是基于主体感知的部分可观测环境,也称为非马尔可夫环境。在这些环境中,XCS要么失败,要么只开发一个次优策略,因为它没有内存。在这项工作中,我们开发了一个新的分类系统的基础上XCS来解决这个问题。它在XCS中增加了一个内部消息表作为记忆表来记录输入感觉历史,并扩展了少量具有记忆条件的分类器。分类器的记忆条件,作为一个立足点,以消除歧义的非马尔可夫状态,用于感测内存列表中的指定元素。此外,还采用了一种检测方法来识别环境中的非马尔可夫状态,以避免这些状态对分类器记忆条件的控制。此外,四套不同的复杂迷宫环境进行了测试,所提出的方法。实验结果表明,我们的系统是最好的技术之一,以解决部分可观察的环境,与一些著名的分类系统提出了这些环境。

英文摘要:

In the family of Learning Classifier Systems, the classifier system XCS has been successfully used for many applications. However, the standard XCS has no memory mechanism and can only learn optimal policy in Markov environments, where the optimal action is determined solely by the state of current sensory input. In practice, most environments are partially observable environments on agent's sensation, which are also known as non-Markov environments. Within these environments, XCS either fails, or only develops a suboptimal policy, since it has no memory. In this work, we develop a new classifier system based on XCS to tackle this problem. It adds an internal message list to XCS as the memory list to record input sensation history, and extends a small number of classifiers with memory conditions. The classifier's memory condition, as a foothold to disambiguate non-Markov states, is used to sense a specified element in the memory list. Besides, a detection method is employed to recognize non-Markov states in environments, to avoid these states controlling over classifiers' memory conditions. Furthermore, four sets of different complex maze environments have been tested by the proposed method. Experimental results show that our system is one of the best techniques to solve partially observable environments, compared with some well-known classifier systems proposed for these environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑