arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1301.6715cs.AI
收藏

我的大脑是满的:当更多的记忆帮助

My Brain is Full: When More Memory Helps

University of Kentucky(肯塔基大学)

Christopher Lusena, Tong Li, Shelia Sittinger, Chris Wells, Judy Goldsmith

摘要:

我们考虑的问题,找到良好的有限时域政策POMDPs下的预期回报度量。所考虑的策略是{em free finite-memory policies with limited memory};策略是从观察-记忆对空间到动作-记忆对空间的映射(策略在其进行时更新记忆),并且可能的记忆状态的数量是策略查找算法的输入的参数。这里考虑的算法是三种搜索算法的初步实现:局部搜索、模拟退火和遗传算法。我们将它们的结果相互比较,并与每个实例的最优策略进行比较。我们比较运行时间的每一个政策和动态规划算法的POMDPs开发的汉森,迭代地提高了有限状态控制器-以前的状态,最先进的有限内存的政策。最佳策略的值只能随着内存量的增加而提高,直到达到最佳有限内存策略所需的内存量。我们最令人惊讶的发现是,更多的内存以另一种方式提供帮助:给定比最优策略所需的更多内存,算法更有可能收敛到最优值策略。

英文摘要:

We consider the problem of finding good finite-horizon policies for POMDPs under the expected reward metric. The policies considered are {em free finite-memory policies with limited memory}; a policy is a mapping from the space of observation-memory pairs to the space of action-memeory pairs (the policy updates the memory as it goes), and the number of possible memory states is a parameter of the input to the policy-finding algorithms. The algorithms considered here are preliminary implementations of three search heuristics: local search, simulated annealing, and genetic algorithms. We compare their outcomes to each other and to the optimal policies for each instance. We compare run times of each policy and of a dynamic programming algorithm for POMDPs developed by Hansen that iteratively improves a finite-state controller --- the previous state of the art for finite memory policies. The value of the best policy can only improve as the amount of memory increases, up to the amount needed for an optimal finite-memory policy. Our most surprising finding is that more memory helps in another way: given more memory than is needed for an optimal policy, the algorithms are more likely to converge to optimal-valued policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑