arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2001.08823cs.AIcs.LG
收藏

什么是好的预测?评估代理人知识的挑战

What's a Good Prediction? Challenges in evaluating an agent's knowledge

University of Alberta(阿尔伯塔大学);Alberta Machine Intelligence Institute(阿尔伯塔机器智能研究所);DeepMind(深度思维)

Alex Kearney, Anna Koop, Patrick M. Pilarski

摘要:

通过学习与任务无关的世界模型来构建通用知识,可以帮助智能体解决具有挑战性的问题。然而,构建和评估这些模型仍然是一个开放的挑战。评估模型的最常见方法是评估其相对于可观察值的准确性。然而,普遍依赖估计的准确性作为知识有用性的代理,有可能使我们误入歧途。我们展示了准确性和实用性之间的冲突,通过一系列说明性的例子,包括思想实验和经验的例子在MineCraft,使用一般价值函数框架(GVF)。在确定了评估代理的知识的挑战,我们提出了一种替代的评价方法,不断出现在网上持续学习设置,我们建议通过检查内部学习过程,特别是GVF的功能的相关性,预测任务在手的评价。本文通过使用预测,预测知识的一个组成部分,这是尚未探索的第一次看评估预测。

英文摘要:

Constructing general knowledge by learning task-independent models of the world can help agents solve challenging problems. However, both constructing and evaluating such models remains an open challenge. The most common approaches to evaluating models is to assess their accuracy with respect to observable values. However, the prevailing reliance on estimator accuracy as a proxy for the usefulness of the knowledge has the potential to lead us astray. We demonstrate the conflict between accuracy and usefulness through a series of illustrative examples including both a thought experiment and empirical example in MineCraft, using the General Value Function framework (GVF). Having identified challenges in assessing an agent's knowledge, we propose an alternate evaluation approach that arises continually in the online continual learning setting we recommend evaluation by examining internal learning processes, specifically the relevance of a GVF's features to the prediction task at hand. This paper contributes a first look into evaluation of predictions through their use, an integral component of predictive knowledge which is as of yet unexplored.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑