利用模型等价求解交互式动态影响图
Exploiting Model Equivalences for Solving Interactive Dynamic Influence Diagrams
摘要:
我们专注于在部分可观察的环境中与其他代理的不确定类型具有类似或冲突的目标共享的顺序决策的问题。这个问题以前已经正式由多个框架,其中之一是交互式动态影响图(I-DID),它概括了众所周知的影响图的多智能体设置。I-DID是图形模型,可用于计算代理的策略,给定其对物理状态和其他模型的信念,其随着代理在多代理设置中的行为和观察而变化。正如我们所料,解决I-DID在计算上是困难的。这主要是由于归因于其他代理的候选模型的大空间及其随时间的指数增长。我们提出了两种方法来减少模型空间的大小,并阻止其指数增长。这两种方法都涉及将单个模型聚合到等价类中。我们的第一种方法将行为等价的模型分组在一起,只选择那些在更新后的模型空间中会产生与其他模型不同的预测行为的模型进行更新。第二种方法通过关注行为预测的部分来进一步压缩模型空间。具体来说,我们集群动作等效模型,规定在一个单一的时间步相同的行动。准确地识别等价性需要我们求解初始集合中的所有模型。我们通过选择性地求解某些模型来避免这种情况,从而引入近似值。我们讨论了近似引入的误差,并实证证明了由于等价性而提高了求解I-DID的效率。
英文摘要:
We focus on the problem of sequential decision making in partially observable environments shared with other agents of uncertain types having similar or conflicting objectives. This problem has been previously formalized by multiple frameworks one of which is the interactive dynamic influence diagram (I-DID), which generalizes the well-known influence diagram to the multiagent setting. I-DIDs are graphical models and may be used to compute the policy of an agent given its belief over the physical state and others models, which changes as the agent acts and observes in the multiagent setting. As we may expect, solving I-DIDs is computationally hard. This is predominantly due to the large space of candidate models ascribed to the other agents and its exponential growth over time. We present two methods for reducing the size of the model space and stemming its exponential growth. Both these methods involve aggregating individual models into equivalence classes. Our first method groups together behaviorally equivalent models and selects only those models for updating which will result in predictive behaviors that are distinct from others in the updated model space. The second method further compacts the model space by focusing on portions of the behavioral predictions. Specifically, we cluster actionally equivalent models that prescribe identical actions at a single time step. Exactly identifying the equivalences would require us to solve all models in the initial set. We avoid this by selectively solving some of the models, thereby introducing an approximation. We discuss the error introduced by the approximation, and empirically demonstrate the improved efficiency in solving I-DIDs due to the equivalences.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。