arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1802.06426cs.AIq-bio.NC
收藏

连续时间尺度不变未来的估计

Estimating scale-invariant future in continuous time

Boston University(波士顿大学);Harvard University(哈佛大学);University of Virginia(弗吉尼亚大学)

Zoran Tiganj, Samuel J. Gershman, Per B. Sederberg, Marc W. Howard

摘要:

自然学习者必须计算出对连续时间内刺激产生的未来结果的估计。广泛使用的强化学习算法将连续时间离散化,并估计从一个步骤到下一个步骤的转移函数(基于模型的算法)或使用Bellman方程的指数折扣未来奖励的标量值(无模型算法)。基于模型的算法的一个重要缺点是计算成本随着要模拟的时间量线性增长。另一方面,无模型算法的一个重要缺点是需要选择指数贴现所需的时间尺度。我们提出了一个计算机制,在心理学和神经科学的工作的基础上开发的,用于计算未来结果的尺度不变的时间轴。该机制有效地计算输入的估计作为未来时间的函数在一个压缩的规模,并可以用来生成一个规模不变的幂律折扣估计的预期未来的回报。未来时间的表示保留了关于何时会发生什么的信息。整个时间轴可以在单个并行操作中构建,该操作生成具体的行为和神经预测。这种计算机制可以被纳入未来的强化学习算法。

英文摘要:

Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially-discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. On the other hand, an important drawback of model-free algorithms is the need to select a time-scale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically-compressed scale, and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation which generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑