arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1807.06473cs.LGstat.ML
收藏

上下文记忆树

Contextual Memory Trees

Carnegie Mellon University(卡内基梅隆大学);Yahoo! Research(雅虎研究院);Microsoft Research(微软研究院);Microsoft(微软公司)

Wen Sun, Alina Beygelzimer, Hal Daumé, John Langford, Paul Mineiro

摘要:

我们设计并研究了上下文记忆树(CMT),一个学习记忆控制器,插入新的记忆到一个无限大小的经验存储。它被设计为有效地从该存储中查询内存,支持对数时间插入和检索操作。因此,CMT可以被集成到现有的统计学习算法作为一个增强的存储器单元,而不会大大增加训练和推理计算。此外,CMT作为分类的简化操作,使其能够从表示或架构的进步中受益。我们证明了CMT的有效性,通过增强现有的多类和多标签分类算法与CMT和观察统计改善。我们还在几个图像字幕任务上测试了CMT学习,以证明它在计算上比简单的最近邻记忆系统更好,同时受益于奖励学习。

英文摘要:

We design and study a Contextual Memory Tree (CMT), a learning memory controller that inserts new memories into an experience store of unbounded size. It is designed to efficiently query for memories from that store, supporting logarithmic time insertion and retrieval operations. Hence CMT can be integrated into existing statistical learning algorithms as an augmented memory unit without substantially increasing training and inference computation. Furthermore CMT operates as a reduction to classification, allowing it to benefit from advances in representation or architecture. We demonstrate the efficacy of CMT by augmenting existing multi-class and multi-label classification algorithms with CMT and observe statistical improvement. We also test CMT learning on several image-captioning tasks to demonstrate that it performs computationally better than a simple nearest neighbors memory system while benefitting from reward learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑