我们考虑的问题,找到良好的有限时域政策POMDPs下的预期回报度量。所考虑的策略是{em free finite-memory policies with limited memory};策略是从观察-记忆对空间到动作-记忆对空间的映射(策略在其进行时更新记忆),并且可能的记忆状态的数量是策略查找算法的输入的参数。这里考虑的算法是三种搜索算法的初步实现:局部搜索、模拟退火和遗传算法。我们将它们的结果相互比较,并与每个实例的最优策略进行比较。我们比较运行时间的每一个政策和动态规划算法的POMDPs开发的汉森,迭代地提高了有限状态控制器-以前的状态,最先进的有限内存的政策。最佳策略的值只能随着内存量的增加而提高,直到达到最佳有限内存策略所需的内存量。我们最令人惊讶的发现是,更多的内存以另一种方式提供帮助:给定比最优策略所需的更多内存,算法更有可能收敛到最优值策略。
英文摘要:
We consider the problem of finding good finite-horizon policies for POMDPs under the expected reward metric. The policies considered are {em free finite-memory policies with limited memory}; a policy is a mapping from the space of observation-memory pairs to the space of action-memeory pairs (the policy updates the memory as it goes), and the number of possible memory states is a parameter of the input to the policy-finding algorithms. The algorithms considered here are preliminary implementations of three search heuristics: local search, simulated annealing, and genetic algorithms. We compare their outcomes to each other and to the optimal policies for each instance. We compare run times of each policy and of a dynamic programming algorithm for POMDPs developed by Hansen that iteratively improves a finite-state controller --- the previous state of the art for finite memory policies. The value of the best policy can only improve as the amount of memory increases, up to the amount needed for an optimal finite-memory policy. Our most surprising finding is that more memory helps in another way: given more memory than is needed for an optimal policy, the algorithms are more likely to converge to optimal-valued policies.
VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction
Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学);Tsinghua University(清华大学);The Chinese University of Hong Kong(香港中文大学)
Rate-Coding Bundle Memory: A Unified Model of Memory and Control for Symbolic Computation in the Brain
Max Planck Institute for Psycholinguistics(马克斯·普朗克心理语言学研究所);Donders Institute for Brain, Cognition and Behaviour(唐德斯大脑、认知与行为研究所);Radboud University(拉德堡德大学)
A Miniature Brain Transformer: Thalamic Gating, Hippocampal Lateralization, Amygdaloid Salience, and Prefrontal Working Memory in Attention-Coupled Latent Memory
Department of Computer Information Engineering(计算机信息工程系)
FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding
College of Future Information Technology, Fudan University, Shanghai, China(复旦大学未来信息科技学院);Shanghai Innovation Institute, Shanghai, China(上海创新研究院);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室);The Chinese University of Hong Kong, Hong Kong, China(香港中文大学)
RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems
FNii-Shenzhen(FNii-深圳);SSE, CUHK-Shenzhen(SSE,CUHK-深圳);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));The University of Hong Kong(香港大学);National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong(香港中文大学);Ising AI
SuperLocalMemory V3.3: The Living Brain -- Biologically-Inspired Forgetting, Cognitive Quantization, and Multi-Channel Retrieval for Zero-LLM Agent Memory Systems
Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning
Centre for Artificial Intelligence and Robotics(人工智能与机器人中心);Hong Kong Institute of Science and Innovation(香港科学与创新研究院);Chinese Academy of Sciences(中国科学院);Department of Computer Science(计算机科学系);City University of Hong Kong(香港城市大学);The Department of Neurosurgery(神经外科部门);Peking University First Hospital(北京大学第一医院);State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation(自动化研究所);Hong Kong SAR, China(香港特别行政区,中国)
Bridging LSTM Architecture and the Neural Dynamics during Reading
Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海市智能信息处理重点实验室);School of Computer Science, Fudan University(复旦大学计算机科学技术学院)