In real-world scenarios, the observation data for reinforcement learning with continuous control is commonly noisy and part of it may be dynamically missing over time, which violates the assumption of many current methods developed for this. We addressed the issue within the framework of partially observable Markov Decision Process (POMDP) using a model-based method, in which the transition model is estimated from the incomplete and noisy observations using a newly proposed surrogate loss function with local approximation, while the policy and value function is learned with the help of belief imputation. For the latter purpose, a generative model is constructed and is seamlessly incorporated into the belief updating procedure of POMDP, which enables robust execution even under a significant incompleteness and noise. The effectiveness of the proposed method is verified on a collection of benchmark tasks, showing that our approach outperforms several compared methods under various challenging scenarios.
渗透测试是一种通过生成和执行可能的攻击来评估网络安全的方法。这样做可以自动进行定期和系统的测试，而无需大量的人力。一个关键问题是如何产生攻击。这自然是一个规划问题。以前的工作（Lucangeli et al. 2010）使用经典规划，因此忽略了所有黑客特有的不完整知识。最近的工作（Sarraute等人，2011）为了扩展而做出了强有力的独立性假设，并且缺乏关于攻击规划问题实际上是什么的明确的正式概念。在这里，我们模型的部分可观察马尔可夫决策过程（POMDP）的问题。这是渗透测试在一个良好的研究形式主义的基础，突出了这个问题的性质的重要方面。POMDPs允许将信息收集建模为问题的一个组成部分，从而首次提供了一种智能地将扫描操作与实际利用混合在一起的方法。
Scaling Internal-State Policy-Gradient Methods for POMDPs
Research School of Information Science and Engineering, Australian Nat. University, ACT 0200, Australia（信息科学与工程研究学校，澳大利亚国立大学，ACT 0200，澳大利亚）；Panscient Pty Ltd, Adelaide, Australia（Panscient Pty Ltd，阿德莱德，澳大利亚）
Dynamic programming with incomplete information to overcome navigational uncertainty in a nautical environment
University of Ottawa（渥太华大学）；National Research Council of Canada（加拿大国家研究委员会）；University of Waterloo（滑铁卢大学）；Vector Institute for Artificial Intelligence（向量人工智能研究所）
Online POMDP Planning with Anytime Deterministic Optimality Guarantees
Technion Autonomous Systems Program (TASP)（技术学院自主系统计划）；Stephen B. Klein Faculty of Aerospace Engineering（斯蒂芬·B·克莱因航空航天工程学院）；Faculty of Data and Decision Sciences（数据与决策科学学院）
REBA: A Revealed Belief Automaton Framework for Online Planning in Continuous POMDPs
School of Computer Science and Technology, Dalian University of Technology（大连理工大学计算机科学与技术学院）；School of Computer Science and Artificial Intelligence, Liaoning Normal University（辽宁师范大学计算机科学与人工智能学院）；Human-Centered AI Lab, Department of Ecosystem Management, Climate and Biodiversity, BOKU University（BOKU大学生态系统管理与气候生物多样性系人本人工智能实验室）；Institute of Human-Centered Computing, Graz University of Technology（格拉茨技术大学人本计算研究所）
How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem
Institute of Physics（物理研究所）；Institut de Recherche sur les Phenomènes Hors Équilibre（非平衡现象研究所）；Centrale Marseille（马塞尔中央理工学院）；CNRS（法国国家科学研究中心）；Aix-Marseille Université（艾克斯-马赛大学）
Near-Optimal Partially Observable Reinforcement Learning with Partial Online State Information
Department of Electrical Engineering, The State University of New York at Buffalo（电气工程系，纽约州立大学布法罗分校）；Department of Electrical and Computer Engineering, The Ohio State University（电气与计算机工程系，俄亥俄州立大学）；Department of Electrical and Computer Engineering and the Department of Computer Science and Engineering, The Ohio State University（电气与计算机工程系和计算机科学与工程系，俄亥俄州立大学）
Reward Machines for Deep RL in Noisy and Uncertain Environments
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Cornell University（康奈尔大学）；Pontificia Universidad Católica de Chile（智利天主教大学）；Centro Nacional de Inteligencia Artificial（国家人工智能中心）；Schwartz Reisman Institute for Technology and Society（施瓦茨·赖斯曼技术与社会研究所）
Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machines
University of Toronto（多伦多大学）；Vector Institute（矢量研究所）；Schwartz Reisman Institute for Technology and Society（施瓦茨·赖斯曼技术与社会研究所）；Pontificia Universidad Católica de Chile（智利天主教大学）；Centro Nacional de Inteligencia Artificial（国家人工智能中心）
Intention Inference Under Execution Noise: Separating Aleatoric and Epistemic Uncertainty in Social Dilemmas
University of Cape Town（开普敦大学）；Neuroscience Institute, University of Cape Town（开普敦大学神经科学研究所）；National Institute for Theoretical & Computational Sciences, Stellenbosch（斯泰伦博斯理论与计算科学国家研究所）；Institut National de la recherche scientifique, Montreal（魁北克国家科学研究院（蒙特利尔））
Robust Belief-State Policy Learning for Quantum Network Routing Under Decoherence and Time-Varying Conditions
Department of Electrical Engineering, Columbia University（哥伦比亚大学电气工程系）；Department of Electrical Engineering, Imam Khomeini International University（伊玛目·霍梅尼国际大学电气工程系）；Department of Electrical Engineering, Qatar University（卡塔尔大学电气工程系）
SP-GCRL: Influence Maximization on Incomplete Social Graphs
School of Systems Science and Engineering（系统科学与工程学院）；Sun Yat-sen University（中山大学）；Jiangxi University of Finance and Economics（江西财经大学）；The Hong Kong Polytechnic University（香港理工大学）；Anhui University of Science and Technology（安徽理工大学）
Toward an Integrated Decision Making Framework for Optimized Stroke Diagnosis with DSA and Treatment under Uncertainty
Institut Teknologi Sepuluh Nopember (ITS)（十十一月科技学院（ITS））；Universitas Islam Negeri (UIN) Syarif Hidayatullah（赛义德·希达亚图拉伊斯兰国立大学）；Stanford University（斯坦福大学）
Learning Nonlinear Factor Models with Unknown Monotone Links from Incomplete and Noisy Data
School of Computation, Information and Technology, Technical University of Munich, Germany（计算、信息与技术学院，慕尼黑技术大学，德国）；Department of Economics, Virginia Tech, USA（经济系，弗吉尼亚理工学院，美国）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与人工智能研究所）
