We investigate the use of temporally abstract actions, or macro-actions, in the solution of Markov decision processes. Unlike current models that combine both primitive actions and macro-actions and leave the state space unchanged, we propose a hierarchical model (using an abstract MDP) that works with macro-actions only, and that significantly reduces the size of the state space. This is achieved by treating macroactions as local policies that act in certain regions of state space, and by restricting states in the abstract MDP to those at the boundaries of regions. The abstract MDP approximates the original and can be solved more efficiently. We discuss several ways in which macro-actions can be generated to ensure good solution quality. Finally, we consider ways in which macro-actions can be reused to solve multiple, related MDPs; and we show that this can justify the computational overhead of macro-action generation.
Hierarchical Representation Learning for Markov Decision Processes
Dept. Information and Communication Technologies, Universitat Pompeu Fabra（庞培法布拉大学信息与通信技术系）；Mila – Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）
HAG: Hierarchical Demographic Tree-based Agent Generation for Topic-Adaptive Simulation
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS（中国科学院计算技术研究所人工智能安全国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences（中国科学院大学前沿交叉科学学院）
Partially Observable Markov Decision Process Modelling for Assessing Hierarchies
Insight Centre for Data Analytics（洞察数据分析中心）；School of Computer Science, University College Dublin（都柏林大学学院计算机科学学院）；Data Science Institute, National University of Ireland, Galway（爱尔兰国立大学戈尔韦数据科学研究所）
A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shenyang Aircraft Design and Research Institute (SADRI)（沈阳飞机设计研究所）
Hierarchical AI Multi-Agent Fundamental Investing: Evidence from China's A-Share Market
Faculty of Business and Economics, University of Hong Kong（香港大学商学院）；Department of Mathematics, University of Hong Kong（香港大学数学系）；Department of Industrial Engineering and Decision Analytics, Hong Kong University of Science and Technology（香港科学与技术大学工业工程与决策分析系）
