Reinforcement learning-assisted quantum architecture search for variational quantum algorithms
Akash Kundu
备注：
With many pages, figures and tables, I, Akash Kundu upload the final version of my thesis! Including reviewers response and a kind of brief overview of recent quantum architecture search methods
A significant hurdle in the noisy intermediate-scale quantum (NISQ) era is identifying functional quantum circuits. These circuits must also adhere to the constraints imposed by current quantum hardware limitations. Variational quantum algorithms (VQAs), a class of quantum-classical optimization algorithms, were developed to address these challenges in the currently available quantum devices. However, the overall performance of VQAs depends on the initialization strategy of the variational circuit, the structure of the circuit (also known as ansatz), and the configuration of the cost function. Focusing on the structure of the circuit, in this thesis, we improve the performance of VQAs by automating the search for an optimal structure for the variational circuits using reinforcement learning (RL). Within the thesis, the optimality of a circuit is determined by evaluating its depth, the overall count of gates and parameters, and its accuracy in solving the given problem. The task of automating the search for optimal quantum circuits is known as quantum architecture search (QAS). The majority of research in QAS is primarily focused on a noiseless scenario. Yet, the impact of noise on the QAS remains inadequately explored. In this thesis, we tackle the issue by introducing a tensor-based quantum circuit encoding, restrictions on environment dynamics to explore the search space of possible circuits efficiently, an episode halting scheme to steer the agent to find shorter circuits, a double deep Q-network (DDQN) with an $ε$-greedy policy for better stability. The numerical experiments on noiseless and noisy quantum hardware show that in dealing with various VQAs, our RL-based QAS outperforms existing QAS. Meanwhile, the methods we propose in the thesis can be readily adapted to address a wide range of other VQAs.
A Conceptual Framework for Externally-influenced Agents: An Assisted Reinforcement Learning Review
Federation University（联邦大学）；Deakin University（迪肯大学）；Universidad Central de Chile（智利中央大学）；University of Alberta（阿尔伯塔大学）；Action Research Associates（行动研究协会）
Reducing Action Space: Reference-Model-Assisted Deep Reinforcement Learning for Inverter-based Volt-Var Control
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学清华-伯克利深圳学院）；Shanghai University of Electric Power（上海电力大学）；Tsinghua University（清华大学）；Beihang University（北京航空航天大学）
Enhancing variational quantum state diagonalization using reinforcement learning techniques
Institute of Theoretical and Applied Informatics, Polish Academy of Sciences（波兰科学院理论与应用信息学研究所）；Silesian University of Technology（西里西亚理工大学）；Warsaw University of Technology（华沙理工大学）；Leiden University（莱顿大学）
RLAE: Reinforcement Learning-Assisted Ensemble for LLMs
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities
National Defense University（国防大学）；University of Kent（肯特大学）；Beijing Jiaotong University（北京交通大学）；Nanyang Technological University（南洋理工大学）；Qatar University（卡塔尔大学）；Beihang University（北京航空航天大学）；National University of Singapore（新加坡国立大学）；University of Alberta（阿尔伯塔大学）；Polish Academy of Sciences（波兰科学院）；Indian Statistical Institute（印度统计学院）；Kyungpook National University（庆北国立大学）
Department of Computer Science, Cornell University（康奈尔大学计算机科学系）；Subharmonic Technologies Inc.（Subharmonic Technologies公司）；Department of Economics, Cornell University（康奈尔大学经济学系）；Department of Engineering Physics and Computation, Technical University of Munich（慕尼黑技术大学工程物理与计算系）；Department of Mechanical Engineering, University of Washington（华盛顿大学机械工程系）
DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing
The University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Texas A&M University at Corpus Christi（德克萨斯农工大学科珀斯克里斯蒂分校）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning
State Key Laboratory of Ocean Engineering, School of Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, People’s Republic of China（海洋工程国家重点实验室，海洋与土木工程学院，上海交通大学，上海200240，中华人民共和国）；State Key Laboratory of Fluid Power and Mechatronic Systems, Department of Engineering Mechanics, Zhejiang University, Hangzhou 310027, People’s Republic of China（流体动力与机械系统国家重点实验室，工程力学系，浙江大学，杭州310027，中华人民共和国）
Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning
Department of Computer Science and Engineering, Sungkyunkwan University, Suwon, Republic of Korea（计算机科学与工程系，成均馆大学，苏旺，韩国）；Department of Artificial Intelligence, Sungkyunkwan University, Suwon, Republic of Korea（人工智能系，成均馆大学，苏旺，韩国）
Adaptive sampling using variational autoencoder and reinforcement learning
Department of Engineering Cybernetics, Norwegian University of Science and Technology（工程 cybernetics 系，挪威科学与技术大学）；Department of Engineering Sciences, University of Agder（工程科学系，阿格德大学）
Variational Bayesian Reinforcement Learning with Regret Bounds
DeepMind（深度思维）
在强化学习中，Q值总结了智能体将获得的预期未来奖励。然而，他们无法捕捉到这些奖励的认知不确定性。在这项工作中，我们推导出一个新的Bellman算子与相关的不动点，我们称之为“知识值”。这些K值将预期的未来回报和认知的不确定性压缩到一个值中，因此高不确定性、高回报或两者都可以产生高K值。关键原则是赋予代理人一个寻求风险的效用函数，该函数经过精心调整，以平衡探索和开发。当智能体在K值上遵循玻尔兹曼策略时，它会产生$\tilde O(L \sqrt{S A T})$的贝叶斯后悔界，其中$L$是时间范围，$S$是状态总数，$A$是动作数，$T$是经过的时间步数。我们展示了这种方法与强化学习研究中的软最大和最大熵链的深层联系。
