Current practice in parameter space exploration in euclidean space is dominated by randomized sampling or design of experiment methods. The biggest issue with these methods is not keeping track of what part of parameter space has been explored and what has not. In this context, we utilize the geometric learning of explored data space using modern machine learning methods to keep track of already explored regions and samples from the regions that are unexplored. For this purpose, we use a modified version of a robust random-cut forest along with other heuristic-based approaches. We demonstrate our method and its progression in two-dimensional Euclidean space but it can be extended to any dimension since the underlying method is generic.
MADE: Exploration via Maximizing Deviation from Explored Regions
UC Berkeley（加州大学伯克利分校）；Facebook AI Research（脸书人工智能研究院）
在在线强化学习（RL）中，有效的探索在具有稀疏奖励的高维环境中仍然特别具有挑战性。在低维环境中，表格参数化是可能的，基于计数的置信上限（UCB）探索方法实现极小极大接近最优率。然而，目前还不清楚如何有效地实现UCB在现实的RL任务，涉及非线性函数逼近。为了解决这个问题，我们提出了一种新的探索方法，通过\textit{maximizing}的偏离下一个政策的占用从探索的区域。我们将此项作为自适应正则化器添加到标准RL目标中，以平衡探索与利用。我们将新的目标与可证明收敛的算法配对，从而产生一个新的内在奖励，以调整现有的奖金。所提出的内在奖励易于实现，并与其他现有的强化学习算法相结合进行探索。作为概念证明，我们评估了各种基于模型和无模型算法的表格示例的新内在奖励，显示了对仅计数探索策略的改进。当对MiniGrid和DeepMind Control Suite基准测试的导航和运动任务进行测试时，我们的方法比最先进的方法显著提高了样本效率。我们的代码可在https://github.com/tianjunz/MADE上获得。
Systematic Evaluation of Initial States and Exploration-Exploitation Strategies in PID Auto-Tuning: A Framework-Driven Approach Applied on Mobile Robots
Department of Computer Information Systems, Collage of Engineering and Computer Science, University of Michigan-Dearborn（计算机信息系统系，工程与计算机科学学院，密歇根大学迪尔伯恩分校）；Department of Electrical and Computer Engineering, Collage of Engineering and Computer Science, University of Michigan-Dearborn（电气与计算机工程系，工程与计算机科学学院，密歇根大学迪尔伯恩分校）
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Université de Montréal（蒙特利尔大学）；Mila - Québec AI Institute（蒙特利尔人工智能研究所）；Université Laval（拉瓦尔大学）；IID - Institut Intelligence et Données（智能与数据研究所）；LawZero；CIFAR AI Chair（CIFAR人工智能主席）
