arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04119cs.RO
收藏

面向自主机器人探索的上下文多臂老虎机中的主动推断

Active Inference in Contextual Multi-Armed Bandits for Autonomous Robotic Exploration

Smead Aerospace Engineering Sciences Department, University of Colorado Boulder(科罗拉多大学博尔德分校航空航天工程科学系);Jet Propulsion Laboratory, California Institute of Technology(加州理工学院喷气推进实验室);Astrophysical & Planetary Sciences Department, University of Colorado Boulder(科罗拉多大学博尔德分校天体物理与行星科学系)

Shohei Wakayama, Alberto Candela, Paul Hayne, Nisar Ahmed

AI总结:

本文将主动推断用于CMAB框架下的模拟矿物勘探选址,基于AVIRIS-NG高光谱数据和地质标签验证其在含噪、偏好在线变化场景中优于标准老虎机方法。

摘要:

在不确定环境中,从多个备选方案中自主选择最优方案以收集数据具有挑战性。当可获得关于各选项的辅助信息时,此类问题可被表述为上下文多臂老虎机(CMAB)。受神经启发的主动推断因其能够利用期望自由能目标函数平衡探索与利用而受到关注。以往研究使用合成数据展示了基于主动推断的CMAB策略的有效性,与之不同,本研究旨在利用一个模拟矿物学调查选址问题,将主动推断应用于现实场景。来自内华达州Cuprite的AVIRIS-NG高光谱数据被用作预测结果概率的上下文信息,而地质学家的矿物标签则代表结果。蒙特卡洛模拟评估了主动推断在专家偏好变化时的鲁棒性。结果表明,在真实世界含噪且有偏的数据上,主动推断比标准老虎机方法需要更少的迭代;并且当结果偏好在线变化时,它通过调整选择策略以契合专家的转变,表现得更好。

英文摘要:

Autonomous selection of optimal options for data collection from multiple alternatives is challenging in uncertain environments. When secondary information about options is accessible, such problems can be framed as contextual multi-armed bandits (CMABs). Neuro-inspired active inference has gained interest for its ability to balance exploration and exploitation using the expected free energy objective function. Unlike previous studies that showed the effectiveness of active inference based strategy for CMABs using synthetic data, this study aims to apply active inference to realistic scenarios, using a simulated mineralogical survey site selection problem. Hyperspectral data from AVIRIS-NG at Cuprite, Nevada, serves as contextual information for predicting outcome probabilities, while geologists' mineral labels represent outcomes. Monte Carlo simulations assess the robustness of active inference against changing expert preferences. Results show that active inference requires fewer iterations than standard bandit approaches with real-world noisy and biased data, and performs better when outcome preferences vary online by adapting the selection strategy to align with expert shifts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑