检验在真实世界预测知识结构中使用时间差增量Delta-Bar-Delta
Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures
摘要：
预测和预测知识最近不仅在改善机器人控制方面取得了成功，而且在从工业过程控制到康复的其他应用方面也取得了成功。使这些预测方法非常适合机器人技术的一个特性是，它们可以通过与环境的交互在线学习和增量学习。然而，许多预测学习方法的剩余挑战是预测学习参数的适当选择，特别是控制学习机对其预测的更新幅度（学习率或步长）的参数。为了应对这一挑战，我们研究了使用传感器丰富的机器人arm. Our方法的选择，时间差增量（TIDBD），学习和适应步长的特征级别;重要的是，TIDBD允许步长调整和表示学习同时发生。我们表明，TIDBD是一个实用的替代经典的时间差（TD）学习通过广泛的参数搜索。这两种方法都在预测机器人数据流的未来方面进行了改进。此外，使用像TIDBD这样的步长自适应方法似乎允许系统自动检测和表征机器人应用中的常见传感器故障。总之，这些结果有望提高机器人设备以稳健的方式从与环境的交互中学习的能力，为自主智能体和机器人提供关键能力。
英文摘要：
Predictions and predictive knowledge have seen recent success in improving not only robot control but also other applications ranging from industrial process control to rehabilitation. A property that makes these predictive approaches well suited for robotics is that they can be learned online and incrementally through interaction with the environment. However, a remaining challenge for many prediction-learning approaches is an appropriate choice of prediction-learning parameters, especially parameters that control the magnitude of a learning machine's updates to its predictions (the learning rate or step size). To begin to address this challenge, we examine the use of online step-size adaptation using a sensor-rich robotic arm. Our method of choice, Temporal-Difference Incremental Delta-Bar-Delta (TIDBD), learns and adapts step sizes on a feature level; importantly, TIDBD allows step-size tuning and representation learning to occur at the same time. We show that TIDBD is a practical alternative for classic Temporal-Difference (TD) learning via an extensive parameter search. Both approaches perform comparably in terms of predicting future aspects of a robotic data stream. Furthermore, the use of a step-size adaptation method like TIDBD appears to allow a system to automatically detect and characterize common sensor failures in a robotic application. Together, these results promise to improve the ability of robotic devices to learn from interactions with their environments in a robust way, providing key capabilities for autonomous agents and robots.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
