DROPO: Sim-to-Real Transfer with Offline Domain Randomization

离线领域随机化的统计保证

Statistical Guarantees for Offline Domain Randomization

本文研究了离线领域随机化（ODR）的统计保证，将其视为参数仿真器家族的最大似然估计，并在温和条件和均匀利普希茨连续性假设下提供弱一致和强一致保证，以指导下游离线强化学习的随机化分布选择。

Online vs. Offline Adaptive Domain Randomization Benchmark

Physics simulators have shown great promise for conveniently learning reinforcement learning policies in safe, unconstrained environments. However, transferring the acquired knowledge to the real world can be challenging due to the reality gap. To this end, several methods have been recently proposed to automatically tune simulator parameters with posterior distributions given real data, for use with domain randomization at training time. These approaches have been shown to work for various robotic tasks under different settings and assumptions. Nevertheless, existing literature lacks a thorou…

Dropout's Dream Land: Generalization from Learned Simulators to Reality

A World Model is a generative model used to simulate an environment. World Models have proven capable of learning spatial and temporal representations of Reinforcement Learning environments. In some cases, a World Model offers an agent the opportunity to learn entirely inside of its own dream environment. In this work we explore improving the generalization capabilities from dream environments to real environments (Dream2Real). We present a general approach to improve a controller's ability to transfer from a neural network dream environment to reality at little additional cost. These improvem…

Discovering an Aid Policy to Minimize Student Evasion Using Offline Reinforcement Learning

High dropout rates in tertiary education expose a lack of efficiency that causes frustration of expectations and financial waste. Predicting students at risk is not enough to avoid student dropout. Usually, an appropriate aid action must be discovered and applied in the proper time for each student. To tackle this sequential decision-making problem, we propose a decision support method to the selection of aid actions for students using offline reinforcement learning to support decision-makers effectively avoid student dropout. Additionally, a discretization of student's state space applying tw…

当离线选择器无法超越最佳单一模型：基于edX辍学预测的诊断研究

When Offline Selectors Cannot Beat the Best Single Model: A Diagnostic Study on edX Dropout Prediction

针对离线选择器在实践中的表现常不如最佳单一模型的问题，提出三阶段诊断方法，通过k-NN标签一致性、离线学习器性能比较和状态特征消融实验，识别瓶颈为局部表示模糊性，建议改进状态或收集新数据而非调优学习器。

Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning

Offline Reinforcement Learning promises to learn effective policies from previously-collected, static datasets without the need for exploration. However, existing Q-learning and actor-critic based off-policy RL algorithms fail when bootstrapping from out-of-distribution (OOD) actions or states. We hypothesize that a key missing ingredient from the existing methods is a proper treatment of uncertainty in the offline setting. We propose Uncertainty Weighted Actor-Critic (UWAC), an algorithm that detects OOD state-action pairs and down-weights their contribution in the training objectives accordi…