Monte Carlo Rollout Policy for Recommendation Systems with Dynamic User Behavior

有限状态躁动多臂老虎机的可索引性与Rollout策略

Indexability of Finite State Restless Multi-Armed Bandit and Rollout Policy

该研究针对有限状态躁动多臂老虎机的无限时域折扣奖励最大化问题，分析单臂模型的可索引性，提出值迭代验证方法，还研究了在线rollout策略，实验表明二者性能均优于短视策略。

策略库CBF：通过并行滚动预测实现有限时间范围内的运行时安全

Policy Library CBF: Finite-Horizon Safety at Runtime via Parallel Rollouts

本文提出PL-CBF，通过并行有限时间滚动预测评估备用策略库，选择最安全模式并最小修改名义策略以确保安全，实验显示在保持毫秒级运行时间的同时提升了安全覆盖率。

The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation

Function approximation is widely used in reinforcement learning to handle the computational difficulties associated with very large state spaces. However, function approximation introduces errors which may lead to instabilities when using approximate dynamic programming techniques to obtain the optimal policy. Therefore, techniques such as lookahead for policy improvement and m-step rollout for policy evaluation are used in practice to improve the performance of approximate dynamic programming with function approximation. We quantitatively characterize, for the first time, the impact of lookah…

Data-driven Rollout for Deterministic Optimal Control

We consider deterministic infinite horizon optimal control problems with nonnegative stage costs. We draw inspiration from learning model predictive control scheme designed for continuous dynamics and iterative tasks, and propose a rollout algorithm that relies on sampled data generated by some base policy. The proposed algorithm is based on value and policy iteration ideas, and applies to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure.

面向带信息论约束马尔可夫决策过程（MDP）的基于Rollout的近似动态规划

Rollout-Based Approximate Dynamic Programming for MDPs with Information-Theoretic Constraints

针对带信息论约束的有限时域MDP求解难题，提出基于截断rollout的前后向ADP框架，通过离线短时域基础策略近似加在线前瞻最小化求解，兼具收敛保证与更优的成本表现。

基于信念聚合与Rollout的自适应网络安全策略

Adaptive Network Security Policies via Belief Aggregation and Rollout

针对现有强化学习方法在网络安全策略适应中缺乏性能保证且适应缓慢的问题，提出一种结合粒子滤波信念估计、离线聚合与在线rollout的可扩展方法，提供理论保证并在多个基准上优于现有方法。

利用随机仿真推演保证机器人系统性能

Guarantees on Robot System Performance Using Stochastic Simulation Rollouts

本文提出基于随机仿真轨迹推演的有限样本安全认证方法，为机器人控制策略的代价风险指标和失败概率提供分布无关保证，并支持约束测试、sim-to-real 偏移鲁棒性及多策略选择校正。

摊销反馈规划：将基于模型的展开转化为可执行策略

Amortized Feedback Planning: Turning Model-Based Rollouts into Executable Policies

本文提出摊销反馈规划方法，通过贝尔曼梯度细化将模型展开转化为可执行策略，在非LQG和推力任务中显著降低计算成本并保持性能。

基于不确定性引导混合动力学的稳定多步回滚

Stable Multi-Step Rollouts via Uncertainty-Guided Hybrid Dynamics

本文提出一种与模型无关的不确定性引导混合动力学框架，用于解决基于模型的强化学习中多步回滚不稳定的问题，在杜芬振子实验中实现了稳定长 horizon 预测并改善了成本-努力权衡。

并行策略梯度方法用于非线性反馈控制器参数优化

Parallel Policy-Gradient Methods for Parameter Optimization of Nonlinear Feedback Controllers

本文提出一种时间并行策略梯度框架，用于非线性反馈控制器的参数优化，通过残差最小化和高斯-牛顿迭代加速梯度评估，并在惯性轮摆示例中验证了性能提升与计算优势。