前瞻性强化学习：从生成路径法则到分布价值函数
Anticipatory Reinforcement Learning: From Generative Path-Laws to Distributional Value Functions
AI总结：
本文提出前瞻性强化学习（ARL），通过生成路径法则和分布价值函数，解决非马尔可夫决策过程与传统强化学习架构在单观测轨迹约束下的衔接问题，提升在高波动连续时间环境中的风险管理和策略稳定性。
摘要：
本文介绍前瞻性强化学习（ARL），一种新的框架，旨在弥合非马尔可夫决策过程与传统强化学习架构之间的差距，特别是在单观测轨迹的约束下。在具有跳跃扩散和结构性断裂的环境中，传统基于状态的方法往往无法捕捉到准确预测所需的路径依赖几何结构。我们通过将状态空间提升为签名增强的流形，将过程的历史嵌入为动态坐标。通过利用自洽场方法，智能体维持对未来路径法则的预期代理，从而实现确定性评估预期回报。从随机分支到单次线性评估的转变显著降低了计算复杂性和方差。我们证明该框架保留了基本收缩性质，并在存在重尾噪声的情况下确保了稳定的泛化。我们的结果表明，通过将强化学习扎根于路径空间的拓扑特征，智能体可以在高波动的连续时间环境中实现前瞻性风险管理和更优的策略稳定性。
英文摘要：
This paper introduces Anticipatory Reinforcement Learning (ARL), a novel framework designed to bridge the gap between non-Markovian decision processes and classical reinforcement learning architectures, specifically under the constraint of a single observed trajectory. In environments characterised by jump-diffusions and structural breaks, traditional state-based methods often fail to capture the essential path-dependent geometry required for accurate foresight. We resolve this by lifting the state space into a signature-augmented manifold, where the history of the process is embedded as a dynamical coordinate. By utilising a self-consistent field approach, the agent maintains an anticipated proxy of the future path-law, allowing for a deterministic evaluation of expected returns. This transition from stochastic branching to a single-pass linear evaluation significantly reduces computational complexity and variance. We prove that this framework preserves fundamental contraction properties and ensures stable generalisation even in the presence of heavy-tailed noise. Our results demonstrate that by grounding reinforcement learning in the topological features of path-space, agents can achieve proactive risk management and superior policy stability in highly volatile, continuous-time environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
