发表机构
The Hong Kong Polytechnic University; Columbia University（香港理工大学; 哥伦比亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究具有无卖空约束和未知投资机会集的连续时间均值-方差投资组合选择问题，通过引入无熵辅助探索问题，证明特定条件下最优高斯策略均值与原始问题一致，进而开发无模型RL算法并通过数值例子展示其性能。
AI 中文摘要
我们从强化学习（RL）的角度研究具有无卖空约束和未知投资机会集的连续时间均值-方差投资组合选择问题。该问题是一个约束随机线性二次控制问题，Wang等人（2020）的熵正则化探索公式在理论分析上存在困难，因为对随机策略的支持施加约束会使可处理的高斯探索无效。为应对这一挑战，我们引入了一个无熵的辅助探索问题，其中探索策略仍然是高斯的，其样本可能违反无卖空要求，但均值满足该要求。然后我们证明，对于合适的探索方差选择，辅助问题的最优高斯策略的均值与原始问题的最优策略一致。基于这一理论结果，我们开发了一种无模型RL算法，该算法直接从轨迹数据中学习辅助（进而原始）问题的最优策略，而无需估计投资机会集。一个数值例子展示了所提出算法的性能。
英文摘要
We study continuous-time mean-variance portfolio selection with no-shorting constraints and unknown investment opportunity sets from a reinforcement learning (RL) perspective. The problem is a constrained stochastic linear -- quadratic control problem for which the entropy-regularized exploratory formulation of Wang et al. (2020) leads to difficulty in theoretical analysis, because enforcing the constraint on the support of randomized policies nullifies the tractable Gaussian exploration. To tackle this challenge, we introduce an auxiliary exploratory problem without entropy in which exploratory policies are still Gaussian whose samples may violate the no-shorting requirement but their means satisfy it. We then prove that, for a suitable choice of exploration variance, the mean of the optimal Gaussian policy of the auxiliary problem coincides with the optimal policy of the original problem. Motivated by this theoretical result, we develop a model-free RL algorithm that learns the optimal policy of the auxiliary (and hence the original) problem directly from trajectory data without estimating the investment opportunity set. A numerical example demonstrates the performance of the proposed algorithm.
