发表机构
Harvard University; Purdue University; Dauch Center for the Management of Manufacturing Enterprises（哈佛大学; 普渡大学; 多奇制造企业管理中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究全球运营中意外冲击下的决策问题，提出双时间尺度分层强化学习框架，通过同步更新长期和短期策略增强弹性，在二手车案例中提高利润并保持稳定，且不改变现有决策结构。
AI 中文摘要
全球运营中意外冲击频发，需要能随市场和运营条件变化的决策规则。许多运营系统具分层结构，长期和短期决策追求共同目标。本文研究分层强化学习如何通过联合调整这些相互依存的规则增强弹性。开发了双时间尺度分层强化学习框架，在各自时间尺度上调整长期和短期策略，同步更新并证明了耦合双时间尺度学习的收敛性。在二手车案例研究中，相对于最强部分自适应基准，该框架在联合供需冲击下平均利润提高9.2%，在长期冲击场景下提高11.8%，并保持更稳定利润轨迹。短期适应应对常规季节性和单边干扰，联合适应通过干扰和恢复产生更高更稳定利润，且不改变现有决策结构。
英文摘要
Unexpected shocks recur in global operations, requiring decision rules that adapt as market and operating conditions change. Many operational systems also have hierarchical structures in which long-term and short-term decisions pursue a shared objective. We study how hierarchical reinforcement learning can strengthen resilience by adapting these interdependent rules jointly. We develop a two-timescale hierarchical reinforcement learning framework that adapts long-term and short-term policies at their respective time scales. Because the policies are interdependent, we synchronize their updates and prove, to our knowledge, the first convergence guarantees for coupled two-timescale learning. Over $T$ periods, our policies' average gap from an optimal policy pair is $O(T^{-1/2})$, improving to $O(\log T/T)$ when poor decisions produce clearer profit losses. In a used-car case study, inventory replenishment is the long-term decision and customer-arrival pricing the short-term decision. Relative to the strongest partially adaptive benchmark, the framework increases mean profit by $9.2\%$ under joint demand-supply shocks and by $11.8\%$ under a prolonged shock scenario, while maintaining a more stable profit trajectory over time. Short-term adaptation addresses routine seasonality and one-sided disruptions by responding immediately to changing conditions. Under joint demand-supply shocks, however, it is insufficient alone; long-term adaptation is also needed to create favorable conditions for short-term decisions. Joint adaptation thus yields higher and more stable profits through disruption and recovery. Because many organizations already use hierarchical planning, the framework strengthens operational resilience without altering existing decision structures.
