粗略Q学习：无差别、不确定性和不稳定性
Learning to Choose with Categories: Theory and an Application to Algorithmic Prominence
AI总结：
本文提出CQL模型，用于具有随机变化菜单的多臂问题，通过粗略聚合解决估值更新问题，揭示高收益敏感性下可能出现多重稳定均衡、全局稳定混合均衡或无稳定均衡等新现象。
摘要：
本文提出粗略Q学习（CQL），一种用于具有随机变化菜单的多臂问题的强化学习模型。替代品被外生地划分为相似性类别，采样替代品的反馈在类别层面汇总为类别级估值。选择遵循类别估值的多项逻辑分布，估值更新方式与Q学习类似。通过随机逼近，我们推导出均场动力学，并将稳态特性作为估值均衡的平滑类比。该模型在高收益敏感性极限下产生新的长期现象：根据环境不同，CQL可能表现出多个稳定的严格均衡、一个具有类间无差异的全局稳定混合均衡，或完全无稳定均衡，估值和选择概率反而收敛到一个稳定的极限环。这些结果由粗略聚合驱动，而不出现在标准替代品层面的基准模型中。
英文摘要：
We introduce Coarse Q-learning, a reinforcement learning framework in which a decision-maker partitions alternatives into coarse similarity classes, chooses among available classes using logit, and learns class valuations by averaging experienced payoffs. Even in stationary bandit problems with independently drawn menus, choices change the composition of pooled feedback. By deriving the continuous-time mean field ODE limit, we show this endogenous sampling can generate a unique globally attracting mixed equilibrium with limiting indifference, multiple stable strict rankings, or persistent cycling. Learning can reduce welfare even when the coarse representation admits efficient choice. An application demonstrates how digital platforms may exploit this feedback by promoting high-quality products to sustain purchases of inferior products under a common brand.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
