arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2507.00518cs.LGcs.IR
收藏

使用von Mises-Fisher采样在超球面嵌入中探索大规模动作集

Exploring Large Action Sets with Hyperspherical Embeddings using von Mises-Fisher Sampling

Deezer Research(Deep Research(Deezer));SPEIT, Shanghai Jiao Tong University(上海交通大学 SPEIT 实验室)

Walid Bendada, Guillaume Salha-Galvan, Romain Hennequin, Théo Bontempelli, Thomas Bouabça, Tristan Cazenave

AI总结:

本文提出vMF-exp方法,利用von Mises-Fisher分布采样超球面嵌入并探索最近邻,以可扩展方式解决大规模动作集探索问题,并在理论和实验中验证其与Boltzmann探索等效且可实际部署。

摘要:

本文提出了von Mises-Fisher探索(vMF-exp),这是一种可扩展的方法,用于在强化学习问题中探索大规模动作集,其中超球面嵌入向量表示这些动作。vMF-exp首先使用von Mises-Fisher分布采样一个状态嵌入表示,然后探索该表示的最近邻,这可以扩展到几乎无限数量的候选动作。我们证明,在理论假设下,vMF-exp渐近地保持与Boltzmann探索(B-exp)相同的探索每个动作的概率,而B-exp是一种流行的替代方法,但由于需要为每个动作计算softmax值而存在可扩展性问题。因此,vMF-exp作为B-exp的可扩展替代方案,用于探索具有超球面嵌入的大规模动作集。在模拟数据、真实世界公开数据上的实验,以及vMF-exp在全球音乐流媒体服务推荐系统上的成功大规模部署,实证验证了所提方法的关键性质。

英文摘要:

This paper introduces von Mises-Fisher exploration (vMF-exp), a scalable method for exploring large action sets in reinforcement learning problems where hyperspherical embedding vectors represent these actions. vMF-exp involves initially sampling a state embedding representation using a von Mises-Fisher distribution, then exploring this representation's nearest neighbors, which scales to virtually unlimited numbers of candidate actions. We show that, under theoretical assumptions, vMF-exp asymptotically maintains the same probability of exploring each action as Boltzmann Exploration (B-exp), a popular alternative that, nonetheless, suffers from scalability issues as it requires computing softmax values for each action. Consequently, vMF-exp serves as a scalable alternative to B-exp for exploring large action sets with hyperspherical embeddings. Experiments on simulated data, real-world public data, and the successful large-scale deployment of vMF-exp on the recommender system of a global music streaming service empirically validate the key properties of the proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑