arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.13221cs.CV
收藏

多模态标签相关性排序的强化学习方法

Multimodal Label Relevance Ranking via Reinforcement Learning

Tencent Youtu Lab(腾讯优图实验室);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)

Taian Guo, Taolin Zhang, Haoqian Wu, Hanjun Li, Ruizhi Qiao, Xing Sun

AI总结:

针对多标签识别忽视偏序关系的问题,提出基于近端策略优化的LR2PPO方法,利用偏序对训练奖励模型并设计排序专用策略,提升排序性能并减少标注需求,在LRMovieNet基准上取得最先进结果。

摘要:

传统的多标签识别方法往往侧重于标签置信度,常常忽略了与人类偏好一致的偏序关系的关键作用。为解决这些问题,我们提出了一种新颖的多模态标签相关性排序方法,名为基于近端策略优化的标签相关性排序(LR\ extsuperscript{2}PPO),该方法能有效识别标签间的偏序关系。LR\ extsuperscript{2}PPO首先利用目标域中的偏序对训练一个奖励模型,旨在捕捉特定场景中固有的人类偏好。此外,我们精心设计了针对排序任务的状态表示和策略损失,使LR\ extsuperscript{2}PPO能够提升标签相关性排序模型的性能,并大幅减少迁移到新场景时所需的偏序标注。为辅助评估我们的方法及类似方法,我们进一步提出了一个新的基准数据集LRMovieNet,包含多模态标签及其对应的偏序数据。大量实验表明,我们的LR\ extsuperscript{2}PPO算法达到了最先进的性能,证明了其在解决多模态标签相关性排序问题上的有效性。代码和所提出的LRMovieNet数据集已在\url{https://github.com/ChazzyGordon/LR2PPO}公开提供。

英文摘要:

Conventional multi-label recognition methods often focus on label confidence, frequently overlooking the pivotal role of partial order relations consistent with human preference. To resolve these issues, we introduce a novel method for multimodal label relevance ranking, named Label Relevance Ranking with Proximal Policy Optimization (LR\textsuperscript{2}PPO), which effectively discerns partial order relations among labels. LR\textsuperscript{2}PPO first utilizes partial order pairs in the target domain to train a reward model, which aims to capture human preference intrinsic to the specific scenario. Furthermore, we meticulously design state representation and a policy loss tailored for ranking tasks, enabling LR\textsuperscript{2}PPO to boost the performance of label relevance ranking model and largely reduce the requirement of partial order annotation for transferring to new scenes. To assist in the evaluation of our approach and similar methods, we further propose a novel benchmark dataset, LRMovieNet, featuring multimodal labels and their corresponding partial order data. Extensive experiments demonstrate that our LR\textsuperscript{2}PPO algorithm achieves state-of-the-art performance, proving its effectiveness in addressing the multimodal label relevance ranking problem. Codes and the proposed LRMovieNet dataset are publicly available at \url{https://github.com/ChazzyGordon/LR2PPO}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑