arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.05063cs.IRcs.AI
收藏

对齐大语言模型以实现可控推荐

Aligning Large Language Models for Controllable Recommendations

College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机与软件学院);Microsoft Research Asia(微软亚洲研究院)

Wensheng Lu, Jianxun Lian, Wei Zhang, Guanghua Li, Mingyang Zhou, Hao Liao, Xing Xie

AI总结:

针对 LLM 推荐系统指令遵循能力不足的问题,本文提出传统推荐模型标签增强的监督任务与强化学习对齐流程,在两个真实数据集上显著提升指令遵循能力并保持高准确性。

摘要:

受大语言模型(LLM)卓越通用智能的启发,研究人员已开始探索将其应用于开创下一代推荐系统——这类系统具备对话性、可解释性和可控性。然而,现有文献主要关注将领域特定知识整合进 LLM 以提升准确性,却往往忽视了遵循指令的能力。为弥补这一差距,我们首先引入了一组监督学习任务,并利用来自传统推荐模型的标签进行增强,旨在明确提升 LLM 遵循推荐特定指令的熟练程度。随后,我们开发了一种基于强化学习的对齐流程,以进一步增强 LLM 回应用户意图并减少格式错误的能力。通过在两个真实世界数据集上的大量实验,我们的方法显著提升了 LLM 在推荐系统中遵循指令的能力,同时保持了高水平的准确性表现。

英文摘要:

Inspired by the exceptional general intelligence of Large Language Models (LLMs), researchers have begun to explore their application in pioneering the next generation of recommender systems - systems that are conversational, explainable, and controllable. However, existing literature primarily concentrates on integrating domain-specific knowledge into LLMs to enhance accuracy, often neglecting the ability to follow instructions. To address this gap, we initially introduce a collection of supervised learning tasks, augmented with labels derived from a conventional recommender model, aimed at explicitly improving LLMs' proficiency in adhering to recommendation-specific instructions. Subsequently, we develop a reinforcement learning-based alignment procedure to further strengthen LLMs' aptitude in responding to users' intentions and mitigating formatting errors. Through extensive experiments on two real-world datasets, our method markedly advances the capability of LLMs to comply with instructions within recommender systems, while sustaining a high level of accuracy performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑