Customer-R1：在在线购物中通过基于RL的LLM智能体进行人类行为的个性化模拟
Customer-R1: Personalized Simulation of Human Behaviors via RL-based LLM Agent in Online Shopping
AI总结：
本文提出Customer-R1，通过以显式用户画像为条件的强化学习与动作正确性奖励，提升在线购物场景中LLM智能体对个性化逐步用户行为的模拟保真度。
摘要：
使用大型语言模型（LLM）模拟逐步人类行为已成为一个新兴研究方向，可在各种实际领域中实现应用。尽管先前方法（包括提示、监督微调（SFT）和强化学习（RL））已在建模逐步行为方面展现出前景，但它们主要学习的是群体层面的策略，而没有以用户画像为条件，因此产生的是通用而非个性化的模拟。在本研究中，我们提出一个关键问题：LLM智能体如何才能更好地模拟个性化用户行为？我们介绍了Customer-R1，这是一种在在线购物环境中用于个性化、逐步用户行为模拟的基于RL的方法。我们的策略以显式画像为条件，并通过动作正确性奖励信号优化下一步推理依据和动作生成。在OPeRA数据集上的实验表明，Customer-R1不仅在下一步动作预测任务中显著优于基于提示和基于SFT的基线，而且更好地匹配了用户的动作分布，表明其在个性化行为模拟中具有更高保真度。
英文摘要：
Simulating step-wise human behavior with Large Language Models (LLMs) has become an emerging research direction, enabling applications in various practical domains. While prior methods, including prompting, supervised fine-tuning (SFT), and reinforcement learning (RL), have shown promise in modeling step-wise behavior, they primarily learn a population-level policy without conditioning on a user's persona, yielding generic rather than personalized simulations. In this work, we pose a critical question: how can LLM agents better simulate personalized user behavior? We introduce Customer-R1, an RL-based method for personalized, step-wise user behavior simulation in online shopping environments. Our policy is conditioned on an explicit persona, and we optimize next-step rationale and action generation via action correctness reward signals. Experiments on the OPeRA dataset emonstrate that Customer-R1 not only significantly outperforms prompting and SFT-based baselines in next-action prediction tasks, but also better matches users' action distribution, indicating higher fidelity in personalized behavior simulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
