评估对话式推荐系统中的用户体验:跨经典与LLM驱动方法的系统综述
Evaluating User Experience in Conversational Recommender Systems: A Systematic Review Across Classical and LLM-Powered Approaches
AI总结:
本综述基于PRISMA方法分析2017-2025年间23项实证研究,揭示对话式推荐系统用户体验评估存在事后调查主导、情感构念缺失及自适应行为关联不足等问题,并提出面向LLM的评估议程。
摘要:
对话式推荐系统(CRSs)在各领域受到越来越多的研究关注,但其用户体验(UX)评估仍然有限。现有综述大多忽视实证用户体验研究,尤其是在自适应和基于大语言模型(LLM)的对话式推荐系统中。为弥补这一空白,我们遵循PRISMA指南进行了一项系统综述,综合了2017年至2025年间发表的23项实证研究。我们分析了用户体验如何被概念化、测量,以及如何受到领域、自适应性和LLM的影响。我们的发现揭示了持续存在的局限性:事后调查占主导地位,回合级情感用户体验构念很少被评估,自适应行为很少与用户体验结果相关联。基于LLM的对话式推荐系统带来了进一步挑战,包括认知不透明性和冗长性,但评估很少涉及这些问题。我们贡献了用户体验指标的结构化综合、自适应与非自适应系统的比较分析,以及面向LLM感知的用户体验评估的前瞻性议程。这些发现支持开发更透明、更具吸引力且以用户为中心的对话式推荐系统评估实践。
英文摘要:
Conversational Recommender Systems (CRSs) are receiving growing research attention across domains, yet their user experience (UX) evaluation remains limited. Existing reviews largely overlook empirical UX studies, particularly in adaptive and large language model (LLM)-based CRSs. To address this gap, we conducted a systematic review following PRISMA guidelines, synthesising 23 empirical studies published between 2017 and 2025. We analysed how UX has been conceptualised, measured, and shaped by domain, adaptivity, and LLM. Our findings reveal persistent limitations: post hoc surveys dominate, turn-level affective UX constructs are rarely assessed, and adaptive behaviours are seldom linked to UX outcomes. LLM-based CRSs introduce further challenges, including epistemic opacity and verbosity, yet evaluations infrequently address these issues. We contribute a structured synthesis of UX metrics, a comparative analysis of adaptive and nonadaptive systems, and a forward-looking agenda for LLM-aware UX evaluation. These findings support the development of more transparent, engaging, and user-centred CRS evaluation practices.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。