arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.08329cs.HC
收藏

理解大语言模型交互中的用户体验

Understanding User Experience in Large Language Model Interactions

Jiayin Wang, Weizhi Ma, Peijie Sun, Min Zhang, Jian-Yun Nie

AI总结:

本研究通过分析真实交互日志和用户调查,构建了LLM用户意图分类体系,揭示了用户满意度的关键洞察,并提出了以用户为中心的未来研究方向。

摘要:

在快速演变的大语言模型(LLMs)领域中,大多数研究主要将其视为独立的个体,侧重于通过标准化基准评估其能力并增强其通用智能。然而,这一视角往往忽视了LLMs作为以用户为中心的服务在人机协作中的重要作用。随着LLMs日益融入人们的日常与职业交互,这一研究空白变得愈发关键。本研究通过探索四个关键方面来回应理解用户对LLMs满意度的迫切需求:理解用户意图、审视用户体验、解决用户对当前LLM服务的主要关切,以及规划未来研究路径以加强人机协作。我们的研究基于对真实用户交互日志的分析和人工验证,建立了LLM交互中7类用户意图的分类体系。随后,我们开展了一项用户调查,以衡量用户对LLM服务的满意度,涵盖使用频率、跨意图的体验以及主要关切。该调查收集了411份匿名回复,揭示了关于当前用户与LLMs互动状况的11项第一手见解。基于这一实证分析,我们确定了6个以用户视角为先的LLM发展未来研究方向。这种以用户为中心的方法对于打造不仅在技术上先进、而且能够契合人类交互和现实世界应用复杂现实的LLMs至关重要。

英文摘要:

In the rapidly evolving landscape of large language models (LLMs), most research has primarily viewed them as independent individuals, focusing on assessing their capabilities through standardized benchmarks and enhancing their general intelligence. This perspective, however, tends to overlook the vital role of LLMs as user-centric services in human-AI collaboration. This gap in research becomes increasingly critical as LLMs become more integrated into people's everyday and professional interactions. This study addresses the important need to understand user satisfaction with LLMs by exploring four key aspects: comprehending user intents, scrutinizing user experiences, addressing major user concerns about current LLM services, and charting future research paths to bolster human-AI collaborations. Our study develops a taxonomy of 7 user intents in LLM interactions, grounded in analysis of real-world user interaction logs and human verification. Subsequently, we conduct a user survey to gauge their satisfaction with LLM services, encompassing usage frequency, experiences across intents, and predominant concerns. This survey, compiling 411 anonymous responses, uncovers 11 first-hand insights into the current state of user engagement with LLMs. Based on this empirical analysis, we pinpoint 6 future research directions prioritizing the user perspective in LLM developments. This user-centered approach is essential for crafting LLMs that are not just technologically advanced but also resonate with the intricate realities of human interactions and real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑