arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.12418cs.CL
收藏

任务导向的用户与GPT交互研究:偏移与被忽视的任务

The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions

Microsoft Azure AI(微软Azure人工智能部门);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, Jiawei Han

AI总结:

本文通过大规模分析用户与GPT的真实对话,发现当前NLP研究任务与用户实际需求存在显著差距,特别是“设计”和“规划”等被忽视的任务,并提出了使LLMs更好对齐用户需求的路线图。

摘要:

大型语言模型(LLMs)的最新进展催生了在各种NLP任务中表现出色的模型。然而,目前NLP研究的重点是否准确捕捉了人类用户的真实需求仍不清楚。本文通过大规模收集用户与GPT的对话,对当前NLP研究与真实世界NLP应用需求之间的差异进行了全面分析。我们分析了大量真实用户向GPT提出的查询,将这些查询与现有NLP基准任务进行比较,发现用户频繁向LLMs请求的任务与学术研究中常被研究的任务之间存在显著差距。例如,我们发现“设计”和“规划”等任务在用户交互中很普遍,但在很大程度上被忽视或与传统NLP基准不同。我们调查了这些被忽视的任务,剖析了它们带来的实际挑战,并为使LLMs更好地对齐用户需求提供了路线图见解。

英文摘要:

Recent progress in Large Language Models (LLMs) has produced models that exhibit remarkable performance across a variety of NLP tasks. However, it remains unclear whether the existing focus of NLP research accurately captures the genuine requirements of human users. This paper provides a comprehensive analysis of the divergence between current NLP research and the needs of real-world NLP applications via a large-scale collection of user-GPT conversations. We analyze a large-scale collection of real user queries to GPT. We compare these queries against existing NLP benchmark tasks and identify a significant gap between the tasks that users frequently request from LLMs and the tasks that are commonly studied in academic research. For example, we find that tasks such as ``design'' and ``planning'' are prevalent in user interactions but are largely neglected or different from traditional NLP benchmarks. We investigate these overlooked tasks, dissect the practical challenges they pose, and provide insights toward a roadmap to make LLMs better aligned with user needs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑