arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.06333cs.HCcs.AIcs.CL
收藏

Parachute:人机交互式LM共写系统的评价

Parachute: Evaluating Interactive Human-LM Co-writing Systems

Pennsylvania State University(宾夕法尼亚州立大学);Carnegie Mellon University(卡内基梅隆大学)

Hua Shen, Tongshuang Wu

摘要:

语言模型(LM)的发展已经引起了人们对使用LM来构建共同写作系统的极大兴趣,在该系统中,人类和LM交互地贡献于共享的写作工件。然而,缺乏研究评估在互动环境中的共同写作系统。我们提出了一个以人为中心的评价框架,降落伞,交互式合作写作系统。Parachute展示了交互评估的综合视图,其中每个评估方面都由分类的实用指标组成。此外,我们提出了降落伞的一个用例来演示如何使用降落伞评估和比较共同编写系统。

英文摘要:

A surge of advances in language models (LMs) has led to significant interest in using LMs to build co-writing systems, in which humans and LMs interactively contribute to a shared writing artifact. However, there is a lack of studies assessing co-writing systems in interactive settings. We propose a human-centered evaluation framework, Parachute, for interactive co-writing systems. Parachute showcases an integrative view of interaction evaluation, where each evaluation aspect consists of categorized practical metrics. Furthermore, we present Parachute with a use case to demonstrate how to evaluate and compare co-writing systems using Parachute.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑