arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.07926cs.HCcs.AI
收藏

利用大语言模型(LLMs)支持人机协同在线风险数据标注

Leveraging Large Language Models (LLMs) to Support Collaborative Human-AI Online Risk Data Annotation

Vanderbilt University(范德堡大学);Rutgers University(罗格斯大学)

Jinkyung Park, Pamela Wisniewski, Vivek Singh

AI总结:

该立场论文探讨了利用LLMs作为交互式研究工具支持人机协同在线风险数据标注的潜力,梳理了相关早期优势与挑战,并为HCI领域提出利用LLMs推动情境化在线数据标注人机协作的未来研究方向。

摘要:

在这篇立场论文中,我们探讨了将LLMs作为交互式研究工具,促进人类标注员与AI协作,以高效完成大规模在线风险数据标注的潜力。人机协同标注是为各类任务标注大规模复杂数据的一种极具前景的方法。然而,目前针对支持高效人机协同数据标注的工具与方法的研究仍显不足。这一研究缺口十分关键,因为协同标注任务需要支持双向交互式讨论,从而补充标注的细微差别与上下文信息,在主观性极强、高度依赖语境的在线风险场景中尤为如此。因此,我们梳理了使用基于LLMs的工具开展风险标注的早期优势与挑战,并为人机交互(HCI)研究领域提出未来方向,即利用LLMs作为研究工具推动情境化在线数据标注中的人机协作。我们的研究方向与“LLMs作为研究工具”研讨会的宗旨高度契合,该研讨会旨在挖掘HCI研究中使用LLMs处理数据的现有应用与挑战。我们期待从研讨会组织者与参与者处获得宝贵见解,探索LLMs如何助力重塑HCI领域的数据处理方法。

英文摘要:

In this position paper, we discuss the potential for leveraging LLMs as interactive research tools to facilitate collaboration between human coders and AI to effectively annotate online risk data at scale. Collaborative human-AI labeling is a promising approach to annotating large-scale and complex data for various tasks. Yet, tools and methods to support effective human-AI collaboration for data annotation are under-studied. This gap is pertinent because co-labeling tasks need to support a two-way interactive discussion that can add nuance and context, particularly in the context of online risk, which is highly subjective and contextualized. Therefore, we provide some of the early benefits and challenges of using LLMs-based tools for risk annotation and suggest future directions for the HCI research community to leverage LLMs as research tools to facilitate human-AI collaboration in contextualized online data annotation. Our research interests align very well with the purposes of the LLMs as Research Tools workshop to identify ongoing applications and challenges of using LLMs to work with data in HCI research. We anticipate learning valuable insights from organizers and participants into how LLMs can help reshape the HCI community's methods for working with data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑