arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.04918cs.CLcs.AI

提示隐式话语关系标注

Prompting Implicit Discourse Relation Annotation

Saarland University(萨尔大学);Utrecht University(乌得勒支大学)

Frances Yung, Mansoor Ahmad, Merel Scholman, Vera Demberg

AI总结:

本研究探讨多种提示技术以提升ChatGPT在隐式话语关系分类中的表现,但实验显示即使分解任务和复杂提示,准确率仍无显著提升,表明该任务在零样本或少样本下尚难解决。

摘要:

预训练的大型语言模型,如ChatGPT,在没有监督训练的情况下在各种推理任务中表现出色,并被发现优于众包工作者。然而,在由标准多项选择题提示的隐式话语关系分类任务中,ChatGPT的表现仍远未令人满意,且明显逊于最先进的监督方法。本研究探讨了几种经过验证的提示技术,以提升ChatGPT对话语关系的识别能力。特别是,我们尝试将涉及众多抽象标签的分类任务分解为更小的子任务。然而,实验结果表明,即使采用复杂的提示工程,推理准确率也几乎没有变化,这表明隐式话语关系分类在零样本或少样本设置下尚无法解决。

英文摘要:

Pre-trained large language models, such as ChatGPT, archive outstanding performance in various reasoning tasks without supervised training and were found to have outperformed crowdsourcing workers. Nonetheless, ChatGPT's performance in the task of implicit discourse relation classification, prompted by a standard multiple-choice question, is still far from satisfactory and considerably inferior to state-of-the-art supervised approaches. This work investigates several proven prompting techniques to improve ChatGPT's recognition of discourse relations. In particular, we experimented with breaking down the classification task that involves numerous abstract labels into smaller subtasks. Nonetheless, experiment results show that the inference accuracy hardly changes even with sophisticated prompt engineering, suggesting that implicit discourse relation classification is not yet resolvable under zero-shot or few-shot settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑