CASA:因果驱动的论证充分性评估
CASA: Causality-driven Argument Sufficiency Assessment
AI总结:
本文提出零样本因果驱动框架CASA,利用充分性概率和大语言模型生成并修订不一致上下文来评估论证充分性,实验证明其能准确识别不充分论证并提升学生写作质量。
摘要:
论证充分性评估任务旨在判断给定论证的前提是否支持其结论。为解决该任务,现有工作通常基于人工标注的数据训练分类器。然而,数据标注工作繁重,且由于主观标准,标注往往不一致。受因果文献中充分性概率(PS)定义的启发,我们提出了CASA,一个零样本、因果驱动的论证充分性评估框架。PS衡量当前提事件和结论事件均未发生时,引入前提事件导致结论发生的可能性。为估计这一概率,我们提出使用大语言模型(LLMs)生成与前提和结论不一致的上下文,并通过注入前提事件对其进行修订。在两个逻辑谬误检测数据集上的实验表明,CASA能够准确识别不充分的论证。我们进一步将CASA部署在一个写作辅助应用中,发现CASA生成的建议增强了学生所写论证的充分性。代码和数据可在https://github.com/xxxiaol/CASA获取。
英文摘要:
The argument sufficiency assessment task aims to determine if the premises of a given argument support its conclusion. To tackle this task, existing works often train a classifier on data annotated by humans. However, annotating data is laborious, and annotations are often inconsistent due to subjective criteria. Motivated by the definition of probability of sufficiency (PS) in the causal literature, we proposeCASA, a zero-shot causality-driven argument sufficiency assessment framework. PS measures how likely introducing the premise event would lead to the conclusion when both the premise and conclusion events are absent. To estimate this probability, we propose to use large language models (LLMs) to generate contexts that are inconsistent with the premise and conclusion and revise them by injecting the premise event. Experiments on two logical fallacy detection datasets demonstrate that CASA accurately identifies insufficient arguments. We further deploy CASA in a writing assistance application, and find that suggestions generated by CASA enhance the sufficiency of student-written arguments. Code and data are available at https://github.com/xxxiaol/CASA.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。