arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.11879cs.AIcs.CL
收藏

基于语言模型案例推理的逻辑谬误分类

Case-Based Reasoning with Language Models for Classification of Logical Fallacies

Information Sciences Institute, University of Southern California(南加州大学信息科学研究所);University of Southern California(南加州大学);armasuisse Science and Technology(瑞士军备科学与技术局)

Zhivar Sourati, Filip Ilievski, Hông-Ân Sandlin, Alain Mermoud

AI总结:

本文提出一种基于案例推理(Case-Based Reasoning)的方法,通过语言模型检索和适应历史案例,并结合目标、解释、反驳和论证结构四种外部信息策略,有效提升了逻辑谬误分类的准确性与泛化能力。

摘要:

网络虚假信息和宣传的快速传播,凸显了开发可信技术以检测自然语言论证中逻辑谬误的需求。然而,当前最先进的语言建模方法在需要复杂推理的任务(如逻辑谬误分类)上表现出鲁棒性不足。本文提出一种 Case-Based Reasoning 方法,通过语言模型驱动的检索和对历史案例的适应,对新出现的逻辑谬误案例进行分类。我们设计了四种互补策略,利用关于目标、解释、反驳论点和论证结构的外部信息来丰富模型的输入表示。在领域内和领域外设置下的实验表明,Case-Based Reasoning 提升了语言模型的准确性和泛化能力。消融研究表明,相似案例的表示对模型性能有重大影响,模型在检索案例较少时表现良好,且案例库规模对性能的影响微乎其微。最后,我们深入探讨了检索案例属性与模型性能之间的关系。

英文摘要:

The ease and speed of spreading misinformation and propaganda on the Web motivate the need to develop trustworthy technology for detecting fallacies in natural language arguments. However, state-of-the-art language modeling methods exhibit a lack of robustness on tasks like logical fallacy classification that require complex reasoning. In this paper, we propose a Case-Based Reasoning method that classifies new cases of logical fallacy by language-modeling-driven retrieval and adaptation of historical cases. We design four complementary strategies to enrich input representation for our model, based on external information about goals, explanations, counterarguments, and argument structure. Our experiments in in-domain and out-of-domain settings indicate that Case-Based Reasoning improves the accuracy and generalizability of language models. Our ablation studies suggest that representations of similar cases have a strong impact on the model performance, that models perform well with fewer retrieved cases, and that the size of the case database has a negligible effect on the performance. Finally, we dive deeper into the relationship between the properties of the retrieved cases and the model performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑