面向上下文的元认知修辞提示用于讽刺检测
Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detection
AI总结：
本文提出一种基于检索的元认知修辞提示方法，通过整合上下文信息提升LLMs在讽刺检测中的性能，实验显示在多个数据集上均取得显著提升。
摘要：
在自然语言处理（NLP）领域，尽管神经网络方法取得了进展，但讽刺检测仍是一个具有挑战性的任务。目前，预训练语言模型（PLMs）和大语言模型（LLMs）是讽刺检测的首选方法。然而，讽刺文本的复杂性，加上语言多样性和社区间的文化差异，使即使对于PLMs和LLMs来说，该任务也变得更加困难。此外，这些模型在需要额外支撑进行分析的词或标记的检测上也表现出不可靠性。基于LLMs中用于讽刺检测的最新提示方法——元认知修辞提示（PMP），我们介绍了一种检索感知的方法，该方法为每个目标文本纳入检索到的上下文信息。我们的流程探索了两种互补的方式来提供上下文：当模型缺乏必要的背景时，通过基于网络的检索添加非参数化知识，以及通过激发模型自身的内部知识来实施自我知识意识策略。我们使用了三个数据集进行评估，例如印尼推特讽刺数据集、SemEval-2018任务3和MUStARD。非参数化检索在印尼推特讽刺数据集上相比原始PMP方法显著提高了9.87%的宏F1值。自我知识检索在SemEval上提高了宏F1值3.29%，在MUStARD上提高了4.08%。这些发现突显了上下文在增强LLMs在讽刺检测任务中的性能方面的重要性，特别是涉及文化特定俚语、参考或对LLMs来说未知的术语。未来的工作将专注于优化相关上下文信息的检索，并研究检索质量如何影响性能。实验代码可在：https://github.com/wllchrst/sarcasm-detection_pmp_knowledge-base 上获得。
英文摘要：
Detecting sarcasm remains a challenging task in the areas of Natural Language Processing (NLP) despite recent advances in neural network approaches. Currently, Pre-trained Language Models (PLMs) and Large Language Models (LLMs) are the preferred approach for sarcasm detection. However, the complexity of sarcastic text, combined with linguistic diversity and cultural variation across communities, has made the task more difficult even for PLMs and LLMs. Beyond that, those models also exhibit unreliable detection of words or tokens that require extra grounding for analysis. Building on a state-of-the-art prompting method in LLMs for sarcasm detection called Pragmatic Metacognitive Prompting (PMP), we introduce a retrieval-aware approach that incorporates retrieved contextual information for each target text. Our pipeline explores two complementary ways to provide context: adding non-parametric knowledge using web-based retrieval when the model lacks necessary background, and eliciting the model's own internal knowledge for a self-knowledge awareness strategy. We evaluated our approach with three datasets, such as Twitter Indonesia Sarcastic, SemEval-2018 Task 3, and MUStARD. Non-parametric retrieval resulted in a significant 9.87% macro-F1 improvement on Twitter Indonesia Sarcastic compared to the original PMP method. Self-knowledge retrieval improves macro-F1 by 3.29% on Semeval and by 4.08% on MUStARD. These findings highlight the importance of context in enhancing LLMs performance in sarcasm detection task, particularly the involvement of culturally specific slang, references, or unknown terms to the LLMs. Future work will focus on optimizing the retrieval of relevant contextual information and examining how retrieval quality affects performance. The experiment code is available at: https://github.com/wllchrst/sarcasm-detection_pmp_knowledge-base.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
