SHROOM-INDElab 参加 SemEval-2024 任务 6：用于幻觉检测的基于 LLM 的零样本与少样本分类
SHROOM-INDElab at SemEval-2024 Task 6: Zero- and Few-Shot LLM-Based Classification for Hallucination Detection
AI总结：
本文介绍 SHROOM-INDElab 系统，通过上下文特定提示定义与自动生成示例扩展 LLM 幻觉检测分类器，在 SemEval-2024 Task 6 两个赛道分获第四、第六名，并发现零样本优于自动少样本。
摘要：
我们描述了阿姆斯特丹大学智能数据工程实验室团队参加 SemEval-2024 Task 6 竞赛的参赛作品。SHROOM-INDElab 系统建立在此前利用提示编程和大语言模型（LLM）上下文学习来构建幻觉检测分类器的工作基础上，并通过纳入针对具体上下文的任务、角色和目标概念定义，以及自动生成用于少样本提示方法的示例来扩展该工作。所得系统在 Task 6 的模型无关赛道和模型感知赛道中分别取得了第四佳和第六佳的性能，并且使用验证集进行的评估表明，该系统的分类决策与众包人工标注者的分类决策一致。我们进一步发现，零样本方法比使用自动生成示例的少样本方法具有更高的准确率。本文所述系统的代码可在 Github 上获取。
英文摘要：
We describe the University of Amsterdam Intelligent Data Engineering Lab team's entry for the SemEval-2024 Task 6 competition. The SHROOM-INDElab system builds on previous work on using prompt programming and in-context learning with large language models (LLMs) to build classifiers for hallucination detection, and extends that work through the incorporation of context-specific definition of task, role, and target concept, and automated generation of examples for use in a few-shot prompting approach. The resulting system achieved fourth-best and sixth-best performance in the model-agnostic track and model-aware tracks for Task 6, respectively, and evaluation using the validation sets showed that the system's classification decisions were consistent with those of the crowd-sourced human labellers. We further found that a zero-shot approach provided better accuracy than a few-shot approach using automatically generated examples. Code for the system described in this paper is available on Github.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
