AILS-NTUA在SemEval-2024任务6中的研究：面向幻觉检测与分析的高效模型调优
AILS-NTUA at SemEval-2024 Task 6: Efficient model tuning for hallucination detection and analysis
AI总结：
本研究针对SemEval-2024 Task-6的幻觉检测二分类任务，通过微调预训练模型与NLI模型并构建集成模型，在两类数据集上取得优于基线、接近竞赛最优的准确率表现。
摘要：
本文介绍了我们团队针对SemEval-2024 Task-6——SHROOM（幻觉及相关可观测过度生成错误共享任务）的参赛提交内容。参赛者需完成二分类任务，识别流畅型过度生成幻觉案例。我们的实验包括对一个预训练模型进行幻觉检测方向的微调，以及微调一个自然语言推理（NLI）模型。效果最佳的策略是构建这些模型的集成，其在模型无关和模型感知数据集上的准确率分别为77.8%和79.9%，优于主办方基线，且与竞赛中最优结果（对应准确率为84.7%和81.3%）相比表现突出。
英文摘要：
In this paper, we present our team's submissions for SemEval-2024 Task-6 - SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes. The participants were asked to perform binary classification to identify cases of fluent overgeneration hallucinations. Our experimentation included fine-tuning a pre-trained model on hallucination detection and a Natural Language Inference (NLI) model. The most successful strategy involved creating an ensemble of these models, resulting in accuracy rates of 77.8% and 79.9% on model-agnostic and model-aware datasets respectively, outperforming the organizers' baseline and achieving notable results when contrasted with the top-performing results in the competition, which reported accuracies of 84.7% and 81.3% correspondingly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
