arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.00964cs.CLcs.LG

MALTO在SemEval-2024任务6:利用合成数据进行LLM幻觉检测

MALTO at SemEval-2024 Task 6: Leveraging Synthetic Data for LLM Hallucination Detection

Politecnico di Torino(托尼诺理工学院)

Federico Borra, Claudio Savelli, Giacomo Rosso, Alkis Koudounas, Flavio Giobergia

AI总结:

MALTO通过合成数据增强和投票集成方法,提升LLM幻觉检测的准确性与鲁棒性。

摘要:

在自然语言生成(NLG)中,现代大语言模型(LLMs)面临几个挑战,例如生成流畅但不准确的输出以及对以流畅性为中心的度量标准的依赖。这往往导致神经网络出现'幻觉'。SHROOM挑战专注于自动识别生成文本中的这些幻觉。为解决这些问题,我们引入了两个关键组件,一个包含LLM辅助伪标签和句子改写的数据增强管道,以及一个来自三个在自然语言推理(NLI)任务上预训练并在多样化数据集上微调的模型的投票集成。

英文摘要:

In Natural Language Generation (NLG), contemporary Large Language Models (LLMs) face several challenges, such as generating fluent yet inaccurate outputs and reliance on fluency-centric metrics. This often leads to neural networks exhibiting "hallucinations". The SHROOM challenge focuses on automatically identifying these hallucinations in the generated text. To tackle these issues, we introduce two key components, a data augmentation pipeline incorporating LLM-assisted pseudo-labelling and sentence rephrasing, and a voting ensemble from three models pre-trained on Natural Language Inference (NLI) tasks and fine-tuned on diverse datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑