arXiv 2407.091522024-07-12cs.AIcs.CL

硬币的两面：以大语言模型作为大语言模型评估器的幻觉生成与检测

The Two Sides of the Coin: Hallucination Generation and Detection with LLMs as Evaluators for LLMs

TH Köln – University of Applied Sciences（科隆应用技术大学）

Anh Thu Maria Bui, Saskia Felizitas Brech, Natalie Hußfeldt, Tobias Jennert, Melanie Ullrich, Timo Breuer, Narjes Nikzad Khasmakhi, Philipp Schaer

备注：: Paper accepted at ELOQUENT@CLEF'24

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究参与CLEF ELOQUENT HalluciGen共享任务，探究Llama 3等四种大语言模型在幻觉生成与检测中的能力，还采用集成多数投票结合四模型做检测，为相关模型的优缺点提供了宝贵见解。

摘要：

大语言模型（LLM）中的幻觉检测对保障其可靠性至关重要。本文介绍了我们参与CLEF ELOQUENT HalluciGen共享任务的工作，该任务的目标是开发可同时生成和检测幻觉内容的评估器。我们探究了四种LLM在此任务中的能力：Llama 3、Gemma、GPT-3.5 Turbo和GPT-4。我们还采用集成多数投票法，将这四个模型结合用于检测任务。实验结果为了解这些LLM在处理幻觉生成与检测任务时的优缺点提供了宝贵见解。

英文摘要：

Hallucination detection in Large Language Models (LLMs) is crucial for ensuring their reliability. This work presents our participation in the CLEF ELOQUENT HalluciGen shared task, where the goal is to develop evaluators for both generating and detecting hallucinated content. We explored the capabilities of four LLMs: Llama 3, Gemma, GPT-3.5 Turbo, and GPT-4, for this purpose. We also employed ensemble majority voting to incorporate all four models for the detection task. The results provide valuable insights into the strengths and weaknesses of these LLMs in handling hallucination generation and detection tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。