keepitsimple at SemEval-2025 Task 3: 基于LLM不确定性的多语言幻觉跨度检测方法
keepitsimple at SemEval-2025 Task 3: LLM-Uncertainty based Approach for Multilingual Hallucination Span Detection
AI总结：
针对多语言幻觉跨度检测任务，提出基于LLM不确定性的方法，利用采样响应的熵分歧识别幻觉，无需额外训练，成本低且适应性强。
摘要：
在黑盒语言模型生成的文本中识别幻觉跨度对于现实世界中的应用至关重要。最近在此方向上的尝试是SemEval-2025任务3，即Mu-SHROOM——关于幻觉及相关可观察过度生成错误的多语言共享任务。在本工作中，我们提出了针对此问题的解决方案，该方案利用随机采样响应的变异性来识别幻觉跨度。我们的假设是，如果语言模型对某个事实确定，其采样响应将是一致的，而幻觉事实将产生不同且冲突的结果。我们通过基于熵的分析来度量这种分歧，从而能够准确识别幻觉片段。我们的方法不依赖于额外训练，因此具有成本效益和适应性。此外，我们进行了广泛的超参数调优并进行了错误分析，为我们提供了对模型行为的关键见解。
英文摘要：
Identification of hallucination spans in black-box language model generated text is essential for applications in the real world. A recent attempt at this direction is SemEval-2025 Task 3, Mu-SHROOM-a Multilingual Shared Task on Hallucinations and Related Observable Over-generation Errors. In this work, we present our solution to this problem, which capitalizes on the variability of stochastically-sampled responses in order to identify hallucinated spans. Our hypothesis is that if a language model is certain of a fact, its sampled responses will be uniform, while hallucinated facts will yield different and conflicting results. We measure this divergence through entropy-based analysis, allowing for accurate identification of hallucinated segments. Our method is not dependent on additional training and hence is cost-effective and adaptable. In addition, we conduct extensive hyperparameter tuning and perform error analysis, giving us crucial insights into model behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
