HalluDetect：法律领域对话系统中幻觉的检测、缓解与基准测试
HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain
AI总结：
本文提出 HalluDetect，一个用于法律领域消费者投诉聊天机器人的幻觉检测系统，并基准测试了五种缓解架构，发现 AgentBot 能在保持最高 token 准确率的同时最有效地降低每轮幻觉率。
摘要：
大型语言模型（LLMs）在工业界被广泛使用，但仍然容易出现幻觉，限制了其在关键应用中的可靠性。这项工作致力于减少基于 LLaMA 3.1 8B Instruct 构建的消费者投诉聊天机器人的幻觉，该模型是工业界常用的紧凑型模型。我们开发了 HalluDetect，这是一个基于 LLM 的幻觉检测系统，其 F1 分数达到 68.92%，比基线检测器高出 22.47%。在对五种幻觉缓解架构进行基准测试时，我们发现其中 AgentBot 将每轮幻觉降至 0.4159，同时保持了最高的 token 准确率（96.13%），使其成为最有效的缓解策略。我们的研究结果为幻觉缓解提供了一个可扩展的框架，证明优化的推理策略能显著提高事实准确性。
英文摘要：
Large Language Models (LLMs) are widely used in industry but remain prone to hallucinations, limiting their reliability in critical applications. This work addresses hallucination reduction in consumer grievance chatbots built using LLaMA 3.1 8B Instruct, a compact model frequently used in industry. We develop HalluDetect, an LLM-based hallucination detection system that achieves an F1 score of 68.92% outperforming baseline detectors by 22.47%. Benchmarking five hallucination mitigation architectures, we find that out of them, AgentBot minimizes hallucinations to 0.4159 per turn while maintaining the highest token accuracy (96.13%), making it the most effective mitigation strategy. Our findings provide a scalable framework for hallucination mitigation, demonstrating that optimized inference strategies can significantly improve factual accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
