Luna：一种以高准确率和低成本捕捉语言模型幻觉的评估基础模型
Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost
AI总结：
提出Luna，一个基于DeBERTa-large微调的轻量级幻觉检测模型，在RAG系统中以低成本和低延迟优于GPT-3.5及商业框架，适用于工业级LLM应用。
摘要：
检索增强生成（RAG）系统通过引入外部知识检索机制，在增强语言模型能力方面变得至关重要。然而，在工业应用中部署这些系统的一个重大挑战是幻觉的检测与缓解：即模型生成未基于检索上下文的信息的情况。解决这个问题对于确保大型语言模型（LLMs）在各种工业场景中生成响应的可靠性和准确性至关重要。当前的幻觉检测技术无法同时实现高准确率、低延迟和低成本。我们提出了Luna：一个在RAG环境下针对幻觉检测进行微调的DeBERTA-large（440M）编码器。我们证明Luna在幻觉检测任务上优于GPT-3.5和商业评估框架，成本和延迟分别降低了97%和91%。Luna轻量级且能泛化到多个行业垂直领域和域外数据，使其成为工业LLM应用的理想选择。
英文摘要：
Retriever Augmented Generation (RAG) systems have become pivotal in enhancing the capabilities of language models by incorporating external knowledge retrieval mechanisms. However, a significant challenge in deploying these systems in industry applications is the detection and mitigation of hallucinations: instances where the model generates information that is not grounded in the retrieved context. Addressing this issue is crucial for ensuring the reliability and accuracy of responses generated by large language models (LLMs) in diverse industry settings. Current hallucination detection techniques fail to deliver accuracy, low latency, and low cost simultaneously. We introduce Luna: a DeBERTA-large (440M) encoder, finetuned for hallucination detection in RAG settings. We demonstrate that Luna outperforms GPT-3.5 and commercial evaluation frameworks on the hallucination detection task, with 97% and 91% reduction in cost and latency, respectively. Luna is lightweight and generalizes across multiple industry verticals and out-of-domain data, making it an ideal candidate for industry LLM applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
