面向资源受限NLP系统的少样本优化幻觉检测框架
Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems
AI总结：
本文提出一种面向资源受限NLP系统的幻觉检测框架，通过DeepSeek Few-shot优化与迭代提示工程生成高质量弱标签，并在其上微调Mistral-7B-Instruct-v0.3模型结合集成学习，在SHROOM任务测试集上达到85.5%准确率。
摘要：
文本生成中的幻觉检测仍然是自然语言处理（NLP）系统持续面临的难题，经常导致机器翻译和定义建模等应用中出现不可靠的输出。现有方法在数据稀缺和无标签数据集的局限性方面存在困难，正如SemEval-2024的SHROOM共享任务所强调的那样。在本研究中，我们提出了一个新颖的框架来应对这些挑战，引入DeepSeek Few-shot优化，通过迭代提示工程增强弱标签生成。我们获得了高质量的标注，通过重构数据以与指令生成模型对齐，显著提升了下游模型的性能。我们进一步在这些优化标注上微调了Mistral-7B-Instruct-v0.3模型，使其能够在资源受限的环境中准确检测幻觉。将这一微调模型与集成学习策略相结合，我们的方法在测试集上达到了85.5%的准确率，为SHROOM任务设定了新的基准。本研究证明了数据重构、少样本优化和微调在构建面向资源受限NLP系统的可扩展且鲁棒的幻觉检测框架方面的有效性。
英文摘要：
Hallucination detection in text generation remains an ongoing struggle for natural language processing (NLP) systems, frequently resulting in unreliable outputs in applications such as machine translation and definition modeling. Existing methods struggle with data scarcity and the limitations of unlabeled datasets, as highlighted by the SHROOM shared task at SemEval-2024. In this work, we propose a novel framework to address these challenges, introducing DeepSeek Few-shot optimization to enhance weak label generation through iterative prompt engineering. We achieved high-quality annotations that considerably enhanced the performance of downstream models by restructuring data to align with instruct generative models. We further fine-tuned the Mistral-7B-Instruct-v0.3 model on these optimized annotations, enabling it to accurately detect hallucinations in resource-limited settings. Combining this fine-tuned model with ensemble learning strategies, our approach achieved 85.5% accuracy on the test set, setting a new benchmark for the SHROOM task. This study demonstrates the effectiveness of data restructuring, few-shot optimization, and fine-tuning in building scalable and robust hallucination detection frameworks for resource-constrained NLP systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
