DSC2025 -- ViHallu挑战：检测越南语大语言模型中的幻觉
DSC2025 -- ViHallu Challenge: Detecting Hallucination in Vietnamese LLMs
AI总结：
DSC2025 ViHallu挑战通过建立首个越南语LLM幻觉检测基准，探索多种检测方法，验证了结构化提示和集成策略在提升检测性能方面的有效性。
摘要：
大型语言模型（LLMs）在生产环境中的可靠性仍然显著受到其生成幻觉的限制——即流畅且听起来合理的输出，这些输出与事实相矛盾或伪造信息。尽管幻觉检测最近已成为英语主导的基准测试中的优先事项，但低至中等资源语言如越南语仍缺乏标准化的评估框架。本文介绍了DSC2025 ViHallu挑战，这是首个大规模共享任务，用于检测越南语LLM中的幻觉。我们提出了ViHallu数据集，包含10,000个标注的三元组（上下文、提示、响应）样本，系统性地划分为三个幻觉类别：无幻觉、内在和外在幻觉。该数据集包含三种提示类型——事实性、噪声和对抗性，以测试模型的鲁棒性。共有111个团队参与，最佳表现的系统在宏F1分数上达到84.80%，相比基于编码器的基线分数32.83%，表明经过指令训练的LLM在结构化提示和集成策略下显著优于通用架构。然而，与完美性能的差距表明，幻觉检测仍是一个具有挑战性的问题，特别是对于基于矛盾的内在幻觉。本工作建立了一个严谨的基准，并探索了多种检测方法，为未来研究越南语AI系统的可信度和可靠性提供了基础。
英文摘要：
The reliability of large language models (LLMs) in production environments remains significantly constrained by their propensity to generate hallucinations -- fluent, plausible-sounding outputs that contradict or fabricate information. While hallucination detection has recently emerged as a priority in English-centric benchmarks, low-to-medium resource languages such as Vietnamese remain inadequately covered by standardized evaluation frameworks. This paper introduces the DSC2025 ViHallu Challenge, the first large-scale shared task for detecting hallucinations in Vietnamese LLMs. We present the ViHallu dataset, comprising 10,000 annotated triplets of (context, prompt, response) samples systematically partitioned into three hallucination categories: no hallucination, intrinsic, and extrinsic hallucinations. The dataset incorporates three prompt types -- factual, noisy, and adversarial -- to stress-test model robustness. A total of 111 teams participated, with the best-performing system achieving a macro-F1 score of 84.80\%, compared to a baseline encoder-only score of 32.83\%, demonstrating that instruction-tuned LLMs with structured prompting and ensemble strategies substantially outperform generic architectures. However, the gap to perfect performance indicates that hallucination detection remains a challenging problem, particularly for intrinsic (contradiction-based) hallucinations. This work establishes a rigorous benchmark and explores a diverse range of detection methodologies, providing a foundation for future research into the trustworthiness and reliability of Vietnamese language AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
