Chinese SafetyQA：面向大型语言模型的安全性简短事实性基准
Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models
AI总结：
本文提出Chinese SafetyQA基准，用于评估大型语言模型回答中文安全简短问题的事实性能力，并全面分析现有模型的事实性表现及其与RAG能力和抗攻击鲁棒性的关系。
摘要：
随着大型语言模型（LLMs）的快速发展，显著的安全问题也随之出现。从根本上说，大型语言模型的安全性与其对安全知识理解的准确性、全面性和清晰度密切相关，特别是在法律、政策和伦理等领域。这种事实性能力对于决定这些模型能否在特定区域内安全合规地部署和应用至关重要。为了应对这些挑战并更好地评估LLMs回答简短问题的事实性能力，我们引入了Chinese SafetyQA基准。Chinese SafetyQA具有多项特性（即中文、多样性、高质量、静态、易于评估、安全相关、无害）。基于Chinese SafetyQA，我们对现有LLMs的事实性能力进行了全面评估，并分析了这些能力与LLM能力（例如RAG能力和抵御攻击的鲁棒性）之间的关系。
英文摘要：
With the rapid advancement of Large Language Models (LLMs), significant safety concerns have emerged. Fundamentally, the safety of large language models is closely linked to the accuracy, comprehensiveness, and clarity of their understanding of safety knowledge, particularly in domains such as law, policy and ethics. This factuality ability is crucial in determining whether these models can be deployed and applied safely and compliantly within specific regions. To address these challenges and better evaluate the factuality ability of LLMs to answer short questions, we introduce the Chinese SafetyQA benchmark. Chinese SafetyQA has several properties (i.e., Chinese, Diverse, High-quality, Static, Easy-to-evaluate, Safety-related, Harmless). Based on Chinese SafetyQA, we perform a comprehensive evaluation on the factuality abilities of existing LLMs and analyze how these capabilities relate to LLM abilities, e.g., RAG ability and robustness against attacks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
