大语言模型在文本之外是否安全：表情符号是否暴露了安全评估的漏洞
Are LLMs Safe Beyond Text: Do Emojis Expose Gaps in Safety Evaluation
浏览论文内容
中文总结 AI 辅助
本研究针对大语言模型安全评估多依赖文本提示的不足，测试表情符号增强提示对4款开源LLM的攻击效果，发现不同模型鲁棒性差异显著，证明输入形式会影响安全评估结果，提示现有评估存在漏洞。
中文摘要 AI 辅助
大语言模型（LLM）的安全评估主要依赖基于文本的对抗性提示，可能忽略了由替代输入表示形式引发的漏洞。本研究将表情符号增强的提示作为这一缺口的测试案例，在4个开源LLM（Mistral 7B、Qwen 2 7B、Gemma 2 9B、Llama 3 8B）上评估了50条提示。结果显示模型鲁棒性存在显著差异：Gemma 2 9B和Mistral 7B表现出非零成功率（10%），Llama 3 8B为6%，而Qwen 2 7B表现出完全抗性（成功率0%）。卡方检验（$χ^2 = 32.94, p < 0.001$）证实结果分布存在显著差异。这些发现表明，鲁棒性对输入表示形式敏感，仅局限于标准文本提示的评估可能低估模型漏洞。
英文摘要
Safety evaluations of large language models (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arising from alternative input representations. This work examines emoji-augmented prompts as a test case for this gap, evaluating 50 prompts across four open-source LLMs (Mistral 7B, Qwen 2 7B, Gemma 2 9B, Llama 3 8B). Results show substantial variation in robustness: Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%), Llama 3 8B 6%, while Qwen 2 7B shows complete resistance (0% success rate). A chi-square test ($χ^2 = 32.94, p < 0.001$) confirms significant differences in outcome distributions. These findings indicate that robustness is sensitive to input representation, and that evaluations restricted to standard text prompts may underrepresent model vulnerabilities.
