EduGuardBench：一个用于评估LLM作为模拟教师的教学保真度与对抗安全性的整体基准
EduGuardBench: A Holistic Benchmark for Evaluating the Pedagogical Fidelity and Adversarial Safety of LLMs as Simulated Teachers
AI总结：
本文提出EduGuardBench基准，从角色扮演保真度和对抗安全性两方面评估LLM作为模拟教师的能力，发现模型性能两极分化、中等规模模型更脆弱，并揭示最安全模型能将有害请求转化为可教时刻的教育转化效应。
摘要：
用于模拟职业的大型语言模型（SP-LLMs），尤其是作为教师，对个性化教育至关重要。然而，确保其专业能力和伦理安全是一项关键挑战，因为现有基准无法衡量角色扮演保真度，也无法应对教育场景中固有的独特教学危害。为此，我们提出了EduGuardBench，一个双组件基准。它使用角色扮演保真度评分（RFS）评估专业保真度，同时诊断教学职业特有的危害。它还使用基于人设的对抗性提示探查安全漏洞，这些提示同时针对一般危害，尤其是学术不端行为，并使用包括攻击成功率（ASR）和三层拒绝质量评估在内的指标进行评估。我们对14个领先模型的大量实验揭示了性能的显著两极分化。虽然面向推理的模型通常表现出更高的保真度，但能力不足仍是大多数模型的主要失败模式。对抗性测试揭示了一个违反直觉的规模悖论：中等规模的模型可能最脆弱，这挑战了单调安全性的假设。关键的是，我们发现了一种强大的教育转化效应：最安全的模型擅长通过提供理想的教育性拒绝，将有害请求转化为可教时刻。这种能力与ASR呈强烈负相关，揭示了高级AI安全的新维度。因此，EduGuardBench提供了一个可复现的框架，超越了孤立的知识测试，转向对专业、伦理和教学对齐的整体评估，揭示了在教育中部署可信AI所必需且复杂的动态。材料见 https://github.com/YL1N/EduGuardBench。
英文摘要：
Large Language Models for Simulating Professions (SP-LLMs), particularly as teachers, are pivotal for personalized education. However, ensuring their professional competence and ethical safety is a critical challenge, as existing benchmarks fail to measure role-playing fidelity or address the unique teaching harms inherent in educational scenarios. To address this, we propose EduGuardBench, a dual-component benchmark. It assesses professional fidelity using a Role-playing Fidelity Score (RFS) while diagnosing harms specific to the teaching profession. It also probes safety vulnerabilities using persona-based adversarial prompts targeting both general harms and, particularly, academic misconduct, evaluated with metrics including Attack Success Rate (ASR) and a three-tier Refusal Quality assessment. Our extensive experiments on 14 leading models reveal a stark polarization in performance. While reasoning-oriented models generally show superior fidelity, incompetence remains the dominant failure mode across most models. The adversarial tests uncovered a counterintuitive scaling paradox, where mid-sized models can be the most vulnerable, challenging monotonic safety assumptions. Critically, we identified a powerful Educational Transformation Effect: the safest models excel at converting harmful requests into teachable moments by providing ideal Educational Refusals. This capacity is strongly negatively correlated with ASR, revealing a new dimension of advanced AI safety. EduGuardBench thus provides a reproducible framework that moves beyond siloed knowledge tests toward a holistic assessment of professional, ethical, and pedagogical alignment, uncovering complex dynamics essential for deploying trustworthy AI in education. See https://github.com/YL1N/EduGuardBench for Materials.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
