发表机构
Imperial College London（伦敦帝国理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
eval-unlearn是一个开源Python库，为文本到图像扩散模型的概念遗忘提供统一可复现的基准测试框架，集成12种遗忘技术和9种评估指标，并发布排行榜，揭示准确性-质量权衡。
AI 中文摘要
针对文本到图像（T2I）扩散模型的概念遗忘技术数量不断增加，导致评估环境碎片化。方法在异构实验条件下进行评估，使得原则性的跨方法比较变得困难。我们提出eval-unlearn，一个开源Python库，为T2I扩散模型中的概念遗忘提供统一、可复现的基准测试框架。eval-unlearn集成了十二种已发表的遗忘技术，涵盖微调、闭式模型编辑和推理时干预，以及九种互补的评估指标，涵盖擦除效果、对抗鲁棒性、生成质量和概念保留。其插件架构允许第三方技术和指标自行注册，无需修改核心框架，其流式、批处理管道支持对标准NSFW概念和任意通用概念进行高效评估。作为进一步贡献，我们在HuggingFace上发布了一个公开排行榜，以及一个用于实时评估遗忘技术的交互式工具。该排行榜比较了所有十二种技术的裸体概念擦除案例研究，揭示了异构评估所掩盖的显著准确性-质量权衡。eval-unlearn在MIT许可证下发布；包、代码、排行榜和文档均可在该https URL获取。
英文摘要
The rising number of concept unlearning techniques for text-to-image (T2I) diffusion models has produced a fragmented evaluation landscape. Methods are assessed under heterogeneous experimental conditions making principled cross-method comparison difficult. We present eval-unlearn, an open-source Python library providing a unified, reproducible benchmarking framework for concept unlearning in T2I Diffusion models. eval-unlearn integrates twelve published unlearning techniques spanning fine-tuning, closed-form model editing, and inference-time intervention, alongside nine complementary evaluation metrics covering erasure efficacy, adversarial robustness, generative quality, and concept retention. Its plugin architecture lets third-party techniques and metrics self-register without modifying the core framework, and its streaming, batched pipeline supports efficient evaluation of both standard NSFW concepts and arbitrary general concepts. As a further contribution, we release a public leaderboard on HuggingFace along with an interactive tool for real-time evaluation of unlearning techniques. The leaderboard compares nudity concept erasure case study across all twelve techniques, exposing significant accuracy-quality trade-offs that are obscured by heterogeneous evaluation. eval-unlearn is released under the MIT license; the package, code, leaderboard, and documentation are all available at https://eval-unlearn.readthedocs.io.
