SafeProtein：蛋白质基础模型的红队框架与基准
SafeProtein: Red-Teaming Framework and Benchmark for Protein Foundation Models
AI总结：
本文提出SafeProtein，首个针对蛋白质基础模型的红队框架与基准，结合多模态提示和束搜索实现高达70%的越狱成功率，揭示生物安全风险。
摘要：
蛋白质在几乎所有生物过程中都发挥着至关重要的作用。深度学习的进步极大地加速了蛋白质基础模型的发展，在蛋白质理解和设计方面取得了显著成功。然而，这些模型缺乏系统性的红队测试，引发了对其潜在滥用（例如生成具有生物安全风险的蛋白质）的严重担忧。本文介绍了SafeProtein，据我们所知，这是首个专为蛋白质基础模型设计的红队框架。SafeProtein结合了多模态提示工程和启发式束搜索，系统地设计红队方法并对蛋白质基础模型进行测试。我们还构建了SafeProtein-Bench，其中包括一个手动构建的红队基准数据集和一个全面的评估协议。SafeProtein在最先进的蛋白质基础模型上实现了持续越狱（对ESM3的攻击成功率高达70%），揭示了当前蛋白质基础模型中潜在的生物安全风险，并为开发面向前沿模型的稳健安全防护技术提供了见解。代码将在https://github.com/jigang-fan/SafeProtein公开发布。
英文摘要：
Proteins play crucial roles in almost all biological processes. The advancement of deep learning has greatly accelerated the development of protein foundation models, leading to significant successes in protein understanding and design. However, the lack of systematic red-teaming for these models has raised serious concerns about their potential misuse, such as generating proteins with biological safety risks. This paper introduces SafeProtein, the first red-teaming framework designed for protein foundation models to the best of our knowledge. SafeProtein combines multimodal prompt engineering and heuristic beam search to systematically design red-teaming methods and conduct tests on protein foundation models. We also curated SafeProtein-Bench, which includes a manually constructed red-teaming benchmark dataset and a comprehensive evaluation protocol. SafeProtein achieved continuous jailbreaks on state-of-the-art protein foundation models (up to 70% attack success rate for ESM3), revealing potential biological safety risks in current protein foundation models and providing insights for the development of robust security protection technologies for frontier models. The codes will be made publicly available at https://github.com/jigang-fan/SafeProtein.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
