arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.12092cs.CLcs.AIcs.CRcs.CV
收藏

DeepSight: 一个全方位的大型模型安全工具包

DeepSight: An All-in-One LM Safety Toolkit

Shanghai AI Laboratory(上海人工智能实验室)
Bo Zhang, Jiaxuan Guo, Lijun Li, Dongrui Liu, Sujin Chen, Guanxu Chen, Zhijie Zheng, Qihao Lin, Lewen Yan, Chen Qian, Yijin Zhou, Yuyao Wu, Shaoxiong Guo, Tiany… 展开作者

Bo Zhang, Jiaxuan Guo, Lijun Li, Dongrui Liu, Sujin Chen, Guanxu Chen, Zhijie Zheng, Qihao Lin, Lewen Yan, Chen Qian, Yijin Zhou, Yuyao Wu, Shaoxiong Guo, Tianyi Du, Jingyi Yang, Xuhao Hu, Ziqi Miao, Xiaoya Lu, Jing Shao, Xia Hu

AI总结:

DeepSight是一个开源工具包,旨在通过整合安全评估与诊断,提升大型模型的安全性与可解释性。

摘要:

随着大型模型(LMs)的快速发展,其安全性也成为了优先考虑的问题。在当前的大型语言模型(LLMs)和多模态大型语言模型(MLLMs)安全工作流程中,评估、诊断和对齐通常由单独的工具处理。具体而言,安全评估只能定位外部行为风险,但无法找出内部根本原因。同时,安全诊断往往偏离具体的危险场景,停留在可解释的层面。在这种情况下,安全对齐缺乏对内部机制变化的专门解释,可能降低通用能力。为系统地解决这些问题,我们提出一个开源项目,即DeepSight,以实践一种新的安全评估-诊断一体化范式。DeepSight是一个低成本、可重复、高效且高度可扩展的大型模型安全评估项目,由评估工具包DeepSafe和诊断工具包DeepScan组成。通过统一任务和数据协议,我们建立了两个阶段之间的联系,将安全评估从黑箱转换为白箱洞察。此外,DeepSight是首个支持前沿AI风险评估和联合安全评估与诊断的开源工具包。

英文摘要:

As the development of Large Models (LMs) progresses rapidly, their safety is also a priority. In current Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) safety workflow, evaluation, diagnosis, and alignment are often handled by separate tools. Specifically, safety evaluation can only locate external behavioral risks but cannot figure out internal root causes. Meanwhile, safety diagnosis often drifts from concrete risk scenarios and remains at the explainable level. In this way, safety alignment lack dedicated explanations of changes in internal mechanisms, potentially degrading general capabilities. To systematically address these issues, we propose an open-source project, namely DeepSight, to practice a new safety evaluation-diagnosis integrated paradigm. DeepSight is low-cost, reproducible, efficient, and highly scalable large-scale model safety evaluation project consisting of a evaluation toolkit DeepSafe and a diagnosis toolkit DeepScan. By unifying task and data protocols, we build a connection between the two stages and transform safety evaluation from black-box to white-box insight. Besides, DeepSight is the first open source toolkit that support the frontier AI risk evaluation and joint safety evaluation and diagnosis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑