(DNN)²：深度神经网络的双重非负松弛
$(\text{DNN})^2$: Doubly Non-Negative Relaxations for Deep Neural Networks
浏览论文内容
中文总结 AI 辅助
该研究针对DNN验证的松弛间隙问题，提出特征值最大化程序，使(DNN)²方法的验证边界比标准SDP更紧且可认证，为安全关键自主系统部署神经网络模块提供支撑。
中文摘要 AI 辅助
针对修正线性单元（ReLU）神经网络（NN）验证的现有线性规划（LP）和半定规划（SDP）松弛方法，因存在显著的松弛间隙，会产生过于保守的安全保证。尽管完全正规划（CPP）公式能缩小该间隙，但其求解是NP难问题。成本最低且易处理的松弛方法——双重非负规划（DNN），保留了作为SDP的关键约束，但其规模超出了内点法在实际场景下的处理能力。虽然Burer-Monteiro（BM）分解已被应用于使基于SDP的验证具备可扩展性，但对于严格更紧的DNN公式，尚无此类结果。一个关键障碍是DNN中的额外非负约束导致最优性认证的对偶乘子不唯一，使得标准认证方法无法适用。我们提出一种新颖的特征值最大化程序，在不唯一的乘子空间中搜索有效认证，即全局最优性保证。实验表明，我们的方法(DNN)²产生的边界始终比标准SDP方法更紧，常能匹配精确解，且当存在有效认证时，我们的认证程序能确认全局最优性。这些结果是朝着提供紧、可认证且计算可扩展的验证保证迈出的关键一步，而此类保证是在安全关键型自主系统中部署神经网络控制器和感知模块所需的。
英文摘要
Existing linear program (LP) and semidefinite program (SDP) relaxations for rectified linear unit (ReLU) neural network (NN) verification yield overly-conservative safety guarantees due to significant relaxation gaps. While the completely positive program (CPP) formulation closes this gap, it is NP-hard to solve. Its cheapest tractable relaxation, the doubly non-negative program (DNN), retains critical constraints as an SDP, but one whose size exceeds the reach of interior-point methods at practical scale. While Burer-Monteiro (BM) factorization has been applied to make SDP-based verification scalable, no such result exists for the strictly tighter DNN formulation. A key obstacle is that additional non-negativity constraints in the DNN cause dual multipliers for optimality certification to be non-unique, making standard certification methods inapplicable. We propose a novel eigenvalue maximization procedure that searches the non-unique multiplier space for a valid certificate, i.e. a global optimality guarantee. Experiments demonstrate that our approach $(\text{DNN})^2$ produces bounds consistently tighter than the standard SDP method, often matching the exact solution, and that our certification procedure confirms global optimality when a valid certificate exists. These results are a key step toward providing tight, certifiable, and computationally scalable verification guarantees needed to deploy neural network controllers and perception modules in safety-critical autonomous systems.
发表机构
- Northeastern University（东北大学）
- University of Michigan（密歇根大学）
机构由 AI 辅助整理，请以论文原文为准。
