数据重建:基于样本分割的可识别性与优化
Data Reconstruction: Identifiability and Optimization with Sample Splitting
AI总结:
本文研究了数据重建的可识别性与优化问题,提出样本分割方法以提升重建性能。
摘要:
从KKT条件训练数据重建已显示出显著的实证成功,但仍然不清楚由此得到的KKT方程何时有唯一解,即使在可识别的范围内,如何通过优化可靠地恢复解仍然存疑。本文聚焦于这两个互补问题:可识别性和优化。在可识别性方面,我们讨论了具有多项式激活函数的两层网络的KKT系统的充分条件,以唯一确定训练数据,提供了理论解释何时以及为何重建是可能的。在优化方面,我们引入了样本分割,这是一种曲率感知的细化步骤,适用于通用的重建目标(不限于基于KKT的公式):它创造了额外的下降方向以摆脱较差的 stationary 点并细化解。实验表明,将几种现有的重建方法与样本分割结合,能够一致地提高重建性能。
英文摘要:
Training data reconstruction from KKT conditions has shown striking empirical success, yet it remains unclear when the resulting KKT equations have unique solutions and, even in identifiable regimes, how to reliably recover solutions by optimization. This work hereby focuses on these two complementary questions: identifiability and optimization. On the identifiability side, we discuss the sufficient conditions for KKT system of two-layer networks with polynomial activations to uniquely determine the training data, providing a theoretical explanation of when and why reconstruction is possible. On the optimization side, we introduce sample splitting, a curvature-aware refinement step applicable to general reconstruction objectives (not limited to KKT-based formulations): it creates additional descent directions to escape poor stationary points and refine solutions. Experiments demonstrate that augmenting several existing reconstruction methods with sample splitting consistently improves reconstruction performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。