具有正交基选择和对齐的数据协作分析
Data Collaboration Analysis with Orthonormal Basis Selection and Alignment
AI总结:
本文提出Orthonormal Data Collaboration (ODC),通过强制正交的私有和目标基,将对齐问题转化为经典的正交Procrustes问题,提升准确性和稳定性,实现高效的多方协作训练。
摘要:
数据协作(DC)通过共享仅有的线性投影来使多方联合训练模型。核心挑战是无需泄露各自秘密基的情况下对齐这些投影的基。尽管现有理论表明任何跨越公共子空间的目标基都足够,但实际中基的选择会显著影响准确性和数值稳定性。本文引入Orthonormal Data Collaboration (ODC),强制秘密和目标基正交,从而将对齐问题转化为经典的正交Procrustes问题,该问题具有闭式解。证明所得到的基变换矩阵实现正交一致性,使所有方的表示在共享正交变换下对齐,并使下游性能对目标基不敏感。计算上,ODC将对齐复杂度从O(min{a(cl)^2,a^2cl})降至O(acl^2),并通过实验证明在多个基准上可实现高达100倍的速度提升,同时保持相同或更好的准确性。ODC保留DC的一轮通信模式和隐私假设,为现有DC流程提供简单且高效的改进。
英文摘要:
Data Collaboration (DC) enables multiple parties to jointly train a model by sharing only linear projections of their private datasets. The core challenge in DC is to align the bases of these projections without revealing each party's secret basis. While existing theory suggests that any target basis spanning the common subspace should suffice, in practice, the choice of basis can substantially affect both accuracy and numerical stability. We introduce Orthonormal Data Collaboration (ODC), which enforces orthonormal secret and target bases, thereby reducing alignment to the classical Orthogonal Procrustes problem, which admits a closed-form solution. We prove that the resulting change-of-basis matrices achieve orthogonal concordance, aligning all parties' representations up to a shared orthogonal transform and rendering downstream performance invariant to the target basis. Computationally, ODC reduces the alignment complexity from O(min{a(cl)^2,a^2cl}) to O(acl^2), and empirical evaluations show up to 100 times speedups with equal or better accuracy across benchmarks. ODC preserves DC's one-round communication pattern and privacy assumptions, providing a simple and efficient drop-in improvement to existing DC pipelines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。