EASE: 通过纠缠感知锚闭合实现联邦多模态去学习
EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure
AI总结:
本文提出EASE框架,通过闭合三个锚通道解决联邦多模态学习中的遗忘知识纠缠问题,采用余弦-正弦分解和方向选择性遗忘锁等方法,在多个数据集上表现优异。
摘要:
联邦多模态学习(FML)在去中心化客户端上训练多模态模型,同时保持图像-文本对的隐私。然而,联合嵌入训练使遗忘知识在两种模态和客户端梯度子空间中纠缠,阻碍了联邦去学习。本文提出联邦多模态对比去学习的锚原理:遗忘对齐通过三种残差锚持续存在,源自双线性跨模态耦合、主角 subspace 熔合和持续的联邦更新。在模态层面,我们显示双侧位移关闭了跨模态重建通道。相应地,我们的方法通过余弦-正弦分解客户端更新子空间解决子空间熔合,隔离遗忘方向与保留支持。此外,我们提出方向选择性遗忘锁,限制残差漂移。结合这些策略,我们提出EASE,一种纠缠感知子空间剔除框架,通过统一设计闭合所有三个锚通道。EASE在多个数据集和去学习场景中表现一致,例如在Flickr30K上使用CLIP-B/32时,匹配重训练参考在遗忘和保留侧分别达到0.2和4.2 R@1点。
英文摘要:
Federated Multimodal Learning (FML) trains multimodal models across decentralized clients while keeping their image-text pairs private. However, joint embedding training entangles forgotten knowledge across both modalities and client gradient subspaces, hindering federated unlearning. Previous federated unlearning approaches neither sever the cross-modal reconstruction channel mediated by bilinear coupling nor separate forget-exclusive update directions from those shared with retained clients. We identify an Anchor Principle for federated multimodal contrastive unlearning: forgotten alignments persist through three residual anchors arising from bilinear cross-modal coupling, principal-angle subspace entanglement, and continued federated updates. At the modality level, we show that bilateral displacement of both visual and language branches closes the cross-modal reconstruction channel. Correspondingly, our method addresses subspace entanglement through Cosine--Sine decomposition of client-update subspaces, isolating forget-exclusive directions from retain support. Moreover, we propose a direction-selective Forget Lock that bounds residual drift across rounds. Combining these strategies, we present EASE, an Entanglement-Aware Subspace Excision framework that closes all three anchor channels under a unified design. EASE demonstrates consistent superiority across multiple datasets and unlearning scenarios, for instance, matching the retrain reference to within 0.2 and 4.2 R@1 points on the forget and retain sides under client unlearning on Flickr30K with CLIP-B/32.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。