AI 中文总结
本研究针对联邦学习中记忆增强自编码器缺乏专用聚合技术的问题，提出新颖的引导式聚合方案，在非IID数据集上验证其对无监督异常检测的鲁棒性，可提升浅层自编码器性能并适配资源受限环境。
AI 中文摘要
注意力层是当前最强大、最具影响力模型的核心基础，拥有数百万乃至数十亿参数的模型依赖注意力层提供的上下文知识，其应用远不止于大型语言模型的核心组件，一个特别有趣的应用是记忆增强自编码器（MemAE），具体用于离群点检测任务中的无监督表示学习。已有研究表明，在集中式学习场景中，注意力有助于这些模型提升效果。本研究旨在解决联邦学习（FL）中MemAE模型缺乏专用聚合技术的问题。本文分析了记忆增强自编码器背后架构的复杂性，并提出了新颖的、有引导性的方法，以在联邦场景中有效聚合这些模型。我们在非独立同分布（non-IID）数据集上验证了所提方法，结果显示，这些新颖的聚合方案在处理不平衡数据集环境中的大量边缘节点时更具鲁棒性，尤其适用于无监督异常检测场景。该方法甚至能提升极浅层自编码器的性能，使其可用于资源受限的环境。
英文摘要
Attention layers are the backbone of today's most powerful and impactful models. Models with multi-million and billion parameters rely on contextual knowledge provided by attention layers. However, their use goes well beyond just being the core component of large language models. One particularly interesting application is in Memory Augmented Autoencoders (MemAE), specifically for unsupervised representation learning in outlier detection tasks. It was shown that attention helps these models be more effective in centralized learning scenarios. Our work aims to address the lack of specialized aggregation techniques in Federated Learning (FL) when it comes to MemAE models. In this paper we analyze the intricacies of the architecture behind Memory Augmented Autoencoders, and propose novel, guided approaches to effectively aggregate these models in federated scenarios. We demonstrate our approach on non-IID datasets and show that these novel aggregation schemes are more robust when dealing with numerous edge nodes in environments with unbalanced datasets, specifically for unsupervised anomaly detection scenarios. This approach improves the performance of even very shallow autoencoders, allowing them to be used in resource constrained environments.
CommentsSubmitted to the 4th IEEE International Conference on Federated Learning Technologies and Applications (FLTA 2026)
