RegionFed：面向异构零售环境中个性化查询理解的联邦学习

中文摘要 AI 辅助

零售搜索系统服务于具有不同查询模式、词汇和产品偏好的不同地理区域，这产生了显著的数据异质性，对隐私保护训练和模型个性化都构成了挑战。联邦学习（FL）为隐私保护提供了自然的解决方案，但标准FL方法会生成全局模型，牺牲区域性能；而现有的个性化FL方法在参数层面操作，由于绑定嵌入和层归一化（LayerNorm）的相互作用，在现代Transformer模型上会出现灾难性崩溃（T5模型上准确率低于10%）。我们提出RegionFed，这是一种对架构鲁棒的联邦学习框架，完全在梯度层面操作以避免这种失败。RegionFed使用区域梯度与全局梯度之间的ℓ₂冲突作为统一信号，实现三个功能：（i）诊断异质性；（ii）为每个区域选择成本最低的足够个性化策略；（iii）自适应控制个性化强度。由于RegionFed将模型视为可微黑盒，因此可在T5-Small、T5-3B、RoBERTa和CNN上部署，无需修改代码，在Transformer模型（参数层面方法会崩溃）上取得大幅提升，在CNN上也有一致改进。在三个公开数据集（Amazon ESCI、Amazon Reviews、LEAF-FEMNIST）和四种架构上，RegionFed-Meta达到92.27%的准确率，缩小了与违反隐私的集中式上界（集中式+区域加权：92.04%，Δ=0.23个百分点，在1σ范围内）的差距，同时提供（ε≈0.60）-差分隐私和O(1/√T)的收敛速度。

英文摘要

Retail search systems serve diverse geographic regions with distinct query patterns, vocabularies, and product preferences, creating significant data heterogeneity that challenges both privacy-preserving training and model personalization. Federated learning offers a natural solution for privacy, but standard FL methods produce global models that sacrifice regional performance, while existing personalized FL approaches operate at the parameter level and catastrophically collapse on modern transformers (below 10\% accuracy on T5) due to tied embeddings and LayerNorm interactions. We introduce RegionFed, an \textit{architecture-robust} federated learning framework that sidesteps this failure by operating entirely at the gradient level. RegionFed uses the $\ell_2$ conflict between regional and global gradients as a unified signal that (i) diagnoses heterogeneity, (ii) routes each region to the cheapest sufficient personalization strategy, and (iii) adaptively controls personalization strength. Because it treats models as differentiable black boxes, RegionFed deploys on T5-Small, T5-3B, RoBERTa, and CNN with zero code changes, providing large gains on transformers (where parameter-level methods collapse) and consistent improvements on CNNs. Across three public datasets (Amazon ESCI, Amazon Reviews, LEAF-FEMNIST) and four architectures, RegionFed-Meta achieves 92.27\%, closing the gap to the privacy-violating centralized upper bound (Centralized + Regional Weighting: 92.04\%, $Δ$=0.23pp, within 1$σ$) while providing $(ε{\approx}0.60)$-differential privacy and $\mathcal{O}(1/\sqrt{T})$ convergence.

发表机构

• Walmart Global Tech（沃尔玛全球技术部门）

机构由 AI 辅助整理，请以论文原文为准。