发表机构
Meta Platforms, Inc.（Meta平台公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对推荐模型一个轮次现象，提出不确定性加权敏感性正则化（UWSR），通过惩罚消费者依赖不确定嵌入来改善泛化，在三个基准上显著降低交叉熵并提升AUC。
AI 中文摘要
具有稀疏嵌入和共享消费者的推荐模型常常表现出一个轮次现象：第二个轮次会降低训练损失，但同时显著削弱泛化能力。我们提出了一种基于前序原则违反的观点。在第一个轮次中，一个样本的标签并未影响用于对其评分的嵌入行。在后续轮次中，这些行包含了由先前标签更新引起的位移。这为共享消费者在后续轮次中利用这种位移创造了激励，而这种利用无法泛化。我们称这种不对称性为自影响不对称。利用一个精确的标量模型和局部影响分析，我们将这种不匹配与嵌入中的不确定性以及消费者在后续轮次中利用这种不确定性的激励联系起来。我们通过深度推荐模型中的嵌入-消费者更新干预验证了这一假设，并提出了不确定性加权敏感性正则化（UWSR），该方法通过增强损失函数来惩罚消费者依赖不确定嵌入的行为，从而抵消这种不匹配。与现有补救措施不同，UWSR保留了学习到的嵌入，并且在三个基准上，四个轮次的UWSR相对于一个轮次训练将测试交叉熵降低了1.38%-6.78%，并将AUC提高了0.0058-0.0231。
英文摘要
Recommendation models with sparse embeddings and a shared consumer often exhibit the one-epoch phenomenon: a second epoch lowers training loss while sharply degrading generalization. We present a view based on the violation of the prequential principle. On the first epoch, an example's label has not affected the embedding rows used to score it. On later epochs, those rows contain a displacement induced by the labels earlier update. This creates an incentive for the shared consumer to exploit this displacement in subsequent epochs, which fails to generalize. We call this self-influence asymmetry. Using an exact scalar model and local influence analysis, we connect this mismatch to the uncertainty in the embeddings and the consumers incentive to exploit it in subsequent epochs. We verify this hypothesis using an embedding-consumer-update interventions in deep recommendation models and propose uncertainty-weighted sensitivity regularization (UWSR) which counteracts this mismatch by augmenting the loss function to penalize the consumer for relying on uncertain embeddings. Unlike existing remedies, UWSR preserves the learned embeddings and across three benchmarks, four-epoch UWSR reduces test cross-entropy by 1.38%-6.78% and improves AUC by 0.0058-0.0231 relative to one-epoch training.
