线性推荐模型的正则化景观研究

中文摘要 AI 辅助

近年来，受深度学习技术启发的一系列推荐算法已成为多个标准推荐基准上的性能领先者。尽管这些算法基于不同的深度学习技术（如dropout、自编码器）构建，但它们表现出相似的性能，甚至具有相似的代价函数。本文研究这些模型的相当性能究竟是纯粹的巧合，还是可以在单一框架下统一。我们发现，所有线性性能领先者实际上仅添加了基于核范数的正则化器或基于Frobenius范数的正则化器。前者具有（令人惊讶的）刚性结构，限制了模型的预测能力，但其解是低秩的且具有闭式形式。后者更具表达力且对推荐更高效，但其解要么是满秩的，要么需要执行难以调优的数值过程，如ADMM。沿着这一发现线索，我们进一步提出了两种低秩、闭式解，这些解源自对基于Frobenius范数的正则化器的仔细推广。新解兼得核范数和Frobenius范数世界的最佳优势。

英文摘要

Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models' comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models' predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.

发表机构

• Kent State University（肯特州立大学）
• College of William and Mary（威廉与玛丽学院）
• iLambda（iLambda公司）

机构由 AI 辅助整理，请以论文原文为准。