arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.14528cs.LGstat.ML
收藏

因子分解机中数值特征的函数基编码

Function Basis Encoding of Numerical Features in Factorization Machines

Yahoo(雅虎);University of Haifa(海法大学)

Alex Shtoff, Elie Abboud, Rotem Stram, Oren Somekh

AI总结:

本文提出一种将数值特征编码为自选函数基函数值向量的方法,以系统化且理论有据的方式改进因子分解机对数值特征的处理,并推荐使用B样条基,实验和在线A/B测试验证了其性能提升。

摘要:

因子分解机(FM)变体被广泛用于大规模实时内容推荐系统,因为它们在模型精度与训练和推理的低计算成本之间提供了出色的平衡。这些系统在同时包含数值列和类别列的表格数据上进行训练。引入数值列是一个挑战,通常通过标量变换或分箱来处理,这些方法可以是学习得到的,也可以是预先选定的。在这项工作中,我们提供了一种系统化且有理论依据的方法,通过将数值特征编码为一组自选函数的函数值向量,将数值特征纳入FM变体。我们将因子分解机视为分段函数的逼近器,即从某个字段的值到实数的函数,假设其余字段被赋予某些给定常数,我们将其称为段。从这个视角出发,我们表明我们的技术产生了一个模型,该模型学习由自选函数集所张成的数值特征的分段函数,即张成系数在不同段之间变化。因此,为了提高模型精度,我们提倡使用已知具有强大逼近能力的函数,并提供B样条基,因为它具有众所周知的逼近能力、在软件库中的可用性以及高效性。我们的技术保持了快速的训练和推理,并且只需要对FM模型的计算图进行很小的修改。因此,它很容易集成到现有系统中以提升其性能。最后,我们通过一组实验来支持我们的主张,包括合成实验、在多个数据集上的性能评估,以及在一个真实在线广告系统上的A/B测试,该测试显示了性能的提升。

英文摘要:

Factorization machine (FM) variants are widely used for large scale real-time content recommendation systems, since they offer an excellent balance between model accuracy and low computational costs for training and inference. These systems are trained on tabular data with both numerical and categorical columns. Incorporating numerical columns poses a challenge, and they are typically incorporated using a scalar transformation or binning, which can be either learned or chosen a-priori. In this work, we provide a systematic and theoretically-justified way to incorporate numerical features into FM variants by encoding them into a vector of function values for a set of functions of one's choice. We view factorization machines as approximators of segmentized functions, namely, functions from a field's value to the real numbers, assuming the remaining fields are assigned some given constants, which we refer to as the segment. From this perspective, we show that our technique yields a model that learns segmentized functions of the numerical feature spanned by the set of functions of one's choice, namely, the spanning coefficients vary between segments. Hence, to improve model accuracy we advocate the use of functions known to have strong approximation power, and offer the B-Spline basis due to its well-known approximation power, availability in software libraries, and efficiency. Our technique preserves fast training and inference, and requires only a small modification of the computational graph of an FM model. Therefore, it is easy to incorporate into an existing system to improve its performance. Finally, we back our claims with a set of experiments, including synthetic, performance evaluation on several data-sets, and an A/B test on a real online advertising system which shows improved performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑