Factorization machine (FM) variants are widely used for large scale real-time content recommendation systems, since they offer an excellent balance between model accuracy and low computational costs for training and inference. These systems are trained on tabular data with both numerical and categorical columns. Incorporating numerical columns poses a challenge, and they are typically incorporated using a scalar transformation or binning, which can be either learned or chosen a-priori. In this work, we provide a systematic and theoretically-justified way to incorporate numerical features into FM variants by encoding them into a vector of function values for a set of functions of one's choice.
We view factorization machines as approximators of segmentized functions, namely, functions from a field's value to the real numbers, assuming the remaining fields are assigned some given constants, which we refer to as the segment. From this perspective, we show that our technique yields a model that learns segmentized functions of the numerical feature spanned by the set of functions of one's choice, namely, the spanning coefficients vary between segments. Hence, to improve model accuracy we advocate the use of functions known to have strong approximation power, and offer the B-Spline basis due to its well-known approximation power, availability in software libraries, and efficiency. Our technique preserves fast training and inference, and requires only a small modification of the computational graph of an FM model. Therefore, it is easy to incorporate into an existing system to improve its performance. Finally, we back our claims with a set of experiments, including synthetic, performance evaluation on several data-sets, and an A/B test on a real online advertising system which shows improved performance.
Stage-dependent integer-binary encoding in factorization-machine black-box optimization
Graduate School of Science and Technology, Keio University(庆应义塾大学理工学研究科);Keio University Sustainable Quantum Artificial Intelligence Center (KSQAIC), Keio University(庆应义塾大学可持续量子人工智能中心);Department of Applied Physics and Physico-Informatics, Keio University(庆应义塾大学应用物理与物理信息学系);Human Biology-Microbiome-Quantum Research Center (WPI-Bio2Q), Keio University(庆应义塾大学人类生物学-微生物组-量子研究中心);Green Computing System Research Organization, Waseda University(早稻田大学绿色计算系统研究机构)
Decoding the Encoding of Functional Brain Networks: an fMRI Classification Comparison of Non-negative Matrix Factorization (NMF), Independent Component Analysis (ICA), and Sparse Coding Algorithms
Black-box optimization for integer-variable problems using Ising machines and factorization machines
Keio University(庆应义塾大学);National Institute for Materials Science(国立材料科学研究所);The University of Tokyo(东京大学);Waseda University(早稻田大学);Tokyo Institute of Technology(东京工业大学)
Factorization Machine with Quadratic-Optimization Annealing for RNA Inverse Folding and Evaluation of Binary-Integer Encoding and Nucleotide Assignment
Graduate School of Science and Technology, Keio University(Keio大学科学与技术研究生院);Keio University Sustainable Quantum Artificial Intelligence Center (KSQAIC)(Keio大学可持续量子人工智能中心);Department of Applied Physics and Physico-Informatics, Keio University(Keio大学应用物理与物理信息学系);Human Biology-Microbiome-Quantum Research Center (WPI-Bio2Q)(人类生物学-微生物组-量子研究中心)
Online Learning for Matrix Factorization and Sparse Coding
INRIA(法国国家信息与自动化研究所);Laboratoire d’Informatique de l’Ecole Normale Supérieure(巴黎高等师范学院信息学实验室);Ecole Normale Supérieure(巴黎高等师范学院);University of Minnesota(明尼苏达大学)
Online Convex Matrix Factorization with Representative Regions
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
矩阵分解(MF)是一种通用的学习方法,在各种数据驱动的学科中得到了广泛的应用。尽管如此,许多MF算法不能充分地随可用数据集的大小扩展和/或缺乏可解释性。为了提高该方法的计算效率,Mairal等人[2010]提出了一种在线(流式)MF算法。为了实现数据的可解释性,Ding等人[2010]引入了MF的约束版本,称为凸MF。在后一种工作中,要求基向量位于数据样本的凸包中,从而确保每个基可以被解释为数据样本的加权组合。目前没有已知的在线凸MF的算法解决方案,因为在不访问完整数据集的情况下找到足够的凸基是具有挑战性的。我们通过提出第一个在线凸MF算法来解决这两个问题,该算法维护解释每个基所需的代表性数据样本的恒定大小集合(Ding et al. [2010]),并且具有与Mairal et al. [2010]的在线学习算法相同的几乎肯定收敛保证。我们的证明技术结合随机坐标下降算法与专门的准鞅收敛分析。合成和真实世界的数据集上的实验表明显着的计算节省所提出的在线凸MF方法相比,经典的凸MF。由于所提出的方法保留了凸解释所需的小的代表性数据样本集,因此它与理论计算机科学中的一系列工作有关,涉及生成点集(Blum et al. [2016]),以及计算机视觉,涉及原型分析(Mei et al. [2018])。然而,它在目标和算法实现方面都不同于这些工作。
Cycle Invariant Positional Encoding for Graph Representation Learning
Wangxuan Institute of Computer Technology(王选计算机研究所);Peking University(北京大学);Stony Brook University(石溪大学);Halıcıoğlu Data Science Institute(哈利哲奥卢数据科学研究所);University of California(加利福尼亚大学)
Predictive Clinical Decision Support System with RNN Encoding and Tensor Decoding
Ludwig-Maximilians-Universität München(慕尼黑大学);Siemens AG(西门子股份公司);University Hospital Erlangen(埃尔朗根大学医院);Comprehensive Cancer Center Erlangen-EMN(埃尔朗根-欧洲大都市纽伦堡综合癌症中心);Friedrich-Alexander University Erlangen-Nuremberg(纽伦堡弗里德里希-亚历山大大学);University Hospital Heidelberg(海德堡大学医院);University Hospital Düsseldorf(杜塞尔多夫大学医院);Heinrich-Heine University Düsseldorf(杜塞尔多夫海因里希·海涅大学);University Hospital Tübingen(图宾根大学医院)