arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.12596cs.LGcs.AI
收藏

KumoRFM-2:关系学习的规模化基础模型

KumoRFM-2: Scaling Foundation Models for Relational Learning

Valter Hudovernik, Federico López, Vid Kocijan, Akihiro Nitta, Jan Eric Lenssen, Jure Leskovec, Matthias Fey

AI总结:

KumoRFM-2是一种新型关系学习基础模型,支持上下文学习和微调,可处理多种预测任务。相比传统表格模型,其原生处理关系数据,无需手动展平表格或生成目标变量,且保持时间一致性。通过实验表明,KumoRFM-2在8%以内超越监督方法,且在冷启动和噪声数据下表现稳定。

摘要:

我们介绍了KumoRFM-2,即关系数据预训练基础模型的下一代版本。KumoRFM-2支持上下文学习以及微调,并适用于广泛预测任务。与传统表格基础模型不同,KumoRFM-2原生处理关系数据,可同时处理一个或多个连接表格,无需手动展平表格或生成目标变量,同时保持时间一致性。KumoRFM-2利用大量合成和现实数据在四个维度上预训练:个体表格层面的行和列维度,以及数据库层面的外键和跨样本维度。与前代相比,KumoRFM-2在任务信息注入方面更早,从而更精确地选择任务相关列并提高对噪声数据的鲁棒性。通过在41个具有挑战性的基准测试中的广泛实验和对表达性和敏感性的分析,我们证明KumoRFM-2在监督和基础方法上最高可提升8%,同时在极端冷启动和噪声数据设置下保持强劲性能。据我们所知,这是首次有少量样本基础模型在常见基准任务上超越监督方法,且微调后性能进一步提升。最后,尽管KumoRFM-1仅限于小规模内存数据集,KumoRFM-2可扩展到十亿级关系数据集。

英文摘要:

We introduce KumoRFM-2, the next iteration of a pre-trained foundation model for relational data. KumoRFM-2 supports in-context learning as well as fine-tuning and is applicable to a wide range of predictive tasks. In contrast to tabular foundation models, KumoRFM-2 natively operates on relational data, processing one or more connected tables simultaneously without manual table flattening or target variable generation, all while preserving temporal consistency. KumoRFM-2 leverages a large corpus of synthetic and real-world data to pre-train across four axes: the row and column dimensions at the individual table level, and the foreign key and cross-sample dimensions at the database level. In contrast to its predecessor, KumoRFM-2 injects task information as early as possible, enabling sharper selection of task-relevant columns and improved robustness to noisy data. Through extensive experiments on 41 challenging benchmarks and analysis around expressivity and sensitivity, we demonstrate that KumoRFM-2 outperforms supervised and foundational approaches by up to 8%, while maintaining strong performance under extreme settings of cold start and noisy data. To our knowledge, this is the first time a few-shot foundation model has been shown to surpass supervised approaches on common benchmark tasks, with performance further improving upon fine-tuning. Finally, while KumoRFM-1 was limited to small-scale in-memory datasets, KumoRFM-2 scales to billion-scale relational datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑