PluRel: 合成数据解锁关系基础模型的扩展定律
PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models
AI总结：
PluRel通过合成多表关系数据库，实现了关系基础模型在扩展定律上的突破，展示了合成数据在提升模型泛化能力方面的潜力。
摘要：
关系基础模型（RFMs）通过从复杂的多表数据库中学习来促进数据驱动的决策。然而，训练此类模型所需的多样化关系数据库很少公开，这受到隐私限制的影响。虽然有方法可以生成任意大小的合成表格数据，但将模式结构和主外键连接整合到多表生成中仍然具有挑战性。在这里，我们介绍了PluRel，一种从头开始合成多表关系数据库的框架。通过逐步的方式，PluRel模型（1）用有向图建模模式，（2）用二分图建模表间主外键连接，以及（3）通过条件因果机制建模表中的特征分布。这些阶段的设计空间支持合成广泛多样的数据库，同时计算上轻量级。使用PluRel，我们首次观察到（1）RFM预训练损失随着合成数据库数量和预训练令牌总数的增加呈现幂律扩展，（2）增加合成数据库的数量提高了对真实数据库的泛化能力，（3）合成预训练生成了在真实数据库上继续预训练的强大基础模型。总体而言，我们的框架和结果将合成数据扩展定位为RFMs的一种有前景的范式。
英文摘要：
Relational Foundation Models (RFMs) facilitate data-driven decision-making by learning from complex multi-table databases. However, the diverse relational databases needed to train such models are rarely public due to privacy constraints. While there are methods to generate synthetic tabular data of arbitrary size, incorporating schema structure and primary-foreign key connectivity for multi-table generation remains challenging. Here we introduce PLUREL, a framework to synthesize multi-tabular relational databases from scratch. In a step-by-step fashion, PLUREL models (1) schemas with directed graphs, (2) inter-table primary-foreign key connectivity with bipartite graphs, and, (3) feature distributions in tables via conditional causal mechanisms. The design space across these stages supports the synthesis of a wide range of diverse databases, while being computationally lightweight. Using PLUREL, we observe for the first time that (1) RFM pretraining loss exhibits power-law scaling with the number of synthetic databases and total pretraining tokens, (2) scaling the number of synthetic databases improves generalization to real databases, and (3) synthetic pretraining yields strong base models for continued pretraining on real databases. Overall, our framework and results position synthetic data scaling as a promising paradigm for RFMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
