关系迁移学习中的任务稀缺性与标签泄漏
Task Scarcity and Label Leakage in Relational Transfer Learning
AI总结:
本文研究了关系迁移学习中任务稀缺性和标签泄漏问题,提出通过梯度投影方法抑制标签泄漏,提升模型在同构数据集上的迁移性能。
摘要:
训练关系基础模型需要学习跨任务转移的表示,但可用监督通常仅限于每个数据库少量预测目标。这种任务稀缺性导致学习的表示编码任务特定的快捷方式,即使在同一模式下也会降低迁移效果,我们称之为标签泄漏。我们使用K-Space,一种结合冻结预训练表格编码器和轻量级消息传递核心的模块化架构进行研究。为抑制泄漏,我们引入一种梯度投影方法,从表示更新中移除标签预测方向。在RelBench上,这平均提高了+0.145 AUROC的跨数据集迁移性能,通常恢复到单任务性能。我们的结果表明,有限的任务多样性,而非仅仅是有限的数据,限制了关系基础模型。
英文摘要:
Training relational foundation models requires learning representations that transfer across tasks, yet available supervision is typically limited to a small number of prediction targets per database. This task scarcity causes learned representations to encode task-specific shortcuts that degrade transfer even within the same schema, a problem we call label leakage. We study this using K-Space, a modular architecture combining frozen pretrained tabular encoders with a lightweight message-passing core. To suppress leakage, we introduce a gradient projection method that removes label-predictive directions from representation updates. On RelBench, this improves within-dataset transfer by +0.145 AUROC on average, often recovering near single-task performance. Our results suggest that limited task diversity, not just limited data, constrains relational foundation models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。