发表机构
Stable AI; Tsinghua University（Stable AI; 清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
LimiX-2采用上下文机制网络范式，通过联合建模和结构因果模型预训练，在多个基准上超越现有表格模型，并具备因果骨架恢复能力。
AI 中文摘要
我们介绍了LimiX-2，这是LimiX系列中的一个新模型，通过我们先前建立的缩放定律指导的模型和数据缩放而开发。LimiX-2采用上下文机制网络（CMNs）范式，并使用上下文条件掩码建模（CCMM）进行预训练。CMNs将上下文学习（in-context learning）的组织原则从以目标为中心的预测转变为以机制为导向的联合建模。它不是围绕传统表格PFN的$p(y \mid x, D_{\mathrm{context}})$目标来构建网络，而是设计为学习$p(x, y \mid D_{\mathrm{context}})$，即数据生成底层联合结构的上下文相关表示。预训练使用由结构因果模型（SCMs）生成的合成数据集，涵盖多样的图结构、功能机制和观测过程。在TabArena、TALENT和BCCO上的评估表明，LimiX-2优于当前特定于数据集的模型和表格基础模型。除了预测性能之外，CMN范式还促进了LimiX-2中的因果意识：其特征注意力编码了直接因果关系，从而能够准确恢复因果骨架。
英文摘要
We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the $p(y \mid x, D_{\mathrm{context}})$ objective of conventional tabular PFNs, it is designed around learning $p(x, y \mid D_{\mathrm{context}})$, a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, functional mechanisms, and observation processes. Evaluations on TabArena, TALENT, and BCCO show that LimiX-2 outperforms current dataset-specific models and tabular foundation models. Beyond predictive performance, the CMN paradigm also promotes causal awareness in LimiX-2: its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
