Structure learning of Conditional Random Fields (CRFs) can be cast into an L1-regularized optimization problem. To avoid optimizing over a fully linked model, gain-based or gradient-based feature selection methods start from an empty model and incrementally add top ranked features to it. However, for high-dimensional problems like statistical relational learning, training time of these incremental methods can be dominated by the cost of evaluating the gain or gradient of a large collection of candidate features. In this study we propose a fast feature evaluation algorithm called Contrastive Feature Induction (CFI), which only evaluates a subset of features that involve both variables with high signals (deviation from mean) and variables with high errors (residue). We prove that the gradient of candidate features can be represented solely as a function of signals and errors, and that CFI is an efficient approximation of gradient-based evaluation methods. Experiments on synthetic and real data sets show competitive learning speed and accuracy of CFI on pairwise CRFs, compared to state-of-the-art structure learning methods such as full optimization over all features, and Grafting.
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Conditional Neural Processes~(CNPs) bridge neural networks with probabilistic inference to approximate functions of Stochastic Processes under meta-learning settings. Given a batch of non-{ \it i.i.d} function instantiations, CNPs are jointly optimized for in-instantiation observation prediction and cross-instantiation meta-representation adaptation within a generative reconstruction pipeline. There can be a challenge in tying together such two targets when the distribution of function observations scales to high-dimensional and noisy spaces. Instead, noise contrastive estimation might be able…
Efficiently Inducing Features of Conditional Random Fields
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
条件随机场（CRF）是无向图模型，其特殊情况对应于条件训练的有限状态机。这些模型的一个关键优势是它们具有很大的灵活性，可以包含大量重叠的、多粒度的、非独立的输入特征。面对这种自由，一个重要的问题是，应该使用什么功能？提出了一种CRF的特征提取方法。该方法建立在仅构建那些显著增加对数似然的特征合取的原则上，基于Della Pietra et al [1997]的方法，但改变为与条件概率而不是联合概率一起工作，并且具有额外的修改以提供专门针对序列模型的易处理性。与传统方法相比，自动化特征归纳提供了更高的准确性和特征数量的数量级减少;它允许使用更丰富，更高阶的马尔可夫模型，并提供更多的自由来自由猜测哪些原子特征可能与任务相关。归纳方法适用于线性链CRF，以及更任意的CRF结构，也称为关系马尔可夫网络[Taskar & Koller，2002]。我们提出了一个命名实体提取任务的实验结果。
A Temporally Disentangled Contrastive Diffusion Model for Spatiotemporal Imputation
School of Computer Science, University of Technology Sydney（悉尼科技大学计算机科学学院）；School of Computer Science and Technology, Zhejiang Normal University（浙江师范大学计算机科学与技术学院）；Department of Computer Science and Engineering, University of California, San Diego（加利福尼亚大学圣地亚哥分校计算机科学与工程学院）
What Can Language Models Inherit from Solomonoff Induction? Limits, Attainable Properties, and What Served Conditionals Reveal
Algocyte；Oxford Immune Algorithmics（牛津免疫算法学）；Oxford University Innovation（牛津大学创新中心）；London Institute for Healthcare Engineering（伦敦医疗保健工程研究所）；Algorithmic Dynamics Lab（算法动力学实验室）；School of Biomedical Engineering and Imaging Sciences（生物医学工程与影像科学学院）；King’s Institute for AI（国王人工智能研究所）；King’s College London（伦敦国王学院）
An Augmentation Overlap Theory of Contrastive Learning
State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（一般人工智能国家重点实验室，智能科学与技术学院，北京大学）；CSAIL, MIT（计算机科学与人工智能实验室，麻省理工学院）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
A Mathematical Perspective On Contrastive Learning
Stores Foundational AI, Amazon, Palo Alto CA 94301 and Pasadena CA 91125（亚马逊公司、帕洛阿尔托加州94301和帕萨迪纳加州91125）；Computing and Mathematical Sciences, California Institute of Technology, Pasadena CA 91125（计算与数学科学系，加州理工学院，帕萨迪纳加州91125）
Conditional Distribution Learning for Graph Classification
College of Computer Science（计算机科学学院）；Sichuan University（四川大学）；Department of Computer and Information Sciences（计算机与信息科学系）；Northumbria University（北安普顿大学）；School of Economics and Management（经济管理学院）；China University of Petroleum (Beijing)（中国石油大学（北京））；Law School（法学院）
