arXiv 2103.126412021-03-23cs.LG

Pairwise Adjusted Mutual Information

Technische Universität München（慕尼黑工业大学）；Institut Polytechnique de Paris（巴黎综合理工学院）

Denys Lazarenko, Thomas Bonald

英文摘要：

A well-known metric for quantifying the similarity between two clusterings is the adjusted mutual information. Compared to mutual information, a corrective term based on random permutations of the labels is introduced, preventing two clusterings being similar by chance. Unfortunately, this adjustment makes the metric computationally expensive. In this paper, we propose a novel adjustment based on {pairwise} label permutations instead of full label permutations. Specifically, we consider permutations where only two samples, selected uniformly at random, exchange their labels. We show that the corresponding adjusted metric, which can be expressed explicitly, behaves similarly to the standard adjusted mutual information for assessing the quality of a clustering, while having a much lower time complexity. Both metrics are compared in terms of quality and performance on experiments based on synthetic and real data.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses

ÉTS Montreal（蒙特利尔高等技术学院）；CentraleSupelec-CNRS-Université Paris-Saclay（法国中央理工-高等电力学院-法国国家科学研究中心-巴黎萨克雷大学）

Recently, substantial research efforts in Deep Metric Learning (DML) focused on designing complex pairwise-distance losses, which require convoluted schemes to ease optimization, such as sample mining or pair weighting. The standard cross-entropy loss for classification has been largely overlooked in DML. On the surface, the cross-entropy may seem unrelated and irrelevant to metric learning as it does not explicitly involve pairwise distances. However, we provide a theoretical analysis that links the cross-entropy to several well-known and recent pairwise losses. Our connections are drawn from…

2021-11-26cs.CVcs.LGstat.ML

在遮蔽离散序列模型中神经估计成对互信息

Neural Estimation of Pairwise Mutual Information in Masked Discrete Sequence Models

University of California, Berkeley, CA, USA（加州大学伯克利分校）

本文提出了一种神经框架，直接从预训练的遮蔽扩散模型（MDMs）的隐藏状态中估计成对条件互信息（MI），利用模型自身条件分布计算的地面真实MI进行监督，从而捕捉模型内部对依赖结构的信念，并在单次前向传递中预测完整的MI矩阵，实现MI引导的并行解码。

2026-01-27cs.AIcs.ITcs.LG

基于互信息度量的异构数据无监督成对因果发现

Unsupervised Pairwise Causal Discovery on Heterogeneous Data using Mutual Information Measures

Alstom（阿尔斯通公司）

本文针对异构数据中的成对因果发现问题，提出一种基于稳健互信息度量的无监督方法，并提供了标准无偏基准结果，以替代依赖监督学习的现有基线，为未知环境下的因果发现提供参考。

2024-08-01cs.AIcs.LGstat.ME

FastAMI——聚类比较指标中随机性调整的蒙特卡罗方法

FastAMI -- a Monte Carlo Approach to the Adjustment for Chance in Clustering Comparison Metrics

针对大规模数据集聚类比较中随机性调整的计算难题，提出基于蒙特卡罗的 FastAMI 方法高效近似 AMI 并扩展至 SMI，在保证比成对方法更高准确率的同时实现了大型数据集的快速信息论比较。

2023-05-03cs.AIcs.LGstat.ML

大型二进制数据集的快速互信息计算

Fast Mutual Information Computation for Large Binary Datasets

BioISI - Biosystems and Integrative Sciences（生物系统与整合科学研究所）；Faculdade de Ciências - Universidade de Lisboa（里斯本大学科学学院）

针对大型二进制数据集计算互信息（MI）时成对方法计算成本过高的问题，提出基于矩阵的算法，通过向量化和优化矩阵运算加速所有变量对的MI计算，实验显示计算时间最多减少50,000倍，扩大了MI的应用范围。

2024-11-29cs.ITcs.LGcs.NA

重新定义线性非高斯无环模型的最短路径问题表述：成对似然比、先验知识与路径枚举

Redefining the Shortest Path Problem Formulation of the Linear Non-Gaussian Acyclic Model: Pairwise Likelihood Ratios, Prior Knowledge, and Path Enumeration

Nara Institute of Science and Technology（奈良科学技术研究所）；Ateneo de Manila University（雅典耀大学）

本文通过成对似然比替代kNN互信息、引入先验知识节点跳过策略及利用路径分布推断因果图属性，三重增强LiNGAM-SPP框架，提升其性能与实用性。

2025-02-27cs.LGstat.ME

更多相关论文收起更多论文

计算成本为线性的互信息估计及性能与HSIC相当的独立性检验

Linear cost mutual information estimation and independence test of similar performance as HSIC

Jagiellonian University（雅盖隆大学）；University of Warsaw（华沙大学）；Collegium Da Vinci in Poznan（波兹南 Collegium Da Vinci）

针对HSIC方法计算复杂度高、难以处理大数据样本的问题，提出线性成本的HCR方法，其依赖检测敏感度更高，还能构建联合分布模型、近似互信息，计算复杂度随维度按多项式增长。

2025-09-01cs.LG

Constrained Clustering: General Pairwise and Cardinality Constraints

University of Oxford（牛津大学）；King Khalid University（哈立德国王大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）

We study constrained clustering, where constraints guide the clustering process. In existing works, two categories of constraints have been widely explored, namely pairwise and cardinality constraints. Pairwise constraints enforce the cluster labels of two instances to be the same (must-link constraints) or different (cannot-link constraints). Cardinality constraints encourage cluster sizes to satisfy a user-specified distribution. Most existing constrained clustering models can only utilize one category of constraints at a time. We enforce the above two categories into a unified clustering mo…

2023-01-27cs.LGstat.ML

Hard Clusters Maximize Mutual Information

Technical University of Munich（慕尼黑工业大学）

In this paper, we investigate mutual information as a cost function for clustering, and show in which cases hard, i.e., deterministic, clusters are optimal. Using convexity properties of mutual information, we show that certain formulations of the information bottleneck problem are solved by hard clusters. Similarly, hard clusters are optimal for the information-theoretic co-clustering problem that deals with simultaneous clustering of two dependent data sets. If both data sets have to be clustered using the same cluster assignment, hard clusters are not optimal in general. We point at interes…

2016-08-17cs.IRcs.ITcs.LG

A3S：一种基于成对约束的通用主动聚类方法

A3S: A General Active Clustering Method with Pairwise Constraints

Alibaba Group（阿里巴巴集团）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）

针对传统半监督主动聚类在大类别数据集上查询成本高的问题，提出基于聚类调整的A3S框架，通过信息论指导的自适应聚合分裂调整初始聚类，在多数据集上以更少查询实现更优性能与可扩展性。

2024-07-14cs.AIcs.LG

非对比互信息的表示学习

Representation Learning via Non-Contrastive Mutual Information

Google DeepMind（谷歌DeepMind）

本文提出互信息非对比（MINC）损失，结合对比与非对比方法优势，在ImageNet上提升表示学习性能。

2025-04-23cs.AIcs.CVcs.LG

Conditional Mutual information-based Contrastive Loss for Financial Time Series Forecasting

KTH Royal Institute of Technology（瑞典皇家理工学院）；RISE Research Institutes of Sweden（瑞典RISE研究院）

We present a representation learning framework for financial time series forecasting. One challenge of using deep learning models for finance forecasting is the shortage of available training data when using small datasets. Direct trend classification using deep neural networks trained on small datasets is susceptible to the overfitting problem. In this paper, we propose to first learn compact representations from time series data, then use the learned representations to train a simpler model for predicting time series movements. We consider a class-conditioned latent variable model. We train …

2021-05-07cs.LGstat.ML

AdaptiveMDL-GenClust：一种融合归一化互信息与进化算法的鲁棒聚类框架

AdaptiveMDL-GenClust: A Robust Clustering Framework Integrating Normalized Mutual Information and Evolutionary Algorithms

Azad University（阿扎德大学）

针对单一聚类方法存在局限、难以适配多样数据集的问题，提出融合MDL原理与遗传算法的AdaptiveMDL-GenClust鲁棒聚类框架，经13个基准数据集验证，其准确率、稳定性等指标均优于传统方法。

2024-12-12cs.LGcs.NE

A PAC-Bayesian Analysis of Graph Clustering and Pairwise Clustering

Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）

We formulate weighted graph clustering as a prediction problem: given a subset of edge weights we analyze the ability of graph clustering to predict the remaining edge weights. This formulation enables practical and theoretical comparison of different approaches to graph clustering as well as comparison of graph clustering with other possible ways to model the graph. We adapt the PAC-Bayesian analysis of co-clustering (Seldin and Tishby, 2008; Seldin, 2009) to derive a PAC-Bayesian generalization bound for graph clustering. The bound shows that graph clustering should optimize a trade-off betw…

2010-09-02cs.DScs.LGstat.ML

估计时间序列与时间事件序列在不同分析任务中的互信息

Estimating Mutual Information between Time Series and Temporal Event Sequences Across Diverse Analysis Tasks

University of Minnesota - Twin Cities（明尼苏达大学-双城分校）；University of Pittsburgh（匹兹堡大学）；Inha University（Inha大学）

提出一种非参数互信息估计器，直接度量连续时间序列与离散事件序列之间的依赖关系，无需数据转换或离散化，通过处理量化伪影和事件冗余实现鲁棒统一框架。

2026-06-23cs.AIcs.ITcs.LG