The analysis of scientific data and complex multivariate systems requires information quantities that capture relationships among multiple random variables. Recently, new information-theoretic measures have been developed to overcome the shortcomings of classical ones, such as mutual information, that are restricted to considering pairwise interactions. Among them, the concept of information synergy and redundancy is crucial for understanding the high-order dependencies between variables. One of the most prominent and versatile measures based on this concept is O-information, which provides a clear and scalable way to quantify the synergy-redundancy balance in multivariate systems. However, its practical application is limited to simplified cases. In this work, we introduce S$Ω$I, which allows for the first time to compute O-information without restrictive assumptions about the system. Our experiments validate our approach on synthetic data, and demonstrate the effectiveness of S$Ω$I in the context of a real-world use case.
DiScoFormer: Plug-In Density and Score Estimation with Transformers
Department of Mathematics, University of Washington, Seattle, USA;Math AI Lab, University of Washington, Seattle, USA;Allen Institute for Artificial Intelligence, Seattle, USA;Paul G.\ Allen School of Computer Science \& Engineering, University of Washington, Seattle, USA
Deconfounding Scores and Representation Learning for Causal Effect Estimation with Weak Overlap
University of Oxford(牛津大学);Google DeepMind(谷歌DeepMind);University of California, Santa Barbara(加州大学圣塔芭芭拉分校);Stanford University(斯坦福大学);Ellison Institute of Technology(埃里森技术研究所);University of California, Berkeley(加州大学伯克利分校)
School of Mathematics, South China University of Technology(华南理工大学数学学院);Tensor Learning Team, RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心张量学习团队);Department of Electrical Engineering, Columbia University(哥伦比亚大学电气工程系);School of Electronic and Information Engineering, South China University of Technology(华南理工大学电子与信息工程学院)
Score-of-Mixture Training: Training One-Step Generative Models Made Simple via Score Estimation of Mixture Distributions
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT), Cambridge, Massachusetts, USA(电气工程与计算机科学系,麻省理工学院(MIT),剑桥,马萨诸塞州,美国)
提出Score-of-Mixture Training (SMT)框架,通过最小化$α$-skew Jensen-Shannon散度并估计混合分布得分来训练单步生成模型,支持从零开始训练与预训练扩散模型蒸馏,实验证明其在CIFAR-10和ImageNet 64x64上表现优异。
Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization
Department of Management Science and Engineering, Stanford University(斯坦福大学管理科学与工程系);Daniel J. Epstein Department of Industrial and Systems Engineering, University of Southern California(南加州大学工业与系统工程系)
Latent Score-Based Bayesian Cramér-Rao Bound Estimation for High-Dimensional Imaging Systems
Oden Institute for Computational Engineering and Sciences, the University of Texas at Austin(德克萨斯大学奥斯汀分校奥登计算工程与科学研究所);Graduate School of Biomedical Sciences at UT Southwestern Medical Center(德克萨斯大学西南医学中心生物医学科学研究生院);Mallinckrodt Institute of Radiology at Washington University in St. Louis(圣路易斯华盛顿大学马林克罗特放射学研究所)
Calibration Strategies for Robust Causal Estimation: Theoretical and Empirical Insights on Propensity Score-Based Estimators
Economic AI, Nürnberger Str. 262 A, 93059 Regensburg, Bayern, Germany(经济人工智能,德国雷根施塔特);Chair of Statistics with Application in Business Analytics, University of Hamburg Business School, Moorweidenstr. 18, 20148 Hamburg, Hamburg, Germany(商务分析统计学系,汉堡大学商业学院);Chair of Mathematics and Statistics in Economics, University of Hamburg Business School, Von-Melle-Park 5, 20146 Hamburg, Hamburg, Germany(经济学数学与统计学系,汉堡大学商业学院);Düsseldorf Institute for Competition Economics, Heinrich Heine University Düsseldorf, Universitätstr. 1, 20225 Düsseldorf, Nordrhein-Westphalen, Germany(多特蒙德竞争经济学研究所,杜伊斯堡-艾森大学)
Propensity score estimation using classification and regression trees in the presence of missing covariate data
Leiden University Medical Center(莱顿大学医学中心)
数据挖掘和机器学习技术,如分类和回归树(CART)代表了一个有前途的替代传统的逻辑斯谛回归的倾向得分估计。尽管不完整的数据排除了对所有受试者进行逻辑斯谛回归的拟合,但CART之所以吸引人,部分原因是某些实现允许将不完整的记录并入树拟合中,并为所有受试者提供倾向得分估计。基于理论上的考虑,我们认为,自动处理缺失数据的CART可能是不合适的。使用一系列模拟实验,我们检查了不同方法处理缺失协变量数据的性能;(i)将CART算法直接应用于(部分)不完整的数据,(ii)完整的案例分析,(iii)多重插补。评估了评估确认-结局效应\add{among the exposed}的偏倚、标准误、均方误差和覆盖率。将CART算法直接应用于不完整的数据会导致偏倚,即使在数据完全随机完全随机缺失的情况下也是如此。总体而言,多重插补后加CART的结果最佳。我们的研究表明,在CART中自动处理缺失数据可能会导致严重偏倚,并且作为解释缺失数据的一种手段,其性能并不优于多重插补。
Laplace-Fisher Gate Identities for Optimal Matrix-Gated Blended Score Estimation
The Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学研究院);The University of Texas at Austin(德克萨斯大学奥斯汀分校);The Department of Aerospace Engineering & Engineering Mechanics(航空航天工程与机械系)
Evaluation of Bagging Predictors with Kernel Density Estimation and Bagging Score
Institute of Digital Engineering, Technical University of Applied Sciences W\" u rzburg-Schweinfurt Ignaz-Schön-Straße 11 Schweinfurt, Bavaria Germany 97421;Institute of Digital Engineering, Technical University of Applied Sciences W\" u rzburg-Schweinfurt
Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores
Faculty of Data and Decision Sciences, Technion --- Israel Institute of Technology(数据与决策科学学院,技术离子研究所);Faculty of Computer Science, Technion --- Israel Institute of Technology(计算机科学学院,技术离子研究所);Faculty of Biomedical Engineering, Technion --- Israel Institute of Technology(生物医学工程学院,技术离子研究所)