arXiv 2106.033402021-06-07cs.LG

Instrument Space Selection for Kernel Maximum Moment Restriction

Australian National University（澳大利亚国立大学）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of Tokyo（东京大学）

Rui Zhang, Krikamol Muandet, Bernhard Schölkopf, Masaaki Imaizumi

英文摘要：

Kernel maximum moment restriction (KMMR) recently emerges as a popular framework for instrumental variable (IV) based conditional moment restriction (CMR) models with important applications in conditional moment (CM) testing and parameter estimation for IV regression and proximal causal learning. The effectiveness of this framework, however, depends critically on the choice of a reproducing kernel Hilbert space (RKHS) chosen as a space of instruments. In this work, we presents a systematic way to select the instrument space for parameter estimation based on a principle of the least identifiable instrument space (LIIS) that identifies model parameters with the least space complexity. Our selection criterion combines two distinct objectives to determine such an optimal space: (i) a test criterion to check identifiability; (ii) an information criterion based on the effective dimension of RKHSs as a complexity measure. We analyze the consistency of our method in determining the LIIS, and demonstrate its effectiveness for parameter estimation via simulations.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Instrumental Variable Regression via Kernel Maximum Moment Loss

Australian National University（澳大利亚国立大学）；The University of Tokyo（东京大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；CISPA–Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）

We investigate a simple objective for nonlinear instrumental variable (IV) regression based on a kernelized conditional moment restriction (CMR) known as a maximum moment restriction (MMR). The MMR objective is formulated by maximizing the interaction between the residual and the instruments belonging to a unit ball in a reproducing kernel Hilbert space (RKHS). First, it allows us to simplify the IV regression as an empirical risk minimization problem, where the risk functional depends on the reproducing kernel on the instrument and can be estimated by a U-statistic or V-statistic. Second, bas…

2023-02-09cs.LG

基于核的近端因果学习：两阶段估计与矩约束

Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction

University College London（伦敦大学学院）；University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）；ENSAE/CREST（法国国家统计与经济管理学院/经济与统计研究中心）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）

针对存在未观测混杂的因果效应估计问题，提出两种基于核的两阶段回归与最大矩约束方法，为 Fredholm 积分方程逆问题提供统一视角与一致性保证，在合成与真实数据上优于早期方法。

2023-03-28cs.LG

Kernel Conditional Moment Test via Maximum Moment Restriction

MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Google Research（谷歌研究院）

We propose a new family of specification tests called kernel conditional moment (KCM) tests. Our tests are built on a novel representation of conditional moment restrictions in a reproducing kernel Hilbert space (RKHS) called conditional moment embedding (CMME). After transforming the conditional moment restrictions into a continuum of unconditional counterparts, the test statistic is defined as the maximum moment restriction (MMR) within the unit ball of the RKHS. We show that the MMR not only fully characterizes the original conditional moment restrictions, leading to consistency in both hyp…

2020-06-19cs.LGecon.EMmath.ST

Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes

Queen Mary University of London（伦敦玛丽女王大学）；Doremir Music Research AB（Doremir音乐研究公司）

This paper proposes a deep convolutional neural network for performing note-level instrument assignment. Given a polyphonic multi-instrumental music signal along with its ground truth or predicted notes, the objective is to assign an instrumental source for each note. This problem is addressed as a pitch-informed classification task where each note is analysed individually. We also propose to utilise several kernel shapes in the convolutional layers in order to facilitate learning of efficient timbre-discriminative feature maps. Experiments on the MusicNet dataset using 7 instrument classes sh…

2021-07-28cs.IRcs.LGcs.NE

Kernel Methods for Unobserved Confounding: Negative Controls, Proxies, and Instruments

MIT Economics（麻省理工学院经济系）

Negative control is a strategy for learning the causal relationship between treatment and outcome in the presence of unmeasured confounding. The treatment effect can nonetheless be identified if two auxiliary variables are available: a negative control treatment (which has no effect on the actual outcome), and a negative control outcome (which is not affected by the actual treatment). These auxiliary variables can also be viewed as proxies for a traditional set of control variables, and they bear resemblance to instrumental variables. I propose a family of algorithms based on kernel ridge regr…

2023-03-23cs.LGecon.EMstat.ML

超越数据重加权的估计：核矩方法

Estimation Beyond Data Reweighting: Kernel Method of Moments

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Weierstrass Institute for Applied Analysis and Stochastics（魏尔斯特拉斯应用分析与随机学研究所）；Eidgenössische Technische Hochschule Zürich（苏黎世联邦理工学院）

本研究突破矩方法中φ-散度仅支持数据重加权的限制，提出基于最大平均差异的核矩方法KMM，其条件矩约束变体具渐近一阶最优性，在相关任务上性能优异。

2023-06-13cs.LGstat.ML

更多相关论文收起更多论文

Learning Tensors in Reproducing Kernel Hilbert Spaces with Multilinear Spectral Penalties

Katholieke Universiteit Leuven（鲁汶天主教大学）

We present a general framework to learn functions in tensor product reproducing kernel Hilbert spaces (TP-RKHSs). The methodology is based on a novel representer theorem suitable for existing as well as new spectral penalties for tensors. When the functions in the TP-RKHS are defined on the Cartesian product of finite discrete sets, in particular, our main problem formulation admits as a special case existing tensor completion problems. Other special cases include transfer learning with multimodal side information and multilinear multitask learning. For the latter case, our kernel-based view i…

2013-10-18cs.LG

Causal Inference with Treatment Measurement Error: A Nonparametric Instrumental Variable Approach

University College London（伦敦大学学院）；University of Oxford（牛津大学）；The Alan Turing Institute（艾伦·图灵研究所）

We propose a kernel-based nonparametric estimator for the causal effect when the cause is corrupted by error. We do so by generalizing estimation in the instrumental variable setting. Despite significant work on regression with measurement error, additionally handling unobserved confounding in the continuous setting is non-trivial: we have seen little prior work. As a by-product of our investigation, we clarify a connection between mean embeddings and characteristic functions, and how learning one simultaneously allows one to learn the other. This opens the way for kernel method research to le…

2022-06-18cs.LGstat.ME

基于学习的工具变量表示的因果效应估计

Towards causal effect estimation with learned instrument representations

本文提出ZNet模型，通过学习观测协变量的工具变量表示，实现无显式工具的因果估计，实验显示其能恢复真实工具或构建潜在工具。

2026-10-06cs.LGstat.MEstat.ML

利用再生核希尔伯特空间学习哈密顿动力学

Learning of Hamiltonian Dynamics with Reproducing Kernel Hilbert Spaces

The Norwegian University of Science and Technology (NTNU)（挪威科技大学）

本文提出一种基于再生核希尔伯特空间的正则化优化方法，利用辛核学习哈密顿动力学，并通过仿真验证了其能量守恒特性和在相空间大范围内的精度提升。

2024-10-24cs.LGcs.ROcs.SY

条件矩约束的泛函广义经验似然估计

Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictions

Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Weierstrass Institute for Applied Analysis and Stochastics（魏尔斯特拉斯应用分析与随机过程研究所）；Eidgenössische Technische Hochschule Zürich（苏黎世联邦理工学院）

针对条件矩约束估计的挑战，提出广义经验似然（GEL）的泛函重构方法，利用任意机器学习模型并基于对偶公式设计实用算法，在多个问题上取得最优性能。

2024-02-16cs.LGmath.STstat.ML