Justin Domke
Maximum likelihood learning with exponential families leads to moment-matching of the sufficient statistics, a classic result. This can be generalized to conditional exponential families and/or when there are hidden data. This document gives a first-principles explanation of these generalized moment-matching conditions, along with a self-contained derivation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
Positional task conditioning for scalable defect detection across product families in large product catalogs
针对大型产品目录中产品族的不一致问题，本文提出将缺陷检测分解为子任务并引入位置任务条件化（PTC）蒸馏方法，将F1从52%提升至87%，成本降低98%，已部署处理超千万产品族。
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks with several families of local distributions, are curved exponential families (CEFs) and graphical models with hidden variables are stratified exponential families (SEFs). An SEF is a finite union of CEFs satisfying a frontier condition. In addition, we illustrate how one can aut…
Spectral Stability of Pseudoinverse-Based Extreme Learning Machine
研究基于伪逆的极限学习机的谱稳定性，通过分析最小奇异值和条件数来探讨其稳定性，比较基于奇异值分解和迭代超幂方法，实验表明病态条件下基于奇异值分解的方法更可靠，揭示ELM稳定性由隐藏层矩阵奇异值结构决定。
Exact, Fast and Expressive Poisson Point Processes via Squared Neural Families
提出平方神经泊松点过程（SNEPPP），用两层神经网络平方范数参数化强度函数，具有闭式解且时空效率高，通过凸优化求解参数，在真实与合成基准上表现优异。
We describe \textit{deep exponential families} (DEFs), a class of latent variable models that are inspired by the hidden structures used in deep neural networks. DEFs capture a hierarchy of dependencies between latent variables, and are easily generalized to many settings through exponential families. We perform inference using recent "black box" variational inference techniques. We then evaluate various DEFs on text and combine multiple DEFs into a model for pairwise recommendation data. In an extensive study, we show that going beyond one layer improves predictions for DEFs. We demonstrate t…
System-Prompt Conditioning and Hidden-State Geometry in Four Open-Weight Models: Corrections and What Survives
研究四种训练后模式的Transformer语言模型，通过五个几何指标比较三种提示条件，发现多模态指令微调会使身份编码从方向重组为大小，此重组特定于该模式，还定位了W_1作为方法贡献。
The information bottleneck principle (Shwartz-Ziv & Tishby, 2017) suggests that SGD-based training of deep neural networks results in optimally compressed hidden layers, from an information theoretic perspective. However, this claim was established on toy data. The goal of the work we present here is to test whether the information bottleneck principle is applicable to a realistic setting using a larger and deeper convolutional architecture, a ResNet model. We trained PixelCNN++ models as inverse representation decoders to measure the mutual information between hidden layers of a ResNet and in…
With the increasing adoption of Deep Neural Network (DNN) models as integral parts of software systems, efficient operational testing of DNNs is much in demand to ensure these models' actual performance in field conditions. A challenge is that the testing often needs to produce precise results with a very limited budget for labeling data collected in field. Viewing software testing as a practice of reliability estimation through statistical sampling, we re-interpret the idea behind conventional structural coverages as conditioning for variance reduction. With this insight we propose an efficie…
Learning Hidden Physics and System Parameters with Deep Operator Networks
本文提出基于深度运算网络的两种框架，用于发现隐藏的物理定律和识别系统参数，通过在稀疏观测中发现未知PDE项和参数估计，实现高精度的物理发现和参数识别。
Distributional Clarity: The Hidden Driver of RL-Friendliness in Large Language Models
研究揭示了大型语言模型中RL友好性的隐藏驱动因素——分布清晰度，通过量化Silhouette系数并提出Silhouette-aware Reweighting策略，提升了模型在数学基准测试中的表现。
Cross-Block Conditioning in Deep Boltzmann Machines for Statistical Data Fusion
本研究提出观察块多预测，使深度玻尔兹曼机在统计数据融合中可用，并发现跨结果块条件化是性能提升的关键，且不随数据规模衰减。
Inertial Measurement Unit (IMU) sensors are present in everyday devices such as smartphones and fitness watches. As a result, the array of health-related research and applications that tap onto this data has been growing, but little attention has been devoted to the prediction of an individual's heart rate (HR) from IMU data, when undergoing a physical activity. Would that be even possible? If so, this could be used to design personalized sets of aerobic exercises, for instance. In this work, we show that it is viable to obtain accurate HR predictions from IMU data using Recurrent Neural Netwo…
In this paper we investigate the family of functions representable by deep neural networks (DNN) with rectified linear units (ReLU). We give an algorithm to train a ReLU DNN with one hidden layer to *global optimality* with runtime polynomial in the data size albeit exponential in the input dimension. Further, we improve on the known lower bounds on size (from exponential to super exponential) for approximating a ReLU deep net function by a shallower ReLU net. Our gap theorems hold for smoothly parametrized families of "hard" functions, contrary to countable, discrete families known in the lit…
A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes
本文提出将生成对抗网络与贝叶斯推断结合的生成建模框架，在数据稀疏情形下无需显式本构模型形式即可推断生物力学本构关系族，并在小鼠主动脉数据上验证了其准确估计均值和标准差的能力。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。