Transformers have achieved significant success in various fields, notably excelling in tasks involving sequential data like natural language processing. Despite these achievements, the theoretical understanding of transformers' capabilities remains limited. In this paper, we investigate the theoretical capabilities of transformers to autoregressively generate sequences in Bayesian networks based on in-context maximum likelihood estimation (MLE). Specifically, we consider a setting where a context is formed by a set of independent sequences generated according to a Bayesian network. We demonstrate that there exists a simple transformer model that can (i) estimate the conditional probabilities of the Bayesian network according to the context, and (ii) autoregressively generate a new sample according to the Bayesian network with estimated conditional probabilities. We further demonstrate in extensive experiments that such a transformer does not only exist in theory, but can also be effectively obtained through training. Our analysis highlights the potential of transformers to learn complex probabilistic models and contributes to a better understanding of large language models as a powerful class of sequence generators.
High-dimensional Inference and FDR Control for Simulated Markov Random Fields
University of California San Diego（加利福尼亚大学圣迭戈分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；University of Alberta（阿尔伯塔大学）；Beihang University（北京航空航天大学）；Zhuhai UM Science & Technology Research Institute（珠海澳大科技研究院）
A Margin-based MLE for Crowdsourced Partial Ranking
Tencent AI Lab（腾讯人工智能实验室）；Peking University（北京大学）；DeepWise AI Lab（深睿医疗人工智能实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong University of Science and Technology（香港科技大学）
从成对比较数据聚合的偏好顺序或排名通常被理解为严格的总顺序。然而，在现实世界的情况下，一些项目在比较中本质上是模糊的，这很可能是数据的固有不确定性。在这种情况下，传统的总订单排名不能捕捉这样的不确定性与单纯的全球排名或效用得分。在本文中，我们特别感兴趣的是最近激增的众包应用程序，以预测部分，但更准确（即，使不太不正确的陈述）命令而不是完整的命令。为此，我们提出了一个新的框架来学习一些偏序概率模型，作为\emph{margin-based Maximum Likelihood Estimate}（MLE）方法。我们证明了诱导的极大似然估计是一个关于所有参数的联合凸优化问题，包括全局排名分数和边缘参数。研究了基本均匀模型、Bradley-Terry模型和Thurstone-Mosteller模型三种广义线性模型，并对所提方法的FDR和功率控制进行了理论分析。这些模型的有效性得到了仿真和真实数据集的实验支持，这表明所提出的模型与传统的最先进的算法相比表现出了改进。
Global Parameter Identification and Uncertainty Quantification of reduced-order ADM1 using Maximum Likelihood Estimation and Bayesian Inference
DBFZ Deutsches Biomasseforschungszentrum gGmbH（德国生物质研究中心）；Chemnitz University of Technology（开姆尼茨工业大学）；Technische Universität Berlin（柏林工业大学）；University of Cambridge（剑桥大学）；Leipzig University（莱比锡大学）；FH Münster University of Applied Sciences（明斯特应用科学大学）
Bivariate Frank Copula: Some More Results on Point Estimation of the Association Parameter from a Bayesian Perspective and Revisiting the Goodness of Fit Tests with an Application to Model Groundwater Data from Dong Thap, Vietnam
Faculty of Applied Science, Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学应用科学学院）；Vietnam National University Ho Chi Minh City（越南国家大学胡志明市分校）；Faculty of Mathematics and Statistics, Ton Duc Thang University（Ton Duc Thang大学数学与统计学院）
Model averaging with mixed criteria for estimating high quantiles of extreme values: Application to heavy rainfall
Honam Research Division, Electronics and Telecommunications Research Institute（韩国电子与电信研究院湖南研究部）；Department of Statistics, Chonnam National University（全南大学统计系）
Adaptive Bayesian Optimization for Robust Identification of Stochastic Dynamical Systems
National Institute of Standards and Technology（国家标准化与技术机构）；Rice University（里士满大学）；Department of Physics, Colorado State University（科罗拉多州立大学物理系）；Electrical Engineering Department, University of Colorado（科罗拉多大学电气工程系）；National Research Institute for Metals（金属研究国家研究所）
Gradient-free variational learning with conditional mixture networks
VERSES AI Research Lab（VERSES AI研究实验室）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Technische Universität Dresden（德累斯顿工业大学）；University of Sussex（萨塞克斯大学）；Duke University（杜克大学）
Adversarial Examples Detection with Bayesian Neural Network
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of California Davis（加利福尼亚大学戴维斯分校）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）
