事件时间分析，又称生存分析，旨在根据一组特征预测事件发生的时间。该领域的主要挑战之一是处理删失数据，这会让学习算法变得更复杂。Cox比例风险模型和加速失效时间（AFT）模型等传统方法在该领域应用广泛，但它们通常需要满足比例风险、线性等假设。尤其是AFT模型，往往需要预先指定参数分布假设。为提升预测性能、放宽严格假设，近年来已有许多针对风险类模型的深度学习方法。然而，尽管AFT模型相比聚焦风险的方法更具简洁性与可解释性，其在神经网络文献中的表示学习研究仍未得到广泛探索。本文提出用于事件时间预测的深度AFT秩回归模型（Deep AFT Rank-regression model for Time-to-event prediction，DART）。该模型采用基于Gehan秩统计量的目标函数，对于表示学习而言高效且可靠。DART无需建立基线事件时间分布，同时保留了标准AFT模型可直接预测事件时间的优势。该方法是一种半参数AFT建模方法，不对生存时间分布施加任何分布假设；与现有基于神经网络的AFT模型不同，它也无需额外超参数或复杂模型架构。通过在多个基准数据集上的定量分析，我们证明DART在建模高通量删失事件时间数据方面具有显著潜力。
英文摘要：
Time-to-event analysis, also known as survival analysis, aims to predict the time of occurrence of an event, given a set of features. One of the major challenges in this area is dealing with censored data, which can make learning algorithms more complex. Traditional methods such as Cox's proportional hazards model and the accelerated failure time (AFT) model have been popular in this field, but they often require assumptions such as proportional hazards and linearity. In particular, the AFT models often require pre-specified parametric distributional assumptions. To improve predictive performance and alleviate strict assumptions, there have been many deep learning approaches for hazard-based models in recent years. However, representation learning for AFT has not been widely explored in the neural network literature, despite its simplicity and interpretability in comparison to hazard-focused methods. In this work, we introduce the Deep AFT Rank-regression model for Time-to-event prediction (DART). This model uses an objective function based on Gehan's rank statistic, which is efficient and reliable for representation learning. On top of eliminating the requirement to establish a baseline event time distribution, DART retains the advantages of directly predicting event time in standard AFT models. The proposed method is a semiparametric approach to AFT modeling that does not impose any distributional assumptions on the survival time distribution. This also eliminates the need for additional hyperparameters or complex model architectures, unlike existing neural network-based AFT models. Through quantitative analysis on various benchmark datasets, we have shown that DART has significant potential for modeling high-throughput censored time-to-event data.
Demo: SGCode: A Flexible Prompt-Optimizing System for Secure Generation of Code
New Jersey Institute of Technology（新泽西理工学院）；Kent State University（肯特州立大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Microsoft Azure AI（微软 Azure 人工智能）
Health economic evaluations often require predictions of survival rates beyond the follow-up period. Parametric survival models can be more convenient for economic modelling than the Cox model. The generalized gamma (GG) and generalized F (GF) distributions are extensive families that contain almost all commonly used distributions with various hazard shapes and arbitrary complexity. In this study, we present a new SAS macro for implementing a wide variety of flexible parametric models including the GG and GF distributions and their special cases, as well as the Gompertz distribution. Proper cu…
Hong Kong University of Science and Technology（香港科技大学）；Huazhong University of Science and Technology（华中科技大学）
We propose Deep Hierarchical Machine (DHM), a model inspired from the divide-and-conquer strategy while emphasizing representation learning ability and flexibility. A stochastic routing framework as used by recent deep neural decision/regression forests is incorporated, but we remove the need to evaluate unnecessary computation paths by utilizing a different topology and introducing a probabilistic pruning technique. We also show a specified version of DHM (DSHM) for efficiency, which inherits the sparse feature extraction process as in traditional decision tree with pixel-difference feature. …
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Max Planck ETH Center for Learning Systems（马克斯·普朗克-苏黎世联邦理工学院学习系统中心）；ETH Zurich（苏黎世联邦理工学院）
High-contrast imaging of exoplanets hinges on powerful post-processing methods to denoise the data and separate the signal of a companion from its host star, which is typically orders of magnitude brighter. Existing post-processing algorithms do not use all prior domain knowledge that is available about the problem. We propose a new method that builds on our understanding of the systematic noise and the causal structure of the data-generating process. Our algorithm is based on a modified version of half-sibling regression (HSR), a flexible denoising framework that combines ideas from the field…
We develop flexible multi-parameter regression survival models for interval censored survival data arising in longitudinal prospective studies and longitudinal randomised controlled clinical trials. A multi-parameter Weibull regression survival model, which is wholly parametric, and has non-proportional hazards, is the main focus of the paper. We describe the basic model, develop the interval-censored likelihood and extend the model to include gamma frailty and a dispersion model. We evaluate the models by means of a simulation study and a detailed re-analysis of data from the Signal Tandmobie…
For a large class of regularized models, leave-one-out cross-validation can be efficiently estimated with an approximate leave-one-out formula (ALO). We consider the problem of adjusting hyperparameters so as to optimize ALO. We derive efficient formulas to compute the gradient and hessian of ALO and show how to apply a second-order optimizer to find hyperparameters. We demonstrate the usefulness of the proposed approach by finding hyperparameters for regularized logistic regression and ridge regression on various real-world data sets.
Aggregating multiple learners through an ensemble of models aim to make better predictions by capturing the underlying distribution of the data more accurately. Different ensembling methods, such as bagging, boosting, and stacking/blending, have been studied and adopted extensively in research and practice. While bagging and boosting focus more on reducing variance and bias, respectively, stacking approaches target both by finding the optimal way to combine base learners. In stacking with the weighted average, ensembles are created from weighted averages of multiple base learners. It is known …
Department of Electrical Engineering（电气工程系）；Stanford University（斯坦福大学）
Cross-validation is the workhorse of modern applied statistics and machine learning, as it provides a principled framework for selecting the model that maximizes generalization performance. In this paper, we show that the cross-validation risk is differentiable with respect to the hyperparameters and training data for many common machine learning algorithms, including logistic regression, elastic-net regression, and support vector machines. Leveraging this property of differentiability, we propose a cross-validation gradient method (CVGM) for hyperparameter optimization. Our method enables eff…
New York University（纽约大学）；Universidade Federal de São Carlos（圣卡洛斯联邦大学）；Universidade de São Paulo（圣保罗大学）
This paper introduces FlexCodeTS, a new conditional density estimator for time series. FlexCodeTS is a flexible nonparametric conditional density estimator, which can be based on an arbitrary regression method. It is shown that FlexCodeTS inherits the rate of convergence of the chosen regression method. Hence, FlexCodeTS can adapt its convergence by employing the regression method that best fits the structure of data. From an empirical perspective, FlexCodeTS is compared to NNKCDE and GARCH in both simulated and real data. FlexCodeTS is shown to generally obtain the best performance among the …
LMU Munich（慕尼黑大学）；Helmholtz AI（亥姆霍兹人工智能研究所）；ETH Zurich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Helmholtz Zentrum Munich（慕尼黑亥姆霍兹中心）；Flatiron Institute（弗莱提荣研究所）
In this paper we describe the implementation of semi-structured deep distributional regression, a flexible framework to learn conditional distributions based on the combination of additive regression models and deep networks. Our implementation encompasses (1) a modular neural network building system based on the deep learning library \pkg{TensorFlow} for the fusion of various statistical and deep learning approaches, (2) an orthogonalization cell to allow for an interpretable combination of different subnetworks, as well as (3) pre-processing steps necessary to set up such models. The softwar…
Flexible Cutoff Learning: Optimizing Machine Learning Potentials After Training
Institute for Numerical Simulation, University of Bonn（波恩大学数值模拟研究所）；Fraunhofer Institute for Algorithms and Scientific Computing SCAI（弗劳恩霍夫算法与科学计算研究所SCAI）
Purpose: Arterial Spin Labeling (ASL) is a quantitative, non-invasive alternative to perfusion imaging with contrast agents. Fixing values of certain model parameters in traditional ASL, which actually vary from region to region, may introduce bias in perfusion estimates. Adopting Magnetic Resonance Fingerprinting (MRF) for ASL is an alternative where these parameters are estimated alongside perfusion, but multiparametric estimation can degrade precision. We aim to improve the sensitivity of ASL-MRF signals to underlying parameters to counter this problem, and provide precise estimates. We als…
A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression
Department of Statistics The University of Chicago（统计学系芝加哥大学）；Department of Statistics University of California, Irvine（统计学系加州大学伊文斯顿分校）；Department of Statistics and Department of Human Genetics The University of Chicago（统计学系和人类遗传学系芝加哥大学）
Hybrid Bernstein Normalizing Flows for Flexible Multivariate Density Regression with Interpretable Marginals
Institute for Applied Research HTWG - University of Applied Sciences（应用研究研究所 HTWG - 高等专业学院）；Chair of Statistics and Campus Institute Data Science (CIDAS) University of Göttingen（统计学教授及校园数据科学研究所 (CIDAS) 雷姆森大学）；Department of Statistics LMU Munich（统计学系 慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Nanjing University of Information Science & Technology（南京信息工程大学）；JD Finance America Corporation（京东金融美国公司）；University of Western Ontario（西安大略大学）；University of Pittsburgh（匹兹堡大学）
Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled. Recent researches have shown that directly optimizing concordance index or AUC can impose a better ranking on the data than optimizing the traditional error rate in ordinal regression (OR) problems. In this paper, we propose an unbiased objective function for S$^2$OR AUC optimization based on ordinal binary decomposition approach. Besides, to handle the large-scale kernelized learning problems, we propose a scala…
University of British Columbia（不列颠哥伦比亚大学）；Carleton University（卡尔顿大学）
This paper proposes a new method to address the long-standing problem of lack of monotonicity in estimation of the conditional and structural quantile function, also known as quantile crossing problem. Quantile regression is a very powerful tool in data science in general and econometrics in particular. Unfortunately, the crossing problem has been confounding researchers and practitioners alike for over 4 decades. Numerous attempts have been made to find a simple and general solution. This paper describes a unique and elegant solution to the problem based on a flexible check function that is e…
Regression-via-Classification (RvC) is the process of converting a regression problem to a classification one. Current approaches for RvC use ad-hoc discretization strategies and are suboptimal. We propose a neural regression tree model for RvC. In this model, we employ a joint optimization framework where we learn optimal discretization thresholds while simultaneously optimizing the features for each node in the tree. We empirically show the validity of our model by testing it on two challenging regression tasks where we establish the state of the art.
Gaussian Process (GP) regression is a flexible non-parametric approach to approximate complex models. In many cases, these models correspond to processes with bounded physical properties. Standard GP regression typically results in a proxy model which is unbounded for all temporal or spacial points, and thus leaves the possibility of taking on infeasible values. We propose an approach to enforce the physical constraints in a probabilistic way under the GP regression framework. In addition, this new approach reduces the variance in the resulting GP model.
Optimizing Estimators of Squared Calibration Errors in Classification
German Cancer Consortium (DKTK)（德国癌症联盟（DKTK））；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Goethe University Frankfurt（法兰克福大学）；Inria（法国国家信息与自动化研究所）；Ecole Normale Supérieure（巴黎高等师范学院）；PSL Research University（巴黎文理研究大学）
IBM Thomas J. Watson Research Center（IBM托马斯·J·沃森研究中心）
Generating high quality uncertainty estimates for sequential regression, particularly deep recurrent networks, remains a challenging and open problem. Existing approaches often make restrictive assumptions (such as stationarity) yet still perform poorly in practice, particularly in presence of real world non-stationary signals and drift. This paper describes a flexible method that can generate symmetric and asymmetric uncertainty estimates, makes no assumptions about stationarity, and outperforms competitive baselines on both drift and non drift scenarios. This work helps make sequential regre…
We introduce a flexible family of fairness regularizers for (linear and logistic) regression problems. These regularizers all enjoy convexity, permitting fast optimization, and they span the rang from notions of group fairness to strong individual fairness. By varying the weight on the fairness regularizer, we can compute the efficient frontier of the accuracy-fairness trade-off on any given dataset, and we measure the severity of this trade-off via a numerical quantity we call the Price of Fairness (PoF). The centerpiece of our results is an extensive comparative study of the PoF across six d…
Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking. In this work we present a novel quasi-Bayesian procedure for IV regression, building upon the recently developed kernelized IV models and the dual/minimax formulation of IV regression. We analyze the frequentist behavior of the proposed method, by establishing minimax optimal contraction rates in $L_2$ and Sobolev norms, and discussing the frequentist validity of credible balls…
Flatiron Institute, a division of the Simons Foundation Center for Cosmology and Particle Physics, Department of Physics, New York University（纽约大学纽约大学西蒙斯基金会宇宙学与粒子物理中心分属的Flatiron研究所，物理系）；Center for Data Science, New York University（纽约大学数据科学中心）；Max-Planck-Institut für Astronomie, Heidelberg（海德堡马克斯-普朗克天文研究所）；Department of Applied Mathematics & Statistics, Johns Hopkins University（约翰斯·霍普金斯大学应用数学与统计系）；Mathematical Institute for Data Science, Johns Hopkins University（约翰斯·霍普金斯大学数据科学数学研究所）
There are many uses for linear fitting; the context here is interpolation and denoising of data, as when you have calibration data and you want to fit a smooth, flexible function to those data. Or you want to fit a flexible function to de-trend a time series or normalize a spectrum. In these contexts, investigators often choose a polynomial basis, or a Fourier basis, or wavelets, or something equally general. They also choose an order, or number of basis functions to fit, and (often) some kind of regularization. We discuss how this basis-function fitting is done, with ordinary least squares an…
University of Oslo（奥斯陆大学）；Norwegian Computing Center（挪威计算中心）；Medical University of Vienna（维也纳医科大学）
Regression models are used in a wide range of applications providing a powerful scientific tool for researchers from different fields. Linear, or simple parametric, models are often not sufficient to describe complex relationships between input variables and a response. Such relationships can be better described through flexible approaches such as neural networks, but this results in less interpretable models and potential overfitting. Alternatively, specific parametric nonlinear functions can be used, but the specification of such functions is in general complicated. In this paper, we introdu…
We present neural mixture distributional regression (NMDR), a holistic framework to estimate complex finite mixtures of distributional regressions defined by flexible additive predictors. Our framework is able to handle a large number of mixtures of potentially different distributions in high-dimensional settings, allows for efficient and scalable optimization and can be applied to recent concepts that combine structured regression models with deep neural networks. While many existing approaches for mixture models address challenges in optimization of such and provide results for convergence u…
