More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach

Homotopy Continuation Approaches for Robust SV Classification and Regression

In support vector machine (SVM) applications with unreliable data that contains a portion of outliers, non-robustness of SVMs often causes considerable performance deterioration. Although many approaches for improving the robustness of SVMs have been studied, two major challenges remain in robust SVM learning. First, robust learning algorithms are essentially formulated as non-convex optimization problems. It is thus important to develop a non-convex optimization method for robust SVM that can find a good local optimal solution. The second practical issue is how one can tune the hyperparameter…

Computing Full Conformal Prediction Set with Approximate Homotopy

If you are predicting the label $y$ of a new object with $\hat y$, how confident are you that $y = \hat y$? Conformal prediction methods provide an elegant framework for answering such question by building a $100 (1 - α)\%$ confidence region without assumptions on the distribution of the data. It is based on a refitting procedure that parses all the possibilities for $y$ to select the most likely ones. Although providing strong coverage guarantees, conformal set is impractical to compute exactly for many regression problems. We propose efficient algorithms to compute conformal prediction set u…

同一枚硬币的两面：通过同伦延拓连接深度平衡模型与神经常微分方程

Two Sides of The Same Coin: Bridging Deep Equilibrium Models and Neural ODEs via Homotopy Continuation

本文通过同伦延拓建立DEQs与Neural ODEs的联系，提出新隐式模型HomoODE，兼具高精度与稳定性，并在图像分类任务中超越现有隐式模型。

自引导置信传播——一种同伦延拓方法

Self-Guided Belief Propagation -- A Homotopy Continuation Method

针对图模型概率推断中BP精度不足或不收敛的问题，提出逐步引入成对势的自引导置信传播（SBP）同伦延拓方法，理论上证明其在吸引模型中可达Bethe近似全局最优，实验验证其在BP收敛时精度更高、不收敛时仍能得到唯一稳定准确解，且不增加计算负担。

Conditional Selective Inference for Robust Regression and Outlier Detection using Piecewise-Linear Homotopy Continuation

In practical data analysis under noisy environment, it is common to first use robust methods to identify outliers, and then to conduct further analysis after removing the outliers. In this paper, we consider statistical inference of the model estimated after outliers are removed, which can be interpreted as a selective inference (SI) problem. To use conditional SI framework, it is necessary to characterize the events of how the robust method identifies outliers. Unfortunately, the existing methods cannot be directly used here because they are applicable to the case where the selection events c…

Convergence Analysis of Homotopy-SGD for non-convex optimization

First-order stochastic methods for solving large-scale non-convex optimization problems are widely used in many big-data applications, e.g. training deep neural networks as well as other complex and potentially non-convex machine learning models. Their inexpensive iterations generally come together with slow global convergence rate (mostly sublinear), leading to the necessity of carrying out a very high number of iterations before the iterates reach a neighborhood of a minimizer. In this work, we present a first-order stochastic algorithm based on a combination of homotopy methods and SGD, cal…