首页/论文/1908.09915

arXiv 1908.099152019-08-26stat.MLcs.LGmath.OC

Convex Programming for Estimation in Nonlinear Recurrent Models

Georgia Institute of Technology（佐治亚理工学院）

Sohail Bahmani, Justin Romberg

备注：: 18 pages

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We propose a formulation for nonlinear recurrent models that includes simple parametric models of recurrent neural networks as a special case. The proposed formulation leads to a natural estimator in the form of a convex program. We provide a sample complexity for this estimator in the case of stable dynamics, where the nonlinear recursion has a certain contraction property, and under certain regularity conditions on the input distribution. We evaluate the performance of the estimator by simulation on synthetic data. These numerical experiments also suggest the extent at which the imposed theoretical assumptions may be relaxed.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

IPBoost -- Non-Convex Boosting via Integer Programming

TU Darmstadt（达姆施塔特工业大学）；TU Berlin（柏林工业大学）；Zuse Institute Berlin（柏林祖斯研究所）

Recently non-convex optimization approaches for solving machine learning problems have gained significant attention. In this paper we explore non-convex boosting in classification by means of integer programming and demonstrate real-world practicability of the approach while circumventing shortcomings of convex boosting approaches. We report results that are comparable to or better than the current state-of-the-art.

2020-02-11cs.LGmath.OCstat.ML

Recurrent Neural Network Training with Convex Loss and Regularization Functions by Extended Kalman Filtering

IMT School for Advanced Studies（IMT高等研究院）

This paper investigates the use of extended Kalman filtering to train recurrent neural networks with rather general convex loss functions and regularization terms on the network parameters, including $\ell_1$-regularization. We show that the learning method is competitive with respect to stochastic gradient descent in a nonlinear system identification benchmark and in training a linear system with binary outputs. We also explore the use of the algorithm in data-driven nonlinear model predictive control and its relation with disturbance models for offset-free closed-loop tracking.

2022-11-02cs.LGcs.SYeess.SY

Step size adaptation in first-order method for stochastic strongly convex programming

We propose a first-order method for stochastic strongly convex optimization that attains $O(1/n)$ rate of convergence, analysis show that the proposed method is simple, easily to implement, and in worst case, asymptotically four times faster than its peers. We derive this method from several intuitive observations that are generalized from existing first order optimization methods.

2011-10-13cs.LGmath.OC

On the convergence of mirror descent beyond stochastic convex programming

Stanford University（斯坦福大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）

In this paper, we examine the convergence of mirror descent in a class of stochastic optimization problems that are not necessarily convex (or even quasi-convex), and which we call variationally coherent. Since the standard technique of "ergodic averaging" offers no tangible benefits beyond convex programming, we focus directly on the algorithm's last generated sample (its "last iterate"), and we show that it converges with probabiility $1$ if the underlying problem is coherent. We further consider a localized version of variational coherence which ensures local convergence of stochastic mirro…

2018-07-16cs.LGmath.OC

凸回归的集成方法及其在基于几何规划的电路设计中的应用

Ensemble Methods for Convex Regression with Applications to Geometric Programming Based Circuit Design

Duke University（杜克大学）

本文提出集成方法（如bagging、smearing和随机划分）来改进分段线性凸回归的稳定性，并应用于基于几何规划的电路设计中的器件建模和约束近似。

2012-06-18cs.LGcs.NAmath.NA

使用凸-凹规划训练单层形态学感知机

Training Single-Layer Morphological Perceptron Using Convex-Concave Programming

Federal University of Technology – Paraná（巴拉那联邦理工大学）；Universidade Estadual de Campinas (Unicamp)（坎皮纳斯州立大学）

本文提出一种名为 K-DDCCP 的训练算法，结合 SLMP 模型与 WDCCP 算法，利用规范化凸-凹规划解决非凸优化问题，有效提升了单层形态学感知机在二分类任务上的能力。

2024-01-04cs.LG

更多相关论文收起更多论文

Strongly-Typed Recurrent Neural Networks

Victoria University of Wellington（惠灵顿维多利亚大学）；Weta Digital（维塔数码）

Recurrent neural networks are increasing popular models for sequential learning. Unfortunately, although the most effective RNN architectures are perhaps excessively complicated, extensive searches have not found simpler alternatives. This paper imports ideas from physics and functional programming into RNN design to provide guiding principles. From physics, we introduce type constraints, analogous to the constraints that forbids adding meters to seconds. From functional programming, we require that strongly-typed architectures factorize into stateless learnware and state-dependent firmware, r…

2016-05-24cs.LGcs.NE

A Convex Parameterization of Robust Recurrent Neural Networks

Australian Centre for Field Robotics（澳大利亚野外机器人中心）；The University of Sydney（悉尼大学）

Recurrent neural networks (RNNs) are a class of nonlinear dynamical systems often used to model sequence-to-sequence maps. RNNs have excellent expressive power but lack the stability or robustness guarantees that are necessary for many applications. In this paper, we formulate convex sets of RNNs with stability and robustness guarantees. The guarantees are derived using incremental quadratic constraints and can ensure global exponential stability of all solutions, and bounds on incremental $ \ell_2 $ gain (the Lipschitz constant of the learned sequence-to-sequence mapping). Using an implicit m…

2020-10-03cs.LGcs.SYeess.SY

Difference of Convex Functions Programming Applied to Control with Expert Data

Univ. Lille（里尔大学）；CRIStAL (UMR CNRS 9189/Lille 1)（CRIStAL研究中心（UMR CNRS 9189/里尔第一大学））；CentraleSupélec（巴黎中央理工-高等电力学院）；IUF（法国大学研究院）；Google DeepMind（谷歌DeepMind）

This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement in performance is demonstrated on two specific algorithms, namely Reward-regularized Classification for Apprenticeship Learning (RCAL) and Reinforcement Learning with Expert Demonstrations (RLED), through experiments on generic Markov Decision Processes (MDP), called Garnets.

2016-09-05cs.LGmath.OCstat.ML

规范化测地凸规划

Disciplined Geodesically Convex Programming

Harvard University（哈佛大学）；MIT（麻省理工学院）

本文将规范化凸规划（DCP）框架扩展到Hadamard流形上的测地凸函数，提出规范化测地凸规划（DGCP），并配套Julia软件包SymbolicAnalysis.jl用于自动验证和求解测地凸规划问题。

2025-08-19cs.LGcs.MSmath.OC

Generalized Uniformly Optimal Methods for Nonlinear Programming

In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search step (gradient descent or Quasi-Newton iteration) into these uniformly optimal convex programming methods, and then enforce a monotone decreasing property of the function values computed along the trajectory. Algorithms of these types will then achieve the best known complexity for nonconvex problems, and the optimal complexity for convex ones without requiring…

2015-09-12math.OCstat.ML

一类结构化非线性规划的序列凸规划方法

Sequential Convex Programming Methods for A Class of Structured Nonlinear Programming

University of Minnesota（明尼苏达大学）

本文研究一类结构化非线性规划问题，提出序列凸规划方法，并证明生成序列的聚点为KKT点。

2022-06-17cs.NAmath.NAmath.OC

$l_0$ 正则化凸锥规划问题的迭代硬阈值方法

Iterative Hard Thresholding Methods for $l_0$ Regularized Convex Cone Programming

Simon Fraser University（西蒙弗雷泽大学）；Texas A&M University（得克萨斯农工大学）

提出迭代硬阈值方法及其变体求解 $l_0$ 正则化凸锥规划，证明收敛到局部极小点并建立迭代复杂度。

2012-11-02cs.LGcs.NAmath.NA

On Generation of Adversarial Examples using Convex Programming

RWTH Aachen University（亚琛工业大学）

It has been observed that deep learning architectures tend to make erroneous decisions with high reliability for particularly designed adversarial instances. In this work, we show that the perturbation analysis of these architectures provides a framework for generating adversarial instances by convex programming which, for classification tasks, is able to recover variants of existing non-adaptive adversarial methods. The proposed framework can be used for the design of adversarial noise under various desirable constraints and different types of networks. Moreover, this framework is capable of …

2018-12-03cs.LGstat.ML