Deep reinforcement learning under signal temporal logic constraints using Lagrangian relaxation

Learning to Solve Soft-Constrained Vehicle Routing Problems with Lagrangian Relaxation

Vehicle Routing Problems (VRPs) in real-world applications often come with various constraints, therefore bring additional computational challenges to exact solution methods or heuristic search approaches. The recent idea to learn heuristic move patterns from sample data has become increasingly promising to reduce solution developing costs. However, using learning-based approaches to address more types of constrained VRP remains a challenge. The difficulty lies in controlling for constraint violations while searching for optimal solutions. To overcome this challenge, we propose a Reinforcement…

Efficient Optimistic Exploration in Linear-Quadratic Regulators via Lagrangian Relaxation

We study the exploration-exploitation dilemma in the linear quadratic regulator (LQR) setting. Inspired by the extended value iteration algorithm used in optimistic algorithms for finite MDPs, we propose to relax the optimistic optimization of \ofulq and cast it into a constrained \textit{extended} LQR problem, where an additional control variable implicitly selects the system dynamics within a confidence interval. We then move to the corresponding Lagrangian formulation for which we prove strong duality. As a result, we show that an $ε$-optimistic controller can be computed efficiently by sol…

多动作部分可观测不安分多臂老虎机的拉格朗日松弛：启发式策略与可索引性

Lagrangian Relaxation for Multi-Action Partially Observable Restless Bandits: Heuristic Policies and Indexability

针对多动作部分可观测不安分多臂老虎机问题，研究其拉格朗日界计算近似方法、值函数性质、启发式策略及Whittle索引策略的局限性，为公共卫生干预等场景的资源分配提供理论支撑。

两阶段确定性批量模式强化学习的最小最大泛化：松弛方案

Min Max Generalization for Two-stage Deterministic Batch Mode Reinforcement Learning: Relaxation Schemes

针对确定性批量模式强化学习中的最小最大优化问题，提出两种松弛方案（约束丢弃和拉格朗日对偶化）以降低计算复杂度，并证明其优于现有方法。

On the Lagrangian Biduality of Sparsity Minimization Problems

Recent results in Compressive Sensing have shown that, under certain conditions, the solution to an underdetermined system of linear equations with sparsity-based regularization can be accurately recovered by solving convex relaxations of the original problem. In this work, we present a novel primal-dual analysis on a class of sparsity minimization problems. We show that the Lagrangian bidual (i.e., the Lagrangian dual of the Lagrangian dual) of the sparsity minimization problems can be used to derive interesting convex relaxations: the bidual of the $\ell_0$-minimization problem is the $\ell_…

Zonotope Domains for Lagrangian Neural Network Verification

Neural network verification aims to provide provable bounds for the output of a neural network for a given input range. Notable prior works in this domain have either generated bounds using abstract domains, which preserve some dependency between intermediate neurons in the network; or framed verification as an optimization problem and solved a relaxation using Lagrangian methods. A key drawback of the latter technique is that each neuron is treated independently, thereby ignoring important neuron interactions. We provide an approach that merges these two threads and uses zonotopes within a La…