We consider forward-backward greedy algorithms for solving sparse feature selection problems with general convex smooth functions. A state-of-the-art greedy method, the Forward-Backward greedy algorithm (FoBa-obj) requires to solve a large number of optimization problems, thus it is not scalable for large-size problems. The FoBa-gdt algorithm, which uses the gradient information for feature selection at each forward iteration, significantly improves the efficiency of FoBa-obj. In this paper, we systematically analyze the theoretical properties of both forward-backward greedy algorithms. Our main contributions are: 1) We derive better theoretical bounds than existing analyses regarding FoBa-obj for general smooth convex functions; 2) We show that FoBa-gdt achieves the same theoretical performance as FoBa-obj under the same condition: restricted strong convexity condition. Our new bounds are consistent with the bounds of a special case (least squares) and fills a previously existing theoretical gap for general convex smooth functions; 3) We show that the restricted strong convexity condition is satisfied if the number of independent samples is more than $\bar{k}\log d$ where $\bar{k}$ is the sparsity number and $d$ is the dimension of the variable; 4) We apply FoBa-gdt (with the conditional random field objective) to the sensor selection problem for human indoor activity recognition and our results show that FoBa-gdt outperforms other methods (including the ones based on forward greedy selection and L1-regularization).
Multi-Objective Maximization of Monotone Submodular Functions with Cardinality Constraint
考虑基数约束下单调次模函数的多目标最大化问题,通常表示为$\max_{|A|=k}\min_{i\in\{1,\dots,m\}}f_i(A)$。虽然众所周知,贪婪方法对单个目标很有效,但对于多个目标,问题变得更加困难。事实上,Krause et al.(2008)表明,当目标$m$的数量随着基数$k$的增长时,即,$m=Ω(k)$,则问题不可近似(除非是$P=NP$)。另一方面,当$m$是常数时,Chekuri et al.(2010)展示了一个随机的$(1-1/e)-ε$近似与运行时(查询函数oracle的次数)$n^{m/ε^3}$。%事实上,Chekuri et al.(2010)是针对更一般的拟阵常数的情况。我们专注于找到一个快速和实用的算法,具有(渐近)近似保证,即使当$m$是超级常数。我们首先修改Chekuri等人的算法。(2010)以实现$m=o(\frac{k}{\log^3 k})$的$(1-1/e)$近似。这证明了在$m=Ω(k)$附近从常数因子可近似性到不可近似性的急剧转变。然后使用多重权重更新(MWU),我们找到了一个更快的$\tilde{O}(n/δ^3)$时间渐近$(1-1/e)^2-δ$近似。虽然上述结果都是随机的,我们也给出了一个简单的确定性$(1-1/e)-ε$近似与运行时$kn^{m/ε^4}$。最后,我们使用Kronecker图进行合成实验,…
Robust Submodular Maximization: A Non-Uniform Partitioning Approach
EPFL(洛桑联邦理工学院)
我们研究的问题最大化的单调子模函数的基数约束$k$,增加的扭曲,一些项目$τ$从返回的集合可能会被删除。我们专注于(Orlin等人,2016),其中为$τ= o(\sqrt{k})$提供了恒定因子近似保证。在本文中,我们解决了其中提出的一个关键的开放问题,提出了一个新的分区鲁棒(PRo)子模块最大化算法,实现了更一般的$τ= o(k)$相同的保证。我们的算法构造分区组成的桶呈指数增长的大小,并应用标准的子模块优化子程序的桶,以构建强大的解决方案。我们用数值方法证明了PRo在数据汇总和影响最大化方面的性能,证明了贪婪算法和(Orlin et al.,2016)。