Given a separation oracle SO for a convex function f defined on R-n that has an integral minimizer inside a box with radius R, we show how to find an exact minimizer of f using at most O(n(n log log(n)/ log(n) + log(R...
详细信息
Given a separation oracle SO for a convex function f defined on R-n that has an integral minimizer inside a box with radius R, we show how to find an exact minimizer of f using at most O(n(n log log(n)/ log(n) + log(R))) calls to SO and poly(n, log(R)) arithmetic operations, or O(n log(nR)) calls to SO and exp(O(n)) center dot poly(log(R)) arithmetic operations. When the set of minimizers of f has integral extreme points, our algorithm outputs an integral minimizer of f. This improves upon the previously best oracle complexity of O(n(2) (n + log(R))) for polynomial time algorithms and O(n(2) log(nR)) for exponential time algorithms obtained by [Grotschel, Lovasz and Schrijver, Prog. Comb. Opt. 1984, Springer 1988] over thirty years ago. Our improvement on Grotschel, Lovasz and Schrijver's result generalizes to the setting where the set of minimizers of f is a rational polyhedron with bounded vertex complexity. For the submodular function minimization problem, our result immediately implies a strongly polynomial algorithm that makes at most O(n(3) log log(n)/ log(n)) calls to an evaluation oracle, and an exponential time algorithm that makes at most O(n(2) log(n)) calls to an evaluation oracle. These improve upon the previously best O(n(3) log(2) (n)) oracle complexity for strongly polynomial algorithms given in [Lee, Sidford and Wong, FOCS 2015] and [Dadush, Vegh and Zambelli, SODA 2018], and an exponential time algorithm with oracle complexity O(n(3) log(n)) given in the former work. Our result is achieved via a reduction to the Shortest Vector Problem in lattices. We show how an approximately shortest vector of an auxiliary lattice can be used to effectively reduce the dimension of the problem. Our analysis of the oracle complexity is based on a potential function that simultaneously captures the size of the search set and the density of the lattice, which we analyze via tools from convex geometry and lattice theory.
In model selection problems for machine learning, the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem. In many scenarios, however, the mea...
详细信息
In model selection problems for machine learning, the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem. In many scenarios, however, the meaningful structure is specified in some discrete space, leading to difficult nonconvex optimization problems. In this paper, we connect the model selection problem with structure-promoting regularizers to submodular function minimization with continuous and discrete arguments. In particular, we leverage the theory of submodularfunctions to identify a class of these problems that can be solved exactly and efficiently with an agnostic combination of discrete and continuous optimization routines. We show how simple continuous or discrete constraints can also be handled for certain problem classes, and extend these ideas to a robust optimization framework. We also show how some problems outside of this class can be embedded into the class, further extending the class of problems our framework can accommodate. Finally, we numerically validate our theoretical results with several proof-of-concept examples with synthetic and real-world data, comparing against state-of-the-art algorithms.
We consider the submodular function minimization (SFM) and the quadratic minimization problems regularized by the Lovász extension of the submodularfunction. These optimization problems are intimately related; f...
详细信息
We consider the submodular function minimization (SFM) and the quadratic minimization problems regularized by the Lovász extension of the submodularfunction. These optimization problems are intimately related; for example, min-cut problems and total variation denoising problems, where the cut function is submodular and its Lovász extension is given by the associated total variation. When a quadratic loss is regularized by the total variation of a cut function, it thus becomes a total variation denoising problem and we use the same terminology in this paper for "general" submodularfunctions. We propose a new active-set algorithm for total variation denoising with the assumption of an oracle that solves the corresponding SFM problem. This can be seen as local descent algorithm over ordered partitions with explicit convergence guarantees. It is more flexible than the existing algorithms with the ability for warm-restarts using the solution of a closely related problem. Further, we also consider the case when a submodularfunction can be decomposed into the sum of two submodularfunctions F1 and F2 and assume SFM oracles for these two functions. We propose a new active-set algorithm for total variation denoising (and hence SFM by thresholding the solution at zero). This algorithm also performs local descent over ordered partitions and its ability to warm start considerably improves the performance of the algorithm. In the experiments, we compare the performance of the proposed algorithms with state-of-the-art algorithms, showing that it reduces the calls to SFM oracles.
暂无评论