We consider the problem of convex constrained minimization of an average of n functions, where the parameter and the features are related through inner products. We focus on second order batch updates, where the curva...
详细信息
ISBN:
(纸本)9781509052943
We consider the problem of convex constrained minimization of an average of n functions, where the parameter and the features are related through inner products. We focus on second order batch updates, where the curvature matrix is obtained by assuming random design and by applying the celebrated Stein's lemma together with subsampling techniques. The proposed algorithm enjoys fast convergence rates similar to the Newton method, yet the per-iteration cost has the same order of magnitude as the gradient descent. We demonstrate its performance on well-known optimization problems where Stein's lemma is not directly applicable, such as M-estimation for robust statistics, and inequality form linear/quadratic programming etc. Under certain assumptions, we show that the constrained optimization algorithm attains a composite convergence rate that is initially quadratic and asymptotically linear. We validate its performance through widely encountered optimization tasks on several real and synthetic datasets by comparing it to classical optimization algorithms.
It is well known from queueing and simulation models that cycle times in capacitated production systems increase nonlinearly with resource utilization, which poses considerable difficulty for the conventional linear p...
详细信息
ISBN:
(纸本)9781424427086
It is well known from queueing and simulation models that cycle times in capacitated production systems increase nonlinearly with resource utilization, which poses considerable difficulty for the conventional linear programming (LP) models used for this purpose. Hung and Leachman (1996) propose a highly intuitive iterative approach where a detailed simulation model of the production facility is used to estimate flow time parameters used in an LP model We examine the convergence of this method under different experimental conditions, and conclude that it is hard to determine precisely when the method converges.
As the problem of minimizing functionals on the Wasserstein space encompasses many applications in machine learning, different optimization algorithms on d have received their counterpart analog on the Wasserstein spa...
详细信息
Quality diversity (QD) algorithms have shown to provide sets of high quality solutions for challenging problems in robotics, games, and combinatorial optimisation. So far, theoretical foundational explaining their goo...
详细信息
We consider a class of sparsity-inducing regularization terms based on submodular functions. While previous work has focused on non-decreasing functions, we explore symmetric submodular functions and their Lovasz exte...
详细信息
ISBN:
(纸本)9781618395993
We consider a class of sparsity-inducing regularization terms based on submodular functions. While previous work has focused on non-decreasing functions, we explore symmetric submodular functions and their Lovasz extensions. We show that the Lovasz extension may be seen as the convex envelope of a function that depends on level sets (i.e., the set of indices whose corresponding components of the underlying predictor are greater than a given constant): this leads to a class of convex structured regularization terms that impose prior knowledge on the level sets, and not only on the supports of the underlying predictors. We provide unified optimization algorithms, such as proximal operators, and theoretical guarantees (allowed level sets and recovery conditions). By selecting specific submodular functions, we give a new interpretation to known norms, such as the total variation; we also define new norms, in particular ones that are based on order statistics with application to clustering and outlier detection, and on noisy cuts in graphs with application to change point detection in the presence of outliers.
We study optimization problems in ergodic theory from the view point of minimax problems. We give minimax characterizations of maximum ergodic averages involving time averages. Our approach also works for the abstract...
详细信息
Joint processing between base stations is a promising technique to improve the quality of service to users at the cell edge, but this technique poses tremendous requirements on the backhaul signaling capabilities. Par...
详细信息
ISBN:
(纸本)9781467309899
Joint processing between base stations is a promising technique to improve the quality of service to users at the cell edge, but this technique poses tremendous requirements on the backhaul signaling capabilities. Partial joint processing is a technique aimed to reduce feedback load, in one approach the users feed back the channel state information of the best links based on a channel gain threshold mechanism. However, it has been shown in the literature that the reduction in the feedback load is not reflected in an equivalent backhaul reduction, unless additional scheduling or precoding techniques are applied. The reason is that reduced feedback from users yields sparse channel state information at the Central Coordination Node. Under these conditions, existing linear precoding techniques fail to remove the interference and reduce backhaul, simultaneously, unless constraints are imposed on scheduling. In this paper, a partial joint processing scheme with efficient backhauling is proposed, based on a stochastic optimization algorithm called particle swarm optimization. The use of particle swarm optimization in the design of the precoder promises efficient backhauling with improved sum rate.
The practical implementation of quantum optimization algorithms on noisy intermediate-scale quantum devices requires accounting for their limited connectivity. As such, the Parity architecture was introduced to overco...
详细信息
Bandit algorithms have garnered significant attention due to their practical applications in real-world scenarios. However, beyond simple settings such as multi-arm or linear bandits, optimal algorithms remain scarce....
详细信息
When are two algorithms the same? How can we be sure a recently proposed algorithm is novel, and not a minor twist on an existing method? In this paper, we present a framework for reasoning about equivalence between a...
详细信息
暂无评论