In this work, we demonstrate provable guarantees on the training of a single ReLU gate in hitherto unexplored regimes. We give a simple iterative stochastic algorithm that can train a ReLU gate in the realizable setti...
详细信息
In this work, we demonstrate provable guarantees on the training of a single ReLU gate in hitherto unexplored regimes. We give a simple iterative stochastic algorithm that can train a ReLU gate in the realizable setting in linear time while using significantly milder conditions on the data distribution than previous such results.& nbsp;Leveraging certain additional moment assumptions, we also show a first-of-its-kind approximate recovery of the true label generating parameters under an (online) data-poisoning attack on the true labels, while training a ReLU gate by the same algorithm. Our guarantee is shown to be nearly optimal in the worst case and its accuracy of recovering the true weight degrades gracefully with increasing probability of attack and its magnitude.& nbsp;For both the realizable and the non-realizable cases as outlined above, our analysis allows for minibatching and computes how the convergence time scales with the mini-batch size. We corroborate our theorems with simulation results which also bring to light a striking similarity in trajectories between our algorithm and the popular S.G.D. algorithm - for which similar guarantees as here are still unknown. (C) 2022 Elsevier Ltd. All rights reserved.
This paper reviews the vast literature on static output feedback design for linear time-invariant systems including classical results and recent developments. In particular, we focus on static output feedback synthesi...
详细信息
This paper reviews the vast literature on static output feedback design for linear time-invariant systems including classical results and recent developments. In particular, we focus on static output feedback synthesis with performance specifications, structured static output feedback, and robustness. The paper provides a comprehensive review on existing design approaches including iterative linear matrix inequalities heuristics, linear matrix inequalities with rank constraints, methods with decoupled Lyapunov matrices, and non-Lyapunov-based approaches. We describe the main difficulties of dealing with static output feedback design and summarize the main features, advantages, and limitations of existing design methods. (C) 2016 International Federation of Automatic Control. Published by Elsevier Ltd. All rights reserved.
We investigate the convergence of subgradient-oriented descent methods in non-smooth non-convex optimization. We prove convergence in the sense of subsequences for functions with a strict standard model, and we show t...
详细信息
We investigate the convergence of subgradient-oriented descent methods in non-smooth non-convex optimization. We prove convergence in the sense of subsequences for functions with a strict standard model, and we show that convergence to a single critical point may be guaranteed if the Kurdyka-Aojasiewicz inequality is satisfied. We show, by way of an example, that the Kurdyka-Aojasiewicz inequality alone is not sufficient to prove the convergence to critical points.
暂无评论