A Flexible Distributed Stochastic Optimization Framework for Concurrent Tasks in Processing Networks

作     者：Shi, Zai Eryilmaz, Atilla 

作者机构：Ohio State Univ Dept Elect & Comp Engn Columbus OH 43210 USA 

出 版 物：《IEEE-ACM TRANSACTIONS ON NETWORKING》 (IEEE/ACM网络汇刊)

年 卷 期：2021年第29卷第5期

页      面：2045-2058页

核心收录：

学科分类：0810[工学-信息与通信工程] 0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：NSF [CNS-NeTS-1717045, CMMI-SMOR-1562065, CNS-ICN-WEN-1719371, CNS-SpecEES-1824337, CNS-NeTS-2007231] Office of Naval Research (ONR) [N00014-19-1-2621] Defense Threat Reduction Agency (DTRA) [HDTRA1-18-1-0050] 

主　　题：Optimization Program processors Stochastic processes Task analysis Servers Convergence Machine learning Distributed algorithms optimization methods 

摘      要：Distributed stochastic optimization has important applications in the practical implementation of machine learning and signal processing setup by providing means to allow interconnected network of processors to work towards the optimization of a global objective with intermittent communication. Existing works on distributed stochastic optimization predominantly assume all the processors storing related data to perform updates for the optimization task in each iteration. However, such optimization processes are typically executed at shared computing/data centers along with other concurrent tasks. Therefore, it is necessary to develop efficient optimization methods that possess the flexibility to share the computing resources with other ongoing tasks. In this work, we propose a new first-order framework that allows this flexibility through a probabilistic computing resource allocation strategy while guaranteeing the satisfactory performance of distributed stochastic optimization. Our results, both analytical and numerical, show that by controlling a flexibility parameter, our suite of algorithms (designed for various scenarios) can achieve the lower computation and communication costs of distributed stochastic optimization than their inflexible counterparts. This framework also enables the fair sharing of the common resources with other concurrent tasks being processed by the processing network.