Aiming at solving the performance degradation of federated learning (FL) under heterogeneous data distribution, personalized FL (PFL) was proposed. It is designed to produce a dedicated model for each client. However,...
详细信息
Aiming at solving the performance degradation of federated learning (FL) under heterogeneous data distribution, personalized FL (PFL) was proposed. It is designed to produce a dedicated model for each client. However, the existing PFL solution only focuses on the performance of personalized model, ignoring the performance of global model, which will affect the willingness of new clients to participate. In order to solve this problem, this paper proposes a new PFL solution, a two-stage PFL based on sparse pretraining, which can not only train a sparse personalized model for each client, but also obtain a sparse global model. The whole training process is divided into sparse pretraining and sparse personalized training, which focus on the performance of global model and personalized model respectively. Also, we propose a mask sparse aggregation technique to maintain the sparsity of the global model in the sparse personalized training stage. Experimental results show that compared with existing algorithms, our proposed algorithm can improve the accuracy of the global model while maintaining advanced personalized model accuracy, and has higher communication efficiency. In order to address the current problem of sparse personalized federated learning where the global model is dense and poorly performing, a new solution is proposed in the authors' work. The two-stage training approach allows for a focus on the training of global and personalized models in the early and late stages, respectively. In addition, sparse mask aggregation techniques have been proposed to guarantee the sparsity of the global model. image
We propose a decentralized optimization algorithm that preserves the privacy of agents' cost functions without sacrificing accuracy, termed EFPSN. The algorithm adopts Paillier cryptosystem to construct zero-sum f...
详细信息
We propose a decentralized optimization algorithm that preserves the privacy of agents' cost functions without sacrificing accuracy, termed EFPSN. The algorithm adopts Paillier cryptosystem to construct zero-sum functional perturbations. Then, based on the perturbed cost functions, any existing decentralized optimization algorithm can be utilized to obtain the accurate solution. We theoretically prove that EFPSN is (epsilon, delta)-differentially private and can achieve infinitesimally small epsilon, delta under deliberate parameter settings. Numerical experiments further confirm the effectiveness of the algorithm.
We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewar...
详细信息
We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards. To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information. In particular, we show that, despite restricting each agent's attention to only its.. -hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in... In addition, we show the finite-sample convergence of LPI to the global optimal policy, which explicitly captures the trade-off between optimality and computational complexity in choosing kappa. Numerical simulations demonstrate the effectiveness of LPI.
Exploration of different network topologies is one of the fundamental problems of distributed systems. The problem has been studied on networks like lines, rings, tori, rectangular grids, etc. In this work, we have co...
详细信息
ISBN:
(纸本)9783031744976;9783031744983
Exploration of different network topologies is one of the fundamental problems of distributed systems. The problem has been studied on networks like lines, rings, tori, rectangular grids, etc. In this work, we have considered a rectangle enclosed triangular grid (RETG). A RETG is a part of an infinite triangular grid and the part is enclosed by a rectangle whose one pair of parallel sides aligns with a family of parallel straight lines of the infinite triangular grid. We have studied the problem of perpetual exploration on a RETG using oblivious robots. We have considered the robots with limited visibility i.e. the robots are myopic. Infinite visibility becomes impractical for a very large network. Limited visibility is more practical than infinite visibility. The robots have neither any chirality nor any axis agreement. An algorithm is provided to explore the RETG perpetually without any collision. The algorithm works under a synchronous scheduler. The algorithm requires three robots with two hop visibility.
Graph coloring is often used in parallelizing scientific computations that run in distributed and multi-GPU environments;it identifies sets of independent data that can be updated in parallel. Many algorithms exist fo...
详细信息
ISBN:
(纸本)9781665415576
Graph coloring is often used in parallelizing scientific computations that run in distributed and multi-GPU environments;it identifies sets of independent data that can be updated in parallel. Many algorithms exist for graph coloring on a single GPU or in distributed memory, but hybrid MPI+GPU algorithms have been unexplored until this work, to the best of our knowledge. We present several MPI+GPU coloring approaches that use implementations of the distributed coloring algorithms of Gebremedhin et al. and the shared-memory algorithms of Deveci et al. The on-node parallel coloring uses implementations in KokkosKernels, which provide parallelization for both multicore CPUs and GPUs. We further extend our approaches to solve for distance-2 coloring, giving the first known distributed and multi-GPU algorithm for this problem. In addition, we propose novel methods to reduce communication in distributed graph coloring. Our experiments show that our approaches operate efficiently on inputs too large to fit on a single GPU and scale up to graphs with 76.7 billion edges running on 128 GPUs.
Transfer learning has proved to be effective for building predictive models even in complex conditions with a low amount of available labeled data, by constructing a predictive model for a target domain also using the...
详细信息
Transfer learning has proved to be effective for building predictive models even in complex conditions with a low amount of available labeled data, by constructing a predictive model for a target domain also using the knowledge coming from a separate domain, called source domain. However, several existing transfer learning methods assume identical feature spaces between the source and the target domains. This assumption limits the possible real -world applications of such methods, since two separate, although related, domains could be described by totally different feature spaces. Heterogeneous transfer learning methods aim to overcome this limitation, but they usually i) make other assumptions on the features, such as requiring the same number of features, ii) are not generally able to distribute the workload over multiple computational nodes, iii) cannot work in the Positive-Unlabeled (PU) learning setting, which we also considered in this study, or iv) their applicability is limited to specific application domains, i.e., they are not general-purpose methods. In this manuscript, we present a novel distributed heterogeneous transfer learning method, implemented in Apache Spark, that overcomes all the above-mentioned limitations. Specifically, it is able to work also in the PU learning setting by resorting to a clustering-based approach, and can align totally heterogeneous feature spaces, without exploiting peculiarities of specific application domains. Moreover, our distributed approach allows us to process large source and target datasets. Our experimental evaluation was performed in three different application domains that can benefit from transfer learning approaches, namely the reconstruction of the human gene regulatory network, the prediction of cerebral stroke in hospital patients, and the prediction of customer energy consumption in power grids. The results show that the proposed approach is able to outperform 4 state -of -the -art heterogeneous transfer le
The emergent behavior of a distributed system is conditioned by the information available to the local decision-makers. Therefore, one may expect that providing decision-makers with more information will improve syste...
详细信息
The emergent behavior of a distributed system is conditioned by the information available to the local decision-makers. Therefore, one may expect that providing decision-makers with more information will improve system performance;in this letter, we find that this is not necessarily the case. In multi-agent maximum coverage problems, we find that even when agents' objectives are aligned with the global welfare, informing agents about the realization of the resource's random values can reduce equilibrium performance by a factor of 1/2. This affirms an important aspect of designing distributed systems: information need be shared carefully. We further this understanding by providing lower and upper bounds on the ratio of system welfare when information is (fully or partially) revealed and when it is not, termed the value-of-informing. We then identify a trade-off that emerges when optimizing the performance of the best-case and worst-case equilibrium.
In the dispersion problem, a group of k <= n mobile robots, initially placed on the vertices of an anonymous graph G with n vertices, must redistribute themselves so that each vertex hosts no more than one robot. W...
详细信息
ISBN:
(纸本)9783031814037;9783031814044
In the dispersion problem, a group of k <= n mobile robots, initially placed on the vertices of an anonymous graph G with n vertices, must redistribute themselves so that each vertex hosts no more than one robot. We address this challenge on an anonymous triangular grid graph, where each vertex can connect to up to six adjacent vertices. We propose a distributed deterministic algorithm that achieves dispersion on an unoriented triangular grid graph in O(root n) time, where n is the number of vertices. Each robot requires O(log n) bits of memory. The time complexity of our algorithm and the memory usage per robot are optimal. This work builds on previous studies by Kshemkalyani et al. [WALCOM 2020 [17]] and Banerjee et al. [ALGOWIN 2024 [3]]. Importantly, our algorithm terminates without requiring prior knowledge of n and resolves a question posed by Banerjee et al. [ALGOWIN 2024 [3]].
In prior work, Gupta et al. (SPAA 2022) presented a distributed algorithm for multiplying sparse.. x.. matrices, using.. computers. They assumed that the input matrices are uniformly sparse-there are at most.. non-zer...
详细信息
ISBN:
(纸本)9798400704161
In prior work, Gupta et al. (SPAA 2022) presented a distributed algorithm for multiplying sparse.. x.. matrices, using.. computers. They assumed that the input matrices are uniformly sparse-there are at most.. non-zeros in each row and column-and the task is to compute a uniformly sparse part of the product matrix. Initially each computer knows one row of each input matrix, and eventually each computer needs to know one row of the product matrix. In each communication round each computer can send and receive one O(log n)-bit message. Their algorithm solves this task in O(d(1.907)) rounds, while the trivial bound is O(d(2)). We improve on the prior work in two dimensions: First, we show that we can solve the same task faster, in only O(d(1.832)) rounds. Second, we explore what happens when matrices are not uniformly sparse. We consider the following alternative notions of sparsity: row-sparse matrices (at most.. non-zeros per row), column-sparse matrices, matrices with bounded degeneracy (we can recursively delete a row or column with at most.. non-zeros), average-sparse matrices (at most dn non-zeros in total), and general matrices. We show that we can still compute X = AB in O(d(1.832)) rounds even if one of the three matrices (A, B, or X) is average-sparse instead of uniformly sparse. We present algorithms that handle a much broader range of sparsity in O(d(2) + log n) rounds, and present conditional hardness results that put limits on further improvements and generalizations.
The Hard-state Protocol Independent Multicast-Dense Mode (HPIM-DM) multicast routing protocol is proposed. HPIM-DM is a hard-state version of PIM-DM that overcomes its poor convergence times and lack of resilience to ...
详细信息
The Hard-state Protocol Independent Multicast-Dense Mode (HPIM-DM) multicast routing protocol is proposed. HPIM-DM is a hard-state version of PIM-DM that overcomes its poor convergence times and lack of resilience to replay attacks. Like PIM-DM, HPIM-DM is meant for dense networks and supports its operation on a unicast routing protocol and reverse path forwarding. However, routers maintain sense of the multicast trees at all times, allowing fast reconfiguration in the presence of network failures or unicast route changes. This is achieved by (i) keeping information on all upstream neighbours from which multicast data can be received, (ii) ensuring the reliable transmission and sequencing of control messages, and (iii) synchronizing the routing information immediately when a new router joins the network. The correctness of the protocol was extensively validated using model checking and logical reasoning. The protocol was fully implemented in Python, and the implementation is publicly available. Finally, we show both theoretically and experimentally that HPIM-DM has much better convergence times than PIM-DM.
暂无评论