检索结果-内蒙古大学图书馆

Two-stage personalized federated learning based on sparse pretraining

ELECTRONICS LETTERS 2023年第17期59卷 e12943-e12943页

作者： Liu, Tong Xie, Kaixuan Kong, Yi Chen, Guojun Xu, Yinfei Xin, Lun Yu, Fei Southeast Univ Nanjing Peoples R China China Mobile Res Inst Beijing Peoples R China

Aiming at solving the performance degradation of federated learning (FL) under heterogeneous data distribution, personalized FL (PFL) was proposed. It is designed to produce a dedicated model for each client. However, the existing PFL solution only focuses on the performance of personalized model, ignoring the performance of global model, which will affect the willingness of new clients to participate. In order to solve this problem, this paper proposes a new PFL solution, a two-stage PFL based on sparse pretraining, which can not only train a sparse personalized model for each client, but also obtain a sparse global model. The whole training process is divided into sparse pretraining and sparse personalized training, which focus on the performance of global model and personalized model respectively. Also, we propose a mask sparse aggregation technique to maintain the sparsity of the global model in the sparse personalized training stage. Experimental results show that compared with existing algorithms, our proposed algorithm can improve the accuracy of the global model while maintaining advanced personalized model accuracy, and has higher communication efficiency. In order to address the current problem of sparse personalized federated learning where the global model is dense and poorly performing, a new solution is proposed in the authors' work. The two-stage training approach allows for a focus on the training of global and personalized models in the early and late stages, respectively. In addition, sparse mask aggregation techniques have been proposed to guarantee the sparsity of the global model. image

关键词： artificial intelligence convolutional neural nets data protection distributed algorithms image classification information and communications neural nets

来源：评论

学校读者我要写书评

暂无评论

Private and Accurate Decentralized Optimization via Encrypted and Structured Functional Perturbation

引用

IEEE CONTROL SYSTEMS LETTERS 2023年 7卷 1339-1344页

作者： Zhou, Yijie Pu, Shi Chinese Univ Hong Kong Shenzhen Sch Data Sci Shenzhen 518172 Peoples R China Chinese Univ Hong Kong Shenzhen Shenzhen Res Inst Big Data Sch Data Sci Shenzhen 518172 Peoples R China

We propose a decentralized optimization algorithm that preserves the privacy of agents' cost functions without sacrificing accuracy, termed EFPSN. The algorithm adopts Paillier cryptosystem to construct zero-sum functional perturbations. Then, based on the perturbed cost functions, any existing decentralized optimization algorithm can be utilized to obtain the accurate solution. We theoretically prove that EFPSN is (epsilon, delta)-differentially private and can achieve infinitesimally small epsilon, delta under deliberate parameter settings. Numerical experiments further confirm the effectiveness of the algorithm.

关键词： Privacy Encryption Perturbation methods Cost function Hilbert space Machine learning Linear programming Optimization methods distributed algorithms differential privacy

来源：评论

学校读者我要写书评

暂无评论

Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning

引用

PROCEEDINGS OF THE ACM ON MEASUREMENT AND ANALYSIS OF COMPUTING SYSTEMS 2023年第1期7卷 1-51页

作者： Zhang, Yizhou Qu, Guannan Xu, Pan Lin, Yiheng Chen, Zaiwei Wierman, Adam Tsinghua Univ Beijing Peoples R China Carnegie Mellon Univ Pittsburgh PA USA Duke Univ Durham NC USA CALTECH Pasadena CA USA

We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of their entropy-regularized long-term rewards. To overcome the curse of dimensionality and to reduce communication, we propose a Localized Policy Iteration (LPI) algorithm that provably learns a near-globally-optimal policy using only local information. In particular, we show that, despite restricting each agent's attention to only its.. -hop neighborhood, the agents are able to learn a policy with an optimality gap that decays polynomially in... In addition, we show the finite-sample convergence of LPI to the global optimal policy, which explicitly captures the trade-off between optimality and computational complexity in choosing kappa. Numerical simulations demonstrate the effectiveness of LPI.

关键词： Multi-agent reinforcement learning networked systems machine learning distributed algorithms

来源：评论

学校读者我要写书评

暂无评论

Brief Announcement: Perpetual Exploration of Triangular Grid by Myopic Oblivious Robots Without Chirality 26th

Brief Announcement: Perpetual Exploration of Triangular Grid...

引用

26th International Symposium on Stabilization, Safety, and Security of distributed Systems

作者： Das, Raja Goswami, Pritam Sau, Buddhadeb Jadavpur Univ Kolkata India

ISBN: (纸本)9783031744976;9783031744983

Exploration of different network topologies is one of the fundamental problems of distributed systems. The problem has been studied on networks like lines, rings, tori, rectangular grids, etc. In this work, we have considered a rectangle enclosed triangular grid (RETG). A RETG is a part of an infinite triangular grid and the part is enclosed by a rectangle whose one pair of parallel sides aligns with a family of parallel straight lines of the infinite triangular grid. We have studied the problem of perpetual exploration on a RETG using oblivious robots. We have considered the robots with limited visibility i.e. the robots are myopic. Infinite visibility becomes impractical for a very large network. Limited visibility is more practical than infinite visibility. The robots have neither any chirality nor any axis agreement. An algorithm is provided to explore the RETG perpetually without any collision. The algorithm works under a synchronous scheduler. The algorithm requires three robots with two hop visibility.

关键词： Myopic robot Autonomous robots Oblivious robot Triangular grid Perpetual exploration distributed algorithms

来源：评论

学校读者我要写书评

暂无评论

distributed Memory Graph Coloring algorithms for Multiple GPUs 10

Distributed Memory Graph Coloring Algorithms for Multiple GP...

引用

10th IEEE/ACM Workshop on Irregular Applications - Architectures and algorithms (IA3)

作者： Bogle, Ian Boman, Erik G. Devine, Karen Rajamanickam, Sivasankaran Slota, George M. Rensselaer Polytech Inst Troy NY 12181 USA Sandia Natl Labs POB 5800 Albuquerque NM 87185 USA

ISBN: (纸本)9781665415576

Graph coloring is often used in parallelizing scientific computations that run in distributed and multi-GPU environments;it identifies sets of independent data that can be updated in parallel. Many algorithms exist for graph coloring on a single GPU or in distributed memory, but hybrid MPI+GPU algorithms have been unexplored until this work, to the best of our knowledge. We present several MPI+GPU coloring approaches that use implementations of the distributed coloring algorithms of Gebremedhin et al. and the shared-memory algorithms of Deveci et al. The on-node parallel coloring uses implementations in KokkosKernels, which provide parallelization for both multicore CPUs and GPUs. We further extend our approaches to solve for distance-2 coloring, giving the first known distributed and multi-GPU algorithm for this problem. In addition, we propose novel methods to reduce communication in distributed graph coloring. Our experiments show that our approaches operate efficiently on inputs too large to fit on a single GPU and scale up to graphs with 76.7 billion edges running on 128 GPUs.

关键词： graph coloring distributed algorithms GPU

来源：评论

学校读者我要写书评

暂无评论

distributed Heterogeneous Transfer Learning

引用

BIG DATA RESEARCH 2024年 37卷

作者： Mignone, Paolo Pio, Gianvito Ceci, Michelangelo Univ Bari Aldo Moro Dept Comp Sci Bari Italy Natl Interuniv Consortium Informat CINI Big Data Lab Rome Italy Jozef Stefan Inst Dept Knowledge Technol Ljubljana Slovenia

Transfer learning has proved to be effective for building predictive models even in complex conditions with a low amount of available labeled data, by constructing a predictive model for a target domain also using the knowledge coming from a separate domain, called source domain. However, several existing transfer learning methods assume identical feature spaces between the source and the target domains. This assumption limits the possible real -world applications of such methods, since two separate, although related, domains could be described by totally different feature spaces. Heterogeneous transfer learning methods aim to overcome this limitation, but they usually i) make other assumptions on the features, such as requiring the same number of features, ii) are not generally able to distribute the workload over multiple computational nodes, iii) cannot work in the Positive-Unlabeled (PU) learning setting, which we also considered in this study, or iv) their applicability is limited to specific application domains, i.e., they are not general-purpose methods. In this manuscript, we present a novel distributed heterogeneous transfer learning method, implemented in Apache Spark, that overcomes all the above-mentioned limitations. Specifically, it is able to work also in the PU learning setting by resorting to a clustering-based approach, and can align totally heterogeneous feature spaces, without exploiting peculiarities of specific application domains. Moreover, our distributed approach allows us to process large source and target datasets. Our experimental evaluation was performed in three different application domains that can benefit from transfer learning approaches, namely the reconstruction of the human gene regulatory network, the prediction of cerebral stroke in hospital patients, and the prediction of customer energy consumption in power grids. The results show that the proposed approach is able to outperform 4 state -of -the -art heterogeneous transfer le

关键词： Heterogeneous transfer learning distributed algorithms Apache spark Cerebral stroke prediction Energy consumption prediction Gene regulatory network reconstruction

来源：评论

学校读者我要写书评

暂无评论

The Cost of Informing Decision-Makers in Multi-Agent Maximum Coverage Problems With Random Resource Values

引用

IEEE CONTROL SYSTEMS LETTERS 2023年 7卷 2928-2933页

作者： Ferguson, Bryce L. Paccagnan, Dario Marden, Jason R. Univ Calif Santa Barbara Dept Elect & Comp Engn Santa Barbara CA 93106 USA Imperial Coll London Dept Comp London SW7 2AZ England

The emergent behavior of a distributed system is conditioned by the information available to the local decision-makers. Therefore, one may expect that providing decision-makers with more information will improve system performance;in this letter, we find that this is not necessarily the case. In multi-agent maximum coverage problems, we find that even when agents' objectives are aligned with the global welfare, informing agents about the realization of the resource's random values can reduce equilibrium performance by a factor of 1/2. This affirms an important aspect of designing distributed systems: information need be shared carefully. We further this understanding by providing lower and upper bounds on the ratio of system welfare when information is (fully or partially) revealed and when it is not, termed the value-of-informing. We then identify a trade-off that emerges when optimizing the performance of the best-case and worst-case equilibrium.

关键词： Game theory agent-based systems uncertain systems distributed algorithms

来源：评论

学校读者我要写书评

暂无评论

Optimal Dispersion in Triangular Grids: Achieving Efficiency Without Prior Knowledge 21st

Optimal Dispersion in Triangular Grids: Achieving Efficiency...

引用

21st International Conference on distributed Computing and Intelligent Technology

作者： Pandit, Himani Pandit, Supantha Dhirubhai Ambani Inst Informat & Commun Technol Gandhinagar Gujarat India

ISBN: (纸本)9783031814037;9783031814044

In the dispersion problem, a group of k <= n mobile robots, initially placed on the vertices of an anonymous graph G with n vertices, must redistribute themselves so that each vertex hosts no more than one robot. We address this challenge on an anonymous triangular grid graph, where each vertex can connect to up to six adjacent vertices. We propose a distributed deterministic algorithm that achieves dispersion on an unoriented triangular grid graph in O(root n) time, where n is the number of vertices. Each robot requires O(log n) bits of memory. The time complexity of our algorithm and the memory usage per robot are optimal. This work builds on previous studies by Kshemkalyani et al. [WALCOM 2020 [17]] and Banerjee et al. [ALGOWIN 2024 [3]]. Importantly, our algorithm terminates without requiring prior knowledge of n and resolves a question posed by Banerjee et al. [ALGOWIN 2024 [3]].

关键词： distributed algorithms Mobile robots Dispersion Triangular grid Deterministic algorithms

来源：评论

学校读者我要写书评

暂无评论

Brief Announcement: Low-Bandwidth Matrix Multiplication: Faster algorithms and More General Forms of Sparsity 24

Brief Announcement: Low-Bandwidth Matrix Multiplication: Fas...

引用

36th ACM Symposium on Parallelism in algorithms and Architectures (SPAA)

作者： Gupta, Chetan Korhonen, Janne H. Studeny, Jan Suomela, Jukka Vahidi, Hossein IIT Roorkee Roorkee Uttar Pradesh India Aalto Univ Espoo Finland

ISBN: (纸本)9798400704161

In prior work, Gupta et al. (SPAA 2022) presented a distributed algorithm for multiplying sparse.. x.. matrices, using.. computers. They assumed that the input matrices are uniformly sparse-there are at most.. non-zeros in each row and column-and the task is to compute a uniformly sparse part of the product matrix. Initially each computer knows one row of each input matrix, and eventually each computer needs to know one row of the product matrix. In each communication round each computer can send and receive one O(log n)-bit message. Their algorithm solves this task in O(d(1.907)) rounds, while the trivial bound is O(d(2)). We improve on the prior work in two dimensions: First, we show that we can solve the same task faster, in only O(d(1.832)) rounds. Second, we explore what happens when matrices are not uniformly sparse. We consider the following alternative notions of sparsity: row-sparse matrices (at most.. non-zeros per row), column-sparse matrices, matrices with bounded degeneracy (we can recursively delete a row or column with at most.. non-zeros), average-sparse matrices (at most dn non-zeros in total), and general matrices. We show that we can still compute X = AB in O(d(1.832)) rounds even if one of the three matrices (A, B, or X) is average-sparse instead of uniformly sparse. We present algorithms that handle a much broader range of sparsity in O(d(2) + log n) rounds, and present conditional hardness results that put limits on further improvements and generalizations.

关键词： distributed algorithms low-bandwidth model matrix multiplication

来源：评论

学校读者我要写书评

暂无评论

HPIM-DM: A fast and reliable dense-mode multicast routing protocol

引用

IET NETWORKS 2023年第3期12卷 122-151页

作者： Oliveira, Pedro Silva, Alexandre Valadas, Rui Univ Lisbon Inst Telecomunicacoes Piso 10Av Rovisco Pais 1 P-1049001 Lisbon Portugal Univ Lisbon Inst Supe Tecn Piso 10Av Rovisco Pais 1 P-1049001 Lisbon Portugal

The Hard-state Protocol Independent Multicast-Dense Mode (HPIM-DM) multicast routing protocol is proposed. HPIM-DM is a hard-state version of PIM-DM that overcomes its poor convergence times and lack of resilience to replay attacks. Like PIM-DM, HPIM-DM is meant for dense networks and supports its operation on a unicast routing protocol and reverse path forwarding. However, routers maintain sense of the multicast trees at all times, allowing fast reconfiguration in the presence of network failures or unicast route changes. This is achieved by (i) keeping information on all upstream neighbours from which multicast data can be received, (ii) ensuring the reliable transmission and sequencing of control messages, and (iii) synchronizing the routing information immediately when a new router joins the network. The correctness of the protocol was extensively validated using model checking and logical reasoning. The protocol was fully implemented in Python, and the implementation is publicly available. Finally, we show both theoretically and experimentally that HPIM-DM has much better convergence times than PIM-DM.

关键词： distributed algorithms fault tolerance multicast communication routing protocols

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：