检索结果-内蒙古大学图书馆

DeFTA: A plug-and-play peer-to-peer decentralized federated learning framework

INFORMATION SCIENCES 2024年 670卷

作者： Zhou, Yuhao Shi, Minjia Tian, Yuxin Ye, Qing Lv, Jiancheng Sichuan Univ Coll Comp Sci 24 South Sect 1Yihuan Rd Chengdu 610064 Sichuan Peoples R China Engn Res Ctr Machine Learning & Ind Intelligence Chengdu 610065 Sichuan Peoples R China

Federated learning (FL) is a pivotal catalyst for enabling large-scale privacy-preserving distributed machine learning (ML). By eliminating the need for local raw dataset sharing, FL substantially reduces privacy concerns and alleviates the isolated data problem. However, in reality, the success of FL is predominantly attributed to a centralized framework called FedAvg [1], in which workers are responsible for model training, and servers are in control of model aggregation. Nevertheless, FedAvg's centralized worker-server architecture has raised new concerns, including low scalability of the cluster, risk of data leakage, and central server failure or even defection. To overcome these challenges, we propose De centralized F ederated T rusted A veraging (DeFTA), a decentralized FL framework that serves as a plug-and-play replacement for FedAvg , bringing instant improvements to security, scalability, and fault-tolerance in the federated learning process. In essence, it primarily consists of a novel model aggregating formula with theoretical performance analysis, and a decentralized trust system (DTS) to significantly enhance system robustness. Extensive experiments conducted on six datasets and six basic models suggest that DeFTA not only exhibits comparable performance with FedAvg in a more realistic setting, but also achieves remarkable resilience even when 67% of workers are malicious.

关键词： distributed computing Federated learning Decentralization Peer-to-peer Framework

来源：评论

学校读者我要写书评

暂无评论

Hierarchical Group Testing for Byzantine Attack Identification in distributed Matrix Multiplication

引用

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS 2022年第3期40卷 1013-1029页

作者： Hong, Sangwoo Yang, Heecheol Lee, Jungwoo Seoul Natl Univ Dept Elect & Comp Engn Commun & Machine Learning Lab Seoul 08826 South Korea Chunguam Natl Univ Div Comp Convergence Distributed Networks & Comp Lab Daejeon 34134 South Korea

Coded computing has proved its efficiency in handling a straggler issue in distributed computing framework. It uses error correcting codes to mitigate the effect of the stragglers. However, in a coded distributed computing framework, there may exist Byzantine workers who send the wrong computation results to a master in order to contaminate the overall computation output. Therefore, it is essential to identify Byzantine workers from their computation results in coded computing. In this paper, we consider Byzantine attack identification problem in coded computing for distributed matrix multiplication tasks. We propose a new coding scheme which facilitates the efficient Byzantine attack identification, namely locally testable codes. We also suggest a hierarchical group testing method for Byzantine attack identification. We claim the required number of tests for group testing in our scheme, and show that it requires smaller number of tests than the conventional group testing method for the existing coded computing schemes.

关键词： Testing Codes distributed computing Task analysis Encoding Decoding Reed-Solomon codes distributed computing Byzantine attack coded computation matrix multiplication group testing

来源：评论

学校读者我要写书评

暂无评论

Secure distributed Matrix Computation With Discrete Fourier Transform

引用

IEEE TRANSACTIONS ON INFORMATION THEORY 2022年第7期68卷 4666-4680页

作者： Mital, Nitish Ling, Cong Gunduz, Deniz Imperial Coll London Dept Elect & Elect Engn London SW7 2BX England

We consider the problem of secure distributed matrix computation (SDMC), where a user queries a function of data matrices generated at distributed source nodes. We assume the availability of N honest but curious computation servers, which are connected to the sources, the user, and each other through orthogonal and reliable communication links. Our goal is to minimize the amount of data that must be transmitted from the sources to the servers, called the upload cost, while guaranteeing that no T colluding servers can learn any information about the source matrices, and the user cannot learn any information beyond the computation result. We first focus on secure distributed matrix multiplication (SDMM), considering two matrices, and propose a novel polynomial coding scheme using the properties of finite field discrete Fourier transform, which achieves an upload cost significantly lower than the existing results in the literature. We then generalize the proposed scheme to include straggler mitigation, and to the multiplication of multiple matrices while keeping the input matrices, the intermediate computation results, as well as the final result secure against any T colluding servers. We also consider a special case, called computation with own data, where the data matrices used for computation belong to the user. In this case, we drop the security requirement against the user, and show that the proposed scheme achieves the minimal upload cost. We then propose methods for performing other common matrix computations securely on distributed servers, including changing the parameters of secret sharing, matrix transpose, matrix exponentiation, solving a linear system, and matrix inversion, which are then used to show how arbitrary matrix polynomials can be computed securely on distributed servers using the proposed procedure.

关键词： Data privacy distributed computing edge computing security information theory

来源：评论

学校读者我要写书评

暂无评论

Designing and prototyping extensions to the Message Passing Interface in MPICH

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE computing APPLICATIONS 2024年第5期38卷 527-545页

作者： Zhou, Hui Raffenetti, Ken Guo, Yanfei Gillis, Thomas Latham, Robert Thakur, Rajeev Argonne Natl Lab Math & Comp Sci 9700 S Cass Ave Lemont IL 60439 USA

As HPC system architectures and the applications running on them continue to evolve, the MPI standard itself must evolve. The trend in current and future HPC systems toward powerful nodes with multiple CPU cores and multiple GPU accelerators makes efficient support for hybrid programming critical for applications to achieve high performance. However, the support for hybrid programming in the MPI standard has not kept up with recent trends. The MPICH implementation of MPI provides a platform for implementing and experimenting with new proposals and extensions to fill this gap and to gain valuable experience and feedback before the MPI Forum can consider them for standardization. In this work, we detail six extensions implemented in MPICH to increase MPI interoperability with other runtimes, with a specific focus on heterogeneous architectures. First, the extension to MPI generalized requests lets applications integrate asynchronous tasks into MPI's progress engine. Second, the iovec extension to datatypes lets applications use MPI datatypes as a general-purpose data layout API beyond just MPI communications. Third, a new MPI object, MPIX_Stream, can be used by applications to identify execution contexts beyond MPI processes, including threads and GPU streams. MPIX stream communicators can be created to make existing MPI functions thread-aware and GPU-aware, thus providing applications with explicit ways to achieve higher performance. Fourth, MPIX Streams are extended to support the enqueue semantics for offloading MPI communications onto a GPU stream context. Fifth, thread communicators allow MPI communicators to be constructed with individual threads, thus providing a new level of interoperability between MPI and on-node runtimes such as OpenMP. Lastly, we present an extension to invoke MPI progress, which lets users spawn progress threads with fine-grained control to adapt the communication performance to their application designs. We describe the design and implem

关键词： distributed computing message passing interface message passing interface plus threads message passing interface plus graphics processing units MPICH

来源：评论

学校读者我要写书评

暂无评论

Network-assisted processing of advanced IoT applications: challenges and proof-of-concept application

引用

CLUSTER computing-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2024年第2期27卷 1849-1865页

作者： Mora, Higinio Pujol, Francisco A. Ramirez, Tamai Jimeno-Morenilla, Antonio Szymanski, Julian Univ Alicante Dept Comp Technol & Computat Carretera San Vicente S-N Alicante 03690 Spain Gdansk Univ Technol Fac Elect Telecommun & Informat Dept Comp Architecture Gabriela Narutowicza 11-12 PL-80233 Gdansk Poland

Recent advances in the area of the Internet of Things shows that devices are usually resource-constrained. To enable advanced applications on these devices, it is necessary to enhance their performance by leveraging external computing resources available in the network. This work presents a study of computational platforms to increase the performance of these devices based on the Mobile Cloud computing (MCC) paradigm. The main contribution of this paper is to research the advantages and possibilities of architectures with multiple offloading options. To this end, a review of architectures that use a combination of the computing layers in the available infrastructure to perform this paradigm and outsource processing load is presented. In addition, a proof-of-concept application is introduced to demonstrate its realization along all the network layers. The results of the simulations confirm the high flexibility to offload numerous tasks using different layers and the ability to overcome unfavorable scenarios.

关键词： Internet of Things Mobile computing Computer networks distributed computing Quality of service Cloud computing

来源：评论

学校读者我要写书评

暂无评论

An open science grid implementation of the steady state genetic algorithm for crystal structure prediction

引用

JOURNAL OF COMPUTATIONAL SCIENCE 2024年 82卷

作者： Varela, Kristal N. Pagola, Gabriel I. Lund, Albert M. Ferraro, Marta B. Orendt, Anita M. Facelli, Julio C. Univ Buenos Aires Fac Ciencias Exactas & Nat Dept Fis Buenos Aires Argentina Inst Fis Buenos Aires IFIBA Buenos Aires Argentina Univ Utah Ctr High Performance Comp Salt Lake City UT USA Univ Utah Dept Biomed Informat Salt Lake City UT 84108 USA

In this paper we report the implementation and testing of algorithmic changes that have been implemented in MGAC, a crystal structure prediction system, to make it scalable and amenable to take advantage of such significant distributed resources as the Open Science Grid (OSG). The changes include the adoption of a steady state Genetic Algorithm (GA) and the adoption of a more general definition of the GA genome that eliminates the need of searching individually for each of the 230 possible space groups and the use of the Density Functional Theory with dispersion correction (DFT-D) as implemented in Quantum Espresso (QE) to calculate crystal energies. The performance of this implementation of MGAC, which in the following we label as MGAC-QE-OSG, is demonstrated for two test cases methanol and ethanol. In both cases the MGAC-QE-OSG can find the experimental structures of these compounds.

关键词： Genetic algorithms Crystal structure prediction Open science grid distributed computing

来源：评论

学校读者我要写书评

暂无评论

Analysis of workflow schedulers in simulated distributed environments

引用

JOURNAL OF SUPERcomputing 2022年第13期78卷 15154-15180页

作者： Beranek, Jakub Bohm, Stanislav Cima, Vojtech VSB Tech Univ Ostrava IT4Innovat Ostrava Czech Republic

Task graphs provide a simple way to describe scientific workflows (sets of tasks with dependencies) that can be executed on both HPC clusters and in the cloud. An important aspect of executing such graphs is the used scheduling algorithm. Many scheduling heuristics have been proposed in existing works;nevertheless, they are often tested in oversimplified environments. We provide an extensible simulation environment designed for prototyping and benchmarking task schedulers, which contains implementations of various scheduling algorithms and is open-sourced, in order to be fully reproducible. We use this environment to perform a comprehensive analysis of workflow scheduling algorithms with a focus on quantifying the effect of scheduling challenges that have so far been mostly neglected, such as delays between scheduler invocations or partially unknown task durations. Our results indicate that network models used by many previous works might produce results that are off by an order of magnitude in comparison to a more realistic model. Additionally, we show that certain implementation details of scheduling algorithms which are often neglected can have a large effect on the scheduler's performance, and they should thus be described in great detail to enable proper evaluation.

关键词： distributed computing DAG scheduling Task Scheduling Network models

来源：评论

学校读者我要写书评

暂无评论

Synchronize Only the Immature Parameters: Communication-Efficient Federated Learning By Freezing Parameters Adaptively

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 2024年第7期35卷 1155-1173页

作者： Chen, Chen Xu, Hong Wang, Wei Li, Baochun Li, Bo Chen, Li Zhang, Gong Shanghai Jiao Tong Univ John Hopcroft Ctr Comp Sci Shanghai 200240 Peoples R China Chinese Univ Hong Kong Dept Comp Sci & Engn Cent Ave Hong Kong Peoples R China Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Clear Water Bay Hong Kong Peoples R China Univ Toronto Dept Elect & Comp Engn Toronto ON Canada Zhongguancun Lab Beijing 100081 Peoples R China Huawei Hong Kong Res Ctr Theory Lab Hong Kong Peoples R China

Federated learning allows edge devices to collaboratively train a global model without sharing their local private data. Yet, with limited network bandwidth at the edge, communication often becomes a severe bottleneck. In this article, we find that it is unnecessary to always synchronize the full model in the entire training process, because many parameters already become mature (i.e., stable) prior to model convergence, and can thus be excluded from later synchronizations. This allows us to reduce the communication overhead without compromising the model accuracy. However, challenges are that the local parameters excluded from global synchronization may diverge on different clients, and meanwhile some parameters may stabilize only temporally. To address these challenges, we propose a novel scheme called Adaptive Parameter Freezing (APF), which fixes (freezes) the non-synchronized stable parameters in intermittent periods. Specifically, the freezing periods are tentatively adjusted in an additively-increase and multiplicatively-decrease manner-depending on whether the previously-frozen parameters remain stable in subsequent iterations. We also extend APF into APF# and APF++, which freeze parameters in a more aggressive manner to achieve larger performance benefit for large complex models. We implemented APF and its variants as Python modules with PyTorch, and extensive experiments show that APF can reduce data transfer amount by over $60\%$60%.

关键词： Training Convergence Servers Synchronization Quantization (signal) Federated learning Data models distributed computing federated learning machine learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Priority-Based Selection of Individuals in Memetic Algorithms for distributed Data-Intensive Web Service Compositions

引用

IEEE TRANSACTIONS ON SERVICES computing 2022年第5期15卷 2939-2953页

作者： Sadeghiram, Soheila Ma, Hui Chen, Gang Victoria Univ Wellington Sch Engn & Comp Sci Wellington 6012 New Zealand

In distributed computing, Web Service Composition (WSC) leads to the effective reuse of existing services and produces added value. WSC must fulfil functional requirements and optimise Quality of Service (QoS) attributes, simultaneously. Memetic Algorithms (MAs) are promising for automatically composing numerous Web services to satisfy the above requirements. Data-intensive Web services focus on providing and updating data with a significant volume of data operation and exchange. However, current composition approaches have ignored the impact of data communication and the distribution of services, which significantly affect the performance when applied to the challenging distributed Data-intensive Web Service Composition (DDWSC) problem. Although recent approaches have revealed the usefulness of local search, they have completely overlooked the question of preferring appropriate composition solutions for local search. To address this research issue, we propose a priority-based selection method for the local search that can be consistently integrated with any MA for DDWSC. This enables us to develop state-of-the-art algorithms for DDWSC by explicitly considering the problem-specific, population and solution-related information for choosing a solution. Extensive experimental evaluation using benchmark datasets shows that our proposed method significantly outperforms several recently proposed methods.

关键词： Web services Quality of service Statistics Sociology Memetics Search problems Genetic algorithms Web services distributed computing memetic algorithms priority-based selection data-intensive Web service composition

来源：评论

学校读者我要写书评

暂无评论

distributed Density Peaks Clustering Revisited

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2022年第8期34卷 3714-3726页

作者： Lu, Jing Zhao, Yuhai Tan, Kian-Lee Wang, Zhengkui Northeastern Univ Sch Comp Sci & Engn Shenyang 110819 Peoples R China Natl Univ Singapore Sch Comp Singapore 117417 Singapore Singapore Inst Technol InfoComm Technol Cluster Singapore 138683 Singapore

Density Peaks (DP) Clustering organizes data into clusters by finding peaks in dense regions. This involves computing density (rho) and distance (delta) of every point. As such, though DP has been very effective in producing high quality clusters, their complexity is O(N-2) where N is the number of data points. In this paper, we propose a fast distributed density peaks clustering algorithm, FDDP, based on the z-value index. In FDDP, we first employ the z-value index to map multi-dimensional data points into one dimensional space, and then range-partition the data according to the z-value to balance the load across the processing nodes. We ensure minimal overlapping range to handle computations at the boundary points. We also propose FC, an efficient algorithm that employs a forward computing strategy to calculate r linearly. Additionally, we propose another algorithm, CB, which uses a caching and efficient searching strategy to compute delta. Moreover, FDDP is able to reduce the time complexity from O(N-2) to O(N . log(N)). We provide a theoretical analysis of FDDP and evaluated FDDP empirically. Our experimental results show that FDDP outperforms the state-of-the-art algorithms significantly.

关键词： Clustering distributed computing z-order curve density peaks clustering

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：