检索结果-内蒙古大学图书馆

Efficient asynchronous Multi-Participant Vertical Federated Learning

IEEE TRANSACTIONS ON BIG DATA 2024年第6期10卷 940-952页

作者： Shi, Haoran Xu, Yonghui Jiang, Yali Yu, Han Cui, Lizhen Shandong Univ Sch Software Jinan 250100 Peoples R China Shandong Univ Joint SDU NTU Ctr Artificial Intelligence Res C FA Jinan 250100 Peoples R China China Singapore Int Joint Res Inst Guangzhou 510000 Peoples R China Nanyang Technol Univ Sch Comp Sci & Engn Singapore 639798 Singapore

Vertical Federated Learning (VFL) is a private-preserving distributed machine learning paradigm that collaboratively trains machine learning models with participants whose local data overlap largely in the sample space, but not so in the feature space. Existing VFL methods are mainly based on synchronous computation and homomorphic encryption (HE). Due to the differences in the communication and computation resources of the participants, straggling participants can cause delays during synchronous VFL model training, resulting in low computational efficiency. In addition, HE incurs high computation and communication costs. Moreover, it is difficult to establish a VFL coordinator (a.k.a. server) that all participants can trust. To address these problems, we propose an efficient asynchronous Multi-participant Vertical Federated Learning method (AMVFL). AMVFL leverages asynchronous training which reduces waiting time. At the same time, secret sharing is used instead of HE for privacy protection, which further reduces the computational cost. In addition, AMVFL does not require a trusted entity to serve as the VFL coordinator. Experimental results based on real-world and synthetic datasets demonstrate that AMVFL can significantly reduce computational cost and improve the accuracy of the model compared to five state-of-the-art VFL methods.

关键词： computational modeling Stochastic processes Training Data models Collaborative work Privacy Servers Federated learning privacy-preserving asynchronous distributed computation

来源：评论

学校读者我要写书评

暂无评论

Privacy-Preserving asynchronous Vertical Federated Learning Algorithms for Multiparty Collaborative Learning

引用

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022年第11期33卷 6103-6115页

作者： Gu, Bin Xu, An Huo, Zhouyuan Deng, Cheng Huang, Heng Mohamed bin Zayed Univ Artificial Intelligence Dept Machine Learning Abu Dhabi U Arab Emirates JD Finance Amer Corp Mountain View CA 94043 USA Univ Pittsburgh Dept Elect & Comp Engn Pittsburgh PA 15260 USA Xidian Univ Sch Elect Engn Xian 710071 Peoples R China

The privacy-preserving federated learning for vertically partitioned (VP) data has shown promising results as the solution of the emerging multiparty joint modeling application, in which the data holders (such as government branches, private finance, and e-business companies) collaborate throughout the learning process rather than relying on a trusted third party to hold data. However, most of the existing federated learning algorithms for VP data are limited to synchronous computation. To improve the efficiency when the unbalanced computation/communication resources are common among the parties in the federated learning system, it is essential to develop asynchronous training algorithms for VP data while keeping the data privacy. In this article, we propose an asynchronous federated stochastic gradient descent (AFSGD-VP) algorithm and its two variance reduction variants, including stochastic variance reduced gradient (SVRG) and SAGA on the VP data. Moreover, we provide the convergence analyses of AFSGD-VP and its SVRG and SAGA variants under the condition of strong convexity and without any restrictions of staleness. We also discuss their model privacy, data privacy, computational complexities, and communication costs. To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for VP data with theoretical guarantees. Extensive experimental results on a variety of VP datasets not only verify the theoretical results of AFSGD-VP and its SVRG and SAGA variants but also show that our algorithms have much higher efficiency than the corresponding synchronous algorithms.

关键词： Stochastic processes Collaborative work Convergence Partitioning algorithms distributed databases Data privacy Data models asynchronous distributed computation privacy-preserving stochastic gradient descent (SGD) vertical federated learning

来源：评论

学校读者我要写书评

暂无评论

A Proactive Data-Parallel Framework for Machine Learning 8

A Proactive Data-Parallel Framework for Machine Learning

引用

8th IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT)

作者： Zhao, Guoyi Zhou, Tian Gao, Lixin Univ Massachusetts Dept Elect & Comp Engn Amherst MA 01003 USA

ISBN: (纸本)9781450391641

Data parallel frameworks become essential for training machine learning models. The classic Bulk Synchronous Parallel (BSP) model updates the model parameters through pre-defined synchronization barriers. However, when a worker computes significantly slower than other workers, waiting for the slow worker will lead to excessive waste of computing resources. In this paper, we propose a novel proactive data-parallel (PDP) framework. PDP enables the parameter server to initiate the update of the model parameter. That is, we can perform the update at any time without pre-defined update points. PDP not only initiates the update but also determines when to update. The global decision on the frequency of updates will accelerate the training. We further propose asynchronous PDP to reduce the idle time caused by synchronizing parameter updates. We theoretically prove the convergence property of asynchronous PDP. We implement a distributed PDP framework and evaluate PDP with several popular machine learning algorithms including Multilayer Perceptron, Convolutional Neural Network, K-means, and Gaussian Mixture Model. Our evaluation shows that PDP can achieve up to 20X speedup over the BSP model and scale to large clusters.

关键词： machine learning asynchronous distributed computation stragglers gradient descent expectation-maximization

来源：评论

学校读者我要写书评

暂无评论

A Proactive Data-Parallel Framework for Machine Learning 21

A Proactive Data-Parallel Framework for Machine Learning

引用

2021 IEEE/ACM 8th International Conference on Big Data Computing, Applications and Technologies (BDCAT '21)

作者： Guoyi Zhao Tian Zhou Lixin Gao University of Massachusetts Amherst USA University of Massachusetts Amherst USA

ISBN: (纸本)9781450391641

关键词： stragglers machine learning gradient descent expectation-maximization asynchronous distributed computation

来源：评论

学校读者我要写书评

暂无评论

Generalized irreducibility of consensus and the equivalence of t-resilient and wait-free implementations of consensus

引用

SIAM JOURNAL ON COMPUTING 2004年第2期34卷 333-357页

作者： Chandra, T Hadzilacos, V Jayanti, P Toueg, S IBM Corp Thomas J Watson Res Ctr Yorktown Hts NY 10598 USA Univ Toronto Dept Comp Sci Toronto ON MSS 3H5 Canada Dartmouth Coll Dept Comp Sci Hanover NH 03755 USA

We study the consensus problem, which requires multiple processes with different input values to agree on one of these values, in the context of asynchronous shared memory systems. Prior research focussed either on t-resilient solutions of this problem (which must be correct even if up to t processes crash) or on wait-free solutions (which must be correct despite the crash of any number of processes). In this paper, we show that these two forms of solvability are closely related. Specifically, for all n > t >= 2 and all sets S of shared object types (that include simple read/write registers), there is a t-resilient solution to n-process consensus using objects of types in S if and only if there is a wait-free solution to (t+1)-process consensus using objects of types in S. Our proof of this equivalence uses another result derived in this paper, which is of independent interest. Roughly speaking, this result states that a wait-free solution to (n-1)-process consensus is never necessary in designing a wait-free solution to n-process consensus, regardless of the types of objects available. More precisely, for all n >= 2 and all sets S of shared object types (that include simple read/write registers), if there is a wait-free solution to n-process consensus that uses a wait-free solution to (n-1)-process consensus and objects of types in S, then there is a wait-free solution to n-process consensus that uses only objects of types in S.

关键词： asynchronous distributed computation consensus wait-free algorithms fault tolerant algorithms impossibility results

来源：评论

学校读者我要写书评

暂无评论

Efficient causality-tracking timestamping

引用

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 2003年第5期15卷 1239-1250页

作者： Hélary, JM Raynal, M Melideo, G Baldoni, R Inst Rech Informat & Syst Aleatoires F-35042 Rennes France Univ Aquila Dept Comp Sci I-67070 Laquila Italy Univ Roma La Sapienza DIS I-00198 Rome Italy

Vector clocks are the appropriate mechanism used to track causality among the events produced by a distributed computation. Traditional implementations of vector clocks require application messages to piggyback a vector of n integers (where n is the number of processes). This paper investigates the tracking of the causality relation on a subset of events (namely, the events that are defined as "relevant" from the application point of view) in a context where communication channels are not required to be FIFO, and where there is no a priori information on the connectivity of the communication graph or the communication pattern. More specifically, the paper proposes a suite of simple and efficient implementations of vector clocks that address the reduction of the size of message timestamps, i.e., they do their best to have message timestamps whose size is less than n. The relevance of such a suite of protocols is twofold. From a practical side, it constitutes the core of an adaptive timestamping software layer that can used by underlying applications. From a theoretical side, it provides a comprehensive view that helps better understand distributed causality-tracking mechanisms.

关键词： asynchronous distributed computation causality message-passing timestamp vector clock

来源：评论

学校读者我要写书评

暂无评论

A note on the determination of the immediate predecessors in a distributed computation

引用

International Journal of Foundations of Computer Science 2002年第6期13.0卷 865-872页

作者： Anceaume, Emmanuelle Helary, Jean-Michel Raynal, Michel IRISA Campus de Beaulieu 35042 Rennes Cedex France

A distributed computation can be modeled as a partially ordered set (poset) of relevant events (the relevant events are the subset of the primitive events that are meaningful for an observer). This short note presents a general protocol that, when superimposed on a distributed computation, provides each relevant event with a timestamp that identifies exactly its immediate predecessors in the poset. This determination is done on the fly and without using additional control messages. So, the proposed protocol provides an on the fly computation of the Hasse diagram (transitive reduction) of the event graph produced by a distributed computation. The protocol is particularly simple and efficient. It is based on a general condition that allows application messages to piggyback control information whose size can be smaller than n (the number of processes). Interestingly, when one is not interested in tracking the immediate predecessors, the proposed protocol can be simplified to get an efficient vector clock protocol that does not require particular channel assumptions. © 2002 World Scientific Publishing Company.

关键词： asynchronous distributed computation Causality Tracking Hasse Diagram Immediate Predecessor Message Passing Timestamp Vector Clock

来源：评论

学校读者我要写书评

暂无评论

Computing global functions in asynchronous distributed systems with perfect failure detectors

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 2000年第9期11卷 897-909页

作者： Hélary, JM Hurfin, M Mostefaoui, A Raynal, M Tronel, F Inst Rech Informat & Syst Aleatoires F-35042 Rennes France

A Global Data is a vector with one entry per process. Each entry must be filled with an appropriate value provided by the corresponding process. Several distributed computing problems amount to compute a function on a global data. This paper proposes a protocol to solve such problems in the context of asynchronous distributed systems where processes may fail by crashing. The main problem that has to be solved lies in computing the global data and in providing each noncrashed process with a copy of it, despite the possible crash of some processes. To be consistent, the global data must contain, at least, all the values provided by the processes that do not crash. This defines the Global Data computation (GDC) problem. To solve this problem, processes execute a sequence of asynchronous rounds during which they construct, in a decentralized way, the value of the global data and eventually each process gets a copy of it. To cope with process crashes, the protocol uses a perfect failure detector. The proposed protocol has been designed to be time efficient: it allows early decision. Let t be the maximum number of processes that may crash, t < n where n is the total number of processes, and f be the actual number of process crashes (f t). In the worst case, the protocol terminates in min(2f + 2, t + 1) rounds. Moreover, the protocol does not require processes to exchange information on their perception of crashes. The message size depends only on the size of the global data.

关键词： asynchronous distributed computation global data global function computation perfect failure detector problem reduction process crash

来源：评论

学校读者我要写书评

暂无评论

Three-processor tasks are undecidable

引用

SIAM JOURNAL ON COMPUTING 1999年第3期28卷 970-983页

作者： Gafni, E Koutsoupias, E Univ Calif Los Angeles Dept Comp Sci Los Angeles CA 90095 USA

We show that no algorithm exists for deciding whether a finite task for three or more processors is wait-free solvable in the asynchronous read-write shared-memory model. This impossibility result implies that there is no constructive (recursive) characterization of wait-free solvable tasks. It also applies to other shared-memory models of distributed computing, such as the comparison-based model.

关键词： asynchronous distributed computation task-solvability wait-free computation contractibility problem

来源：评论

学校读者我要写书评

暂无评论

Communication-induced determination of consistent snapshots

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 1999年第9期10卷 865-877页

作者： Hélary, JM Mostefaoui, A Raynal, M Inst Rech Informat & Syst Aleatoires F-35042 Rennes France

A classical way to determine consistent snapshots consists in using Chandy-Lamport's algorithm. This algorithm relies on specific control messages that allow processes to synchronize local checkpoint determination and message recording in order for the resulting snapshot to be consistent. This paper investigates a communication-induced approach to determine consistent snapshots. In such an approach, control information is carried out by application messages. Two abstract necessary and sufficient conditions are stated: one associated with global checkpoint consistency, the other associated with message recording. A general protocol is derived from these abstract conditions. Actually, this general protocol can be instantiated in distinct ways, giving rise to a family of communication-induced snapshot protocols. This general protocol shows there is an intrinsic trade-off between the number of forced checkpoints and the number of recorded messages. Finally, a particular instantiation of the general protocol is provided.

关键词： asynchronous distributed computation checkpointing communication-induced protocol consistency global checkpoint message recording snapshot

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：