检索结果-内蒙古大学图书馆

Synchronize Only the Immature Parameters: Communication-Efficient Federated Learning By Freezing Parameters Adaptively

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 2024年第7期35卷 1155-1173页

作者： Chen, Chen Xu, Hong Wang, Wei Li, Baochun Li, Bo Chen, Li Zhang, Gong Shanghai Jiao Tong Univ John Hopcroft Ctr Comp Sci Shanghai 200240 Peoples R China Chinese Univ Hong Kong Dept Comp Sci & Engn Cent Ave Hong Kong Peoples R China Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Clear Water Bay Hong Kong Peoples R China Univ Toronto Dept Elect & Comp Engn Toronto ON Canada Zhongguancun Lab Beijing 100081 Peoples R China Huawei Hong Kong Res Ctr Theory Lab Hong Kong Peoples R China

Federated learning allows edge devices to collaboratively train a global model without sharing their local private data. Yet, with limited network bandwidth at the edge, communication often becomes a severe bottleneck. In this article, we find that it is unnecessary to always synchronize the full model in the entire training process, because many parameters already become mature (i.e., stable) prior to model convergence, and can thus be excluded from later synchronizations. This allows us to reduce the communication overhead without compromising the model accuracy. However, challenges are that the local parameters excluded from global synchronization may diverge on different clients, and meanwhile some parameters may stabilize only temporally. To address these challenges, we propose a novel scheme called Adaptive Parameter Freezing (APF), which fixes (freezes) the non-synchronized stable parameters in intermittent periods. Specifically, the freezing periods are tentatively adjusted in an additively-increase and multiplicatively-decrease manner-depending on whether the previously-frozen parameters remain stable in subsequent iterations. We also extend APF into APF# and APF++, which freeze parameters in a more aggressive manner to achieve larger performance benefit for large complex models. We implemented APF and its variants as Python modules with PyTorch, and extensive experiments show that APF can reduce data transfer amount by over $60\%$60%.

关键词： Training Convergence Servers Synchronization Quantization (signal) Federated learning Data models distributed computing federated learning machine learning algorithms

来源：评论

学校读者我要写书评

暂无评论

Efficient distributed discovery of bidirectional order dependencies

引用

VLDB JOURNAL 2022年第1期31卷 49-74页

作者： Schmidl, Sebastian Papenbrock, Thorsten Univ Potsdam Hasso Plattner Inst Prof Dr Helmert Str 2-3 D-14482 Potsdam Germany

Bidirectional order dependencies (bODs) capture order relationships between lists of attributes in a relational table. They can express that, for example, sorting books by publication date in ascending order also sorts them by age in descending order. The knowledge about order relationships is useful for many data management tasks, such as query optimization, data cleaning, or consistency checking. Because the bODs of a specific dataset are usually not explicitly given, they need to be discovered. The discovery of all minimal bODs (in set-based canonical form) is a task with exponential complexity in the number of attributes, though, which is why existing bOD discovery algorithms cannot process datasets of practically relevant size in a reasonable time. In this paper, we propose the distributed bOD discovery algorithm DISTOD, whose execution time scales with the available hardware. DISTOD is a scalable, robust, and elastic bOD discovery approach that combines efficient pruning techniques for bOD candidates in set-based canonical form with a novel, reactive, and distributed search strategy. Our evaluation on various datasets shows that DISTOD outperforms both single-threaded and distributed state-of-the-art bOD discovery algorithms by up to orders of magnitude;it can, in particular, process much larger datasets.

关键词： Bidirectional order dependencies distributed computing Actor programming Parallelization Data profiling Dependency discovery

来源：评论

学校读者我要写书评

暂无评论

Fast and accurate variable batch size convolution neural network training on large scale distributed systems

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2022年第21期34卷 e7119-e7119页

作者： Hu, Zhongzhe Xiao, Junmin Sun, Ninghui Tan, Guangming Chinese Acad Sci Inst Comp Technol 6 Kexueyuan South Rd Beijing Peoples R China Univ Chinese Acad Sci Beijing Peoples R China

Large-scale distributed convolution neural network (CNN) training brings two performance challenges: model performance and system performance. Large batch size usually leads to model test accuracy loss, which counteracts the benefits of parallel SGD. The existing solutions require massive hyperparameter hand-tuning. To overcome this difficult, we analyze the training process and find that earlier training stages are more sensitive to batch size. Accordingly, we assert that different stages should use different batch size, and propose a variable batch size strategy. In order to remain high test accuracy under larger batch size cases, we design an auto-tuning engine for automatic parameter tuning in the proposed variable batch size strategy. Furthermore, we develop a dataflow implementation approach to achieve the high-throughput CNN training on supercomputer system. Our approach has achieved high generalization performance on SOAT CNN networks. For the ShuffleNet, ResNet-50, and ResNet-101 training with ImageNet-1K dataset, we scale the batch size to 120 K without accuracy loss and to 128 K with only a slight loss. And the dataflow implementation approach achieves 93.5% scaling efficiency on 1024 GPUs compared with the state-of-the-art.

关键词： deep learning distributed computing ImageNet-1K large-batch training synchronous SGD

来源：评论

学校读者我要写书评

暂无评论

Priority-Based Selection of Individuals in Memetic Algorithms for distributed Data-Intensive Web Service Compositions

引用

IEEE TRANSACTIONS ON SERVICES computing 2022年第5期15卷 2939-2953页

作者： Sadeghiram, Soheila Ma, Hui Chen, Gang Victoria Univ Wellington Sch Engn & Comp Sci Wellington 6012 New Zealand

In distributed computing, Web Service Composition (WSC) leads to the effective reuse of existing services and produces added value. WSC must fulfil functional requirements and optimise Quality of Service (QoS) attributes, simultaneously. Memetic Algorithms (MAs) are promising for automatically composing numerous Web services to satisfy the above requirements. Data-intensive Web services focus on providing and updating data with a significant volume of data operation and exchange. However, current composition approaches have ignored the impact of data communication and the distribution of services, which significantly affect the performance when applied to the challenging distributed Data-intensive Web Service Composition (DDWSC) problem. Although recent approaches have revealed the usefulness of local search, they have completely overlooked the question of preferring appropriate composition solutions for local search. To address this research issue, we propose a priority-based selection method for the local search that can be consistently integrated with any MA for DDWSC. This enables us to develop state-of-the-art algorithms for DDWSC by explicitly considering the problem-specific, population and solution-related information for choosing a solution. Extensive experimental evaluation using benchmark datasets shows that our proposed method significantly outperforms several recently proposed methods.

关键词： Web services Quality of service Statistics Sociology Memetics Search problems Genetic algorithms Web services distributed computing memetic algorithms priority-based selection data-intensive Web service composition

来源：评论

学校读者我要写书评

暂无评论

The integration strategy of information system based on artificial intelligence big data technology in metaverse environment

引用

CLUSTER computing-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2024年第5期27卷 7049-7057页

作者： Lin, Yechuan Liu, Shixing Liaodong Univ Network & Informatizat Ctr Dandong 118000 Liaoning Peoples R China Liaoning Univ Coll Phys Shenyang 110036 Liaoning Peoples R China

The concept of the meta-universe is still in its early stages, but many leading tech companies have invested heavily in research and development for this technology. The development of meta-smart cities is a significant trend. In the meta-universe environment, integrating information systems is crucial for analyzing AI big data. Establishing an integrated platform for medical information systems is key to advancing information technology. In the context of the meta-universe, creating an efficient and unified integration platform to eliminate medical information silos and reduce system integration costs has become a pressing issue in medical informatization. This paper proposes a medical information system integration method based on an integration platform and utilizing cloud computing technology as a data center. The core business layer uses the integration software "Ensemble" as the integration platform. The underlying data center employs a Hadoop storage cluster with distributed data storage and parallel computing technology, and the existing scheduling algorithm is studied and analyzed to enhance the resource scheduling algorithm for medical small file data. The effectiveness of the algorithm is simulated and verified on an experimental platform, demonstrating improved efficiency in resource scheduling.

关键词： Metaverse Big data Healthcare information system integration Hadoop distributed computing

来源：评论

学校读者我要写书评

暂无评论

Time-Constrained Node Visit Planning for Collaborative UAV-WSN distributed Applications

引用

SENSORS 2022年第14期22卷 5298-5298页

作者： Augello, Andrea Gaglio, Salvatore Lo Re, Giuseppe Peri, Daniele Univ Palermo Dept Engn Viale SciEd 6 I-90128 Palermo Italy Natl Res Council CNR Inst High Performance Comp & Networking ICAR Via Ugo La Malfa 153 I-90146 Palermo Italy

Unmanned Aerial Vehicles (UAVs) are often studied as tools to perform data collection from Wireless Sensor Networks (WSNs). Path planning is a fundamental aspect of this endeavor. Works in the current literature assume that data are always ready to be retrieved when the UAV passes. This operational model is quite rigid and does not allow for the integration of the UAV as a computational object playing an active role in the network. In fact, the UAV could begin the computation on a first visit and retrieve the data later. Potentially, the UAV could orchestrate the distributed computation to improve its performance, change its parameters, and even upload new applications to the sensor network. In this paper, we analyze a scenario where a UAV plays an active role in the operation of multiple sensor networks by visiting different node clusters to initiate distributed computation and collect the final outcomes. The experimental results validate the effectiveness of the proposed method in optimizing total flight time, Average Age of Information, Average cluster computation end time, and Average data collection time compared to prevalent approaches to UAV path-planning that are adapted to the purpose.

关键词： distributed computing UAV UAV-WSN integration UAV path planning UAV task scheduling UAV-supported IoT systems for smart cities

来源：评论

学校读者我要写书评

暂无评论

Research on adaptive control strategy for distributed new energy 9

Research on adaptive control strategy for distributed new en...

引用

9th International Conference on Energy Materials and Electrical Engineering, ICEMEE 2023

作者： Zhi, Qingyu Xia, Lei Wang, Xianjun Zhang, Shilin Zheng, Ke State Grid Henan Marketing Service Center Zhengzhou China

ISBN: (纸本)9781510672765

In this paper, adaptive control strategies for distributed new energy systems are discussed in detail. The capacity of new energy sources is often affected by climate and weather conditions, so traditional control methods may not be effective in dealing with these uncertainties. This study introduces adaptive control strategies, which can be adjusted in real time according to the performance or behavior 'ic theory and method of adaptive control. Finally, an adaptive control strategy based on Model Predictive Control (MPC) is proposed, and its effect is verified by simulation experiments. © COPYRIGHT SPIE. Downloading of the abstract is permitted for personal use only.

关键词： Solar energy Wind energy Control systems Data modeling Adaptive control Solar radiation models distributed computing Energy efficiency Instrument modeling Solar cells

来源：评论

学校读者我要写书评

暂无评论

Determining Distributions of Security Means for WSNs Based on the Model of a Neighborhood Watch

引用

IEEE ACCESS 2024年 12卷 74343-74366页

作者： Foerster, Benjamin Langendoerfer, Peter Hinze, Thomas Wireless Syst Innovat High Performance Microelect D-15236 Frankfurt Germany Friedrich Schiller Univ Jena Fac Biol Sci D-07743 Jena Germany

Neighbourhood watch is a concept allowing a community to distribute a complex security task in between all members. Members carry out security tasks in a distributed and cooperative manner ensuring their mutual security and reducing the individual workload while increasing the overall security of the community. Wireless sensor networks (WSNs) are composed of resource-constraint independent battery driven computers as nodes communicating wirelessly. Security in WSNs is essential to prevent attackers from eavesdropping, tampering monitoring results or denying critical nodes from providing their services and potentially cutting off larger network parts. The resource-constraint nature of sensor nodes prevents them from running full-fledged security protocols. Instead, it is necessary to assess the most significant security threats and implement specialised security solutions. A neighbourhood watch inspired distributed security scheme for WSNs has been introduced by Langend & ouml;rfer aiming to increase the variety of attacks a WSN can fend off. The framework intends to statically distribute requirement-based selections of online security means intended to cooperate in close proximity on large-scale static homogeneous WSNs. A framework of such complexity has to be designed in multiple steps. We determine suitable distributions of security means based on graph partitioning concepts. The partitioning algorithms we provide are NP-hard. To evaluate their computability, we implement them as 0 - 1 linear programs (LPs) and test them on WSN models generated with our novel lambda -precision unit disk graph (UDG) generator.

关键词： Security Wireless sensor networks Generators Task analysis Collaboration Computational modeling Optimization Cooperative systems distributed computing Cooperative security framework distributed security means graph generator linear programming neighbourhood watch unit disk graphs wireless sensor networks

来源：评论

学校读者我要写书评

暂无评论

Four node graphlet and triad enumeration on distributed platforms

引用

distributed AND PARALLEL DATABASES 2022年第2-3期40卷 335-372页

作者： Santoso, Yudi Liu, Xiaozhou Srinivasan, Venkatesh Thomo, Alex Univ Victoria Dept Comp Sci Victoria BC Canada

Graphlet enumeration is a basic task in graph analysis with many applications. Thus it is important to be able to perform this task within a reasonable amount of time. However, this objective is challenging when the input graph is very large, with millions of nodes and edges. Known solutions are limited in terms of the scale of the graph that they can process. distributed computing is often proposed as a solution to improve the maximum scale. However, it has to be done carefully to reduce the overhead cost and to really benefit from the distributed solution. We study the enumeration of four-node graphlets in undirected graphs and triads in directed graphs using a distributed platform. We propose an efficient distributed solution that significantly surpasses the existing solutions on the scale and performance. With this method, we are able to process larger graphs that have never been processed before and enumerate quadrillions of graphlets using a modest cluster of machines. Our experimental results show that our solution has a strong machine scalability close to one.

关键词： distributed computing Graph mining Massive networks Subgraph enumeration

来源：评论

学校读者我要写书评

暂无评论

NEO-DNND: Communication-Optimized distributed Nearest Neighbor Graph Construction

NEO-DNND: Communication-Optimized Distributed Nearest Neighb...

引用

2024 Workshops of the International Conference for High Performance computing, Networking, Storage and Analysis, SC Workshops 2024

作者： Iwabuchi, Keita Steil, Trevor Priest, Benjamin W. Pearce, Roger Sanders, Geoffrey Lawrence Livermore National Laboratory Center for Applied Scientific Computing United States

ISBN: (纸本)9798350355543

Graph-based approximate nearest neighbor algorithms have shown high neighbor structure representation quality. NN-Descent is a widely known graph-based approximate nearest neighbor (ANN) algorithm. However, graph-based approaches are memory- and *** address the drawbacks, we develop a scalable distributed NN-Descent. Our NEO-DNND (neighbor-checking efficiency optimized distributed NN-Descent) is built on top of MPI and designed to utilize network bandwidth efficiently. NEO-DNND reduces duplicate elements, increases intra-node data sharing, and leverages available DRAM to replicate data that may be sent ***-DNND showed remarkable scalability up to 256 nodes and was able to construct neighborhood graphs from billion-scale datasets. Compared to a leading shared-memory ANN library, NEO-DNND achieved competitive performance even on a single node and exhibited 41.7X better performance by scaling up to 32 nodes. Furthermore, NEO-DNND outperformed a state-of-the-art distributed NN-Descent implementation, achieving up to a 6.0X speedup. © 2024 IEEE.

关键词： approximate nearest neighbor distributed computing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：