检索结果-内蒙古大学图书馆

distributed machine learning in Edge Computing: Challenges, Solutions and Future Directions

ACM COMPUTING SURVEYS 2025年第5期57卷 1-37页

作者： Tu, Jingke Yang, Lei Cao, Jiannong South China Univ Technol Guangzhou Peoples R China South China Univ Technol Sch Software Engn Guangzhou Peoples R China Hong Kong Polytech Univ Dept Comp Hong Kong Peoples R China

distributed machine learning on edges is widely used in intelligent transportation, smart home, industrial manufacturing, and underground pipe network monitoring to achieve low latency and real time data processing and prediction. However, the presence of a large number of sensing and edge devices with limited computing, storage, and communication capabilities prevents the deployment of huge machine learning models and hinders its application. At the same time, although distributed machine learning on edges forms an emerging and rapidly growing research area, there has not been a systematic survey on this topic. The article begins by detailing the challenges of distributed machine learning in edge environments, such as limited node resources, data heterogeneity, privacy, security issues, and summarizes common metrics for model optimization. We then present a detailed analysis of parallelism patterns, distributed architectures, and model communication and aggregation schemes in edge computing. we subsequently present a comprehensive classification and intensive description of node resource-constrained processing, heterogeneous data processing, attacks and protection of privacy. The article ends by summarizing the applications of distributed machine learning in edge computing and presenting problems and challenges for further research.

关键词： Edge computing distributed machine learning model optimization data heterogeneity communication constraints

来源：评论

学校读者我要写书评

暂无评论

A distributed machine learning-Based Scheme for Real-Time Highway Traffic Flow Prediction in Internet of Vehicles

引用

FUTURE INTERNET 2025年第3期17卷 131-131页

作者： Alnami, Hani Mahgoub, Imad Al-Najada, Hamzah Alalwany, Easa Florida Atlantic Univ Elect Engn & Comp Sci 777 Glades Rd Boca Raton FL 33431 USA Jazan Univ Comp Sci Dept Jazan 82917 Saudi Arabia Taibah Univ Coll Comp Sci & Engn Yanbu 46421 Saudi Arabia

Abnormal traffic flow prediction is crucial for reducing traffic congestion. Most recent studies utilized machine learning models in traffic flow detection systems. However, these detection systems do not support real-time analysis. Centralized machine learning methods face a number of challenges due to the sheer volume of traffic data that needs to be processed in real-time. Thus, it is not scalable and lacks fault tolerance and data privacy. This study designs and evaluates a scalable distributed machine learning-based scheme to predict highway traffic flows in real-time. The proposed system is segment-based where the vehicles in each segment form a cluster. We train and validate a local Random Forest Regression (RFR) model for each vehicle's cluster (highway-segment) using six different hyper parameters. Due to the variance of traffic flow patterns between segments, we build a global distributed machine learning Random Forest (DMLRF) regression model to improve the system performance for abnormal traffic flows. Kappa Architecture is utilized to enable real-time prediction. The proposed model is evaluated and compared to other base-line models, Linear Regression (LR), Logistic Regression (LogR), and K Nearest Neighbor (KNN) regression in terms of Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), R-squared (R2), and Adjusted R-Squared (AR2). The proposed scheme demonstrates high accuracy in predicting abnormal traffic flows while maintaining scalability and data privacy.

关键词： internet of vehicles real-time traffic flow prediction intelligent transportation systems big data processing distributed machine learning distributed architecture Apache Spark

来源：评论

学校读者我要写书评

暂无评论

NetJIT: Bridging the Gap from Traffic Prediction to Preknowledge for distributed machine learning

引用

PROCEEDINGS OF THE ACM ON MEASUREMENT AND ANALYSIS OF COMPUTING SYSTEMS 2025年第1期9卷 1-25页

作者： Ai, Xin Li, Zijian Zhu, Yuanyi Chen, Zixuan Liu, Sen Xu, Yang Fudan Univ Shanghai Peoples R China

Today's distributed machine learning (DML) introduces heavy traffic load, making the interconnection network one of the primary bottlenecks. To mitigate this bottleneck, existing state-of-the-art network optimization methods, such as traffic or topology engineering, are proposed to adapt to real-time traffic. However, current traffic measurement and prediction methods struggle to collect sufficiently fine-grained and accurate traffic patterns. This limitation impedes the ability of cutting-edge network optimization techniques to react agilely to the ever-changing traffic demands of DML jobs. This paper proposes NetJIT, a novel program-behavior-aware toolkit for accurately foreseeing the traffic pattern of DML. To the best of our knowledge, this is the first work proposing the use of just-in-time (JIT) program analysis for real-time traffic measurement. In DML applications, communication behavior is primarily determined by the previously computed results. NetJIT leverages this characteristic to anticipate communication details by tracing and analyzing the data relations in the computation process. This capability enables the deployment of optimization strategies in advance. We deploy NetJIT in real-world network optimization for traffic preknowledge. Evaluation with the self-built testbed prototype demonstrates that NetJIT can achieve up to about 97% less error of detecting communication events compared with other methods. Simulations with real-world DML workloads further illustrate that NetJIT enables more precise network optimization, leading to approximately 50% better network performance w.r.t the metrics including average iteration time, throughput, and average packet delay.

关键词： distributed machine learning Traffic Prediction Just-in-time Program Analysis JIT Network Optimization

来源：评论

学校读者我要写书评

暂无评论

Edge-cloud solutions for big data analysis and distributed machine learning-2

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 167卷

作者： Belcastro, Loris Carretero, Jesus Talia, Domenico Univ Calabria Comp Engn Arcavacata Di Rende CS Italy Univ Carlos III Comp Architecture & Technol Madrid Spain

In recent years, edge-cloud solutions have gained widespread adoption for efficiently collecting and analyzing IoT-generated data across various domains like urban mobility, healthcare, and smart cities. These solutions integrate resources from edge to cloud to support real-time processing and analysis tasks, reducing latency and network congestion. Big data analysis within this paradigm involves sophisticated techniques for distributed data processing, enabling applications such as predictive maintenance and smart grid management. Nevertheless, carrying out big data analysis within the edge-cloud presents several challenges, including data privacy and security, interoperability, scalability, and energy efficiency. Addressing these challenges is imperative for providing efficient and scalable solutions for data-intensive applications like federated learning, social data analysis, smart city services, and text mining. The special issue concludes with 27 scientific papers, divided into two parts for a streamlined editorial process. This editorial, as part two, presents 12 rigorously peer-reviewed papers, complementing the 15 papers covered in the previous editorial.

关键词： Edge computing Edge-cloud continuum Big data distributed machine learning Internet-of-things Federated learning

来源：评论

学校读者我要写书评

暂无评论

The Fast Inertial ADMM optimization framework for distributed machine learning

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2025年 164卷

作者： Wang, Guozheng Wang, Dongxia Li, Chengfan Lei, Yongmei Shanghai Univ Sch Comp Engn & Sci Shanghai 200444 Peoples R China Hunan Inst Sci & Technol Yueyang 414006 Hunan Peoples R China

The ADMM (Alternating Direction Method of Multipliers) optimization framework is known for its property of decomposition and assembly, which effectively bridges distributed computing and optimization algorithms, making it well-suited for distributed machine learning in the context of big data. However, it suffers from slow convergence speed and lacks the ability to coordinate worker computations, resulting in inconsistent speeds in solving subproblems in distributed systems and mutual waiting among workers. In this paper, we propose a novel optimization framework to address these challenges in support vector regression (SVR) and probit regression training through the FIADMM (Fast I nertial ADMM). The key concept of the FIADMM lies in the introduction of inertia acceleration and an adaptive subproblem iteration mechanism based on the ADMM, aimed at accelerating convergence speed and reducing the variance in solving speeds among workers. Further, we prove that FIADMM has a fast linear convergence rate O (1/k). Experimental results on six benchmark datasets demonstrate that the proposed FIADMM significantly enhances convergence speed and computational efficiency compared to multiple baseline algorithms and related efforts.

关键词： ADMM Optimization framework distributed machine learning Inertial acceleration

来源：评论

学校读者我要写书评

暂无评论

Enhancing Knowledge Reusability: A distributed Multitask machine learning Approach

引用

IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTING 2025年第1期13卷 207-221页

作者： Long, Qianyu Anagnostopoulos, Christos Kolomvatsos, Kostas Univ Glasgow Sch Comp Sci Glasgow G12 8QQ Scotland Univ Thessaly Dept Informat & Telecommun Volos 38221 Greece

In the era of the Internet of Things, the unprecedented growth of data surpasses current predictive analytics and processing capabilities. Due to the potential redundancy of similar data and analytics tasks, it is imperative to extract patterns from distributed data and predictive models so that existing schemes can be efficiently reused in distributed computing environments. This is expected to avoid building and maintaining reduplicative predictive models. The fundamental challenge, however, is the detection of reusable tasks and tuning models in order to improve predictive capacity while being reused. We introduce a two-phase distributed Multi-task machine learning (DMtL) framework coping with this challenge. In the first phase, similar tasks are identified and efficiently grouped together according to locally trained models' performance meta-features, using Partial learning Curves (PLC). In the subsequent phase, we leverage the PLC-driven DMtL paradigm to boost the performance of candidate reusable models per group of tasks in distributed computing environments. We provide a thorough analysis of our framework along with a comparative assessment against relevant approaches and prior work found in the respective literature. Our experimental results showcase the feasibility of the PLC-driven DMtL method in terms of adaptability and reusability of existing knowledge in distributed computing systems.

关键词： Task analysis Computational modeling Predictive models Data models Analytical models Training distributed databases distributed machine learning knowledge reuse knowledge management multitask learning

来源：评论

学校读者我要写书评

暂无评论

distributed machine learning Based Downlink Channel Estimation for RIS Assisted Wireless Communications

引用

IEEE TRANSACTIONS ON COMMUNICATIONS 2022年第7期70卷 4900-4909页

作者： Dai, Linglong Wei, Xiuhong Tsinghua Univ Beijing Natl Res Ctr Informat Sci & Technol BNRis Beijing 100084 Peoples R China Tsinghua Univ Dept Elect Engn Beijing 100084 Peoples R China

The downlink channel estimation requires a huge pilot overhead in the reconfigurable intelligent surface (RIS) assisted communication system. By exploiting the powerful learning ability of the neural network, the machine learning (ML) technique can be used to estimate the high-dimensional channel from a few received pilot signals at the user. However, since the training dataset collected by the single user only contains the information of part of the channel scenarios of a cell, the neural network trained by the single user is not able to work when the user moves from one channel scenario to another. To solve this challenge, we propose to leverage the distributed machine learning (DML) technique to enable the reliable downlink channel estimation. Specifically, we firstly build a downlink channel estimation neural network shared by all users, which can be collaboratively trained by the BS and the users with the help of the DML technique. Then, we further propose a hierarchical neural network architecture to improve the channel estimation accuracy, which can extract different channel features for different channel scenarios. Simulation results show that compared with the neural network trained by the single user, the proposed DML based neural networks can achieve better estimation performance with the reduced pilot overhead for all users from different scenarios.

关键词： Channel estimation Neural networks Feature extraction Downlink Computer architecture Training Wireless communication distributed machine learning reconfigurable intelligent surface (RIS) channel estimation

来源：评论

学校读者我要写书评

暂无评论

distributed machine learning in networks by consensus

引用

NEUROCOMPUTING 2014年 124卷 2-12页

作者： Georgopoulos, Leonidas Hasler, Martin Ecole Polytech Fed Lausanne Sch Comp & Commun Sci EPFL IC ISC LANOS Stn 14 CH-1015 Lausanne Switzerland

We propose an algorithm to learn from distributed data on a network of arbitrarily connected machines without exchange of the data-points. Parts of the dataset are processed locally at each machine, and then the consensus communication algorithm is employed to consolidate the results. This iterative two stage process converges as if the entire dataset had been on a single machine. The principal contribution of this paper is the proof of convergence of the distributed learning process in the general case that the learning algorithm is a contraction. Moreover, we derive the distributed update equation of a feed-forward neural network with back-propagation for the purpose of verifying the theoretical results. We employ a toy classification example and a real world binary classification dataset. (C) 2013 Elsevier B.V. All rights reserved.

关键词： distributed machine learning Parallel machine learning Gradient descent Consensus Peer-to-peer learning Neural networks

来源：评论

学校读者我要写书评

暂无评论

distributed machine learning for Wireless Communication Networks: Techniques, Architectures, and Applications

引用

IEEE COMMUNICATIONS SURVEYS AND TUTORIALS 2021年第3期23卷 1458-1493页

作者： Hu, Shuyan Chen, Xiaojing Ni, Wei Hossain, Ekram Wang, Xin Fudan Univ Sch Informat Sci & Technol State Key Lab ASIC & Syst Shanghai 200433 Peoples R China Shanghai Univ Shanghai Inst Adv Commun & Data Sci Shanghai 200444 Peoples R China Commonwealth Sci & Ind Res Org Digital Prod & Serv Flagship Sydney NSW 2122 Australia Univ Manitoba Dept Elect & Comp Engn Winnipeg MB R3T 5V6 Canada

distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications. This is due to improved capabilities of terminal devices, explosively growing data volume, congestion in the radio interfaces, and increasing concern of data privacy. The unique features of wireless systems, such as large scale, geographically dispersed deployment, user mobility, and the massive amount of data, give rise to new challenges in the design of DML techniques. There is a clear gap in the existing literature that the DML techniques are yet to be systematically reviewed for their applicability to wireless systems. This survey bridges the gap by providing a contemporary and comprehensive survey of DML techniques with a focus on wireless networks. Specifically, we review the latest applications of DML in power control, spectrum management, user association, and edge cloud computing. The optimality, accuracy, convergence rate, computation cost, and communication overhead of DML are analyzed. We also discuss the potential adversarial attacks faced by DML applications, and describe state-of-the-art countermeasures to preserve privacy and security. Last but not least, we point out a number of key issues yet to be addressed, and collate potentially interesting and challenging topics for future research.

关键词： Wireless communication Wireless networks Communication system security Wireless sensor networks Computer architecture Servers Security distributed machine learning wireless communication networks convergence computation and communication cost architecture and platform data privacy and security

来源：评论

学校读者我要写书评

暂无评论

distributed machine learning load balancing strategy in cloud computing services

引用

WIRELESS NETWORKS 2020年第8期26卷 5517-5533页

作者： Li, Mingwei Zhang, Jilin Wan, Jian Ren, Yongjian Zhou, Li Wu, Baofu Yang, Rui Wang, Jue Hangzhou Dianzi Univ Sch Comp Sci & Technol Hangzhou 310018 Peoples R China Minist Educ Key Lab Complex Syst Modeling & Simulat Hangzhou 310018 Peoples R China Chinese Acad Sci Inst Comp Technol Beijing 100190 Peoples R China Zhejiang Univ Sci & Technol Sch Informat & Elect Engn Hangzhou Zhejiang Peoples R China Chinese Acad Sci Supercomp Ctr Comp Network Informat Ctr Beijing 100190 Peoples R China

Mobile service computing is a new cloud computing model that provides various cloud services for mobile intelligent terminal users through mobile internet access. The quality of service is an essential problem faced by mobile service computing. In this paper, we demonstrate a series of research studies on how to accelerate the training of a distributed machine learning (ML) model based on cloud service. distributed ML has become the mainstream way of today's ML models training. In traditional distributed ML based on bulk synchronous parallel, the temporary slowdown of any node in the cluster will delay the calculation of other nodes because of the frequent occurrence of synchronous barriers, resulting in overall performance degradation. Our paper proposes a load balancing strategy named adaptive fast reassignment (AdaptFR). Based on this, we built a distributed parallel computing model called adaptive-dynamic synchronous parallel (A-DSP). A-DSP uses a more relaxed synchronization model to reduce the performance consumption caused by synchronous operations while ensuring the consistency of the model. At the same time, A-DSP also implements the AdaptFR load balancing strategy, which addresses the straggler problem caused by the performance difference between nodes under the premise of ensuring the accuracy of the model. The experiments show that A-DSP can effectively improve the training speed while ensuring the accuracy of the model in the distributed ML model training.

关键词： Mobile service computing Cloud service distributed machine learning Load balancing Adaptive fast reassignment

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：