With the rapid growth of data volume, knowledge acquisition for bigdata has become a new challenge. To address this issue, the hierarchical decision table is defined and implemented in this work. The properties of di...
详细信息
ISBN:
(纸本)9781467372220
With the rapid growth of data volume, knowledge acquisition for bigdata has become a new challenge. To address this issue, the hierarchical decision table is defined and implemented in this work. The properties of different hierarchical decision tables are discussed under the different granularity of conditional attributes. A novel knowledge acquisition algorithm for bigdata using MapReduce is proposed. Experimental results demonstrate that the proposed algorithm is able to deal with bigdata and mine hierarchical decision rules under the different granularity.
Adding to societal changes today, are the miscellaneous bigdata produced in different fields. Coupled with these data is the appearance of risk management. Admittedly, to predict future trend by using these data is c...
详细信息
Adding to societal changes today, are the miscellaneous bigdata produced in different fields. Coupled with these data is the appearance of risk management. Admittedly, to predict future trend by using these data is conducive to make everything more efficient and easy. Now, no matter companies or individuals, they increasingly focus on identifying risks and managing them before risks. Effective risk management will lead them to deal with potential problems. This thesis focuses on risk management of flight delay area using big real time data. It proposes two different prediction models, one is called General Long Term Departure Prediction Model and the other is named as Improved Real Time Arrival Prediction Model. By studying the main factors lead to flight delay, this thesis takes weather, carrier, National Aviation System, security and previous late aircraft as analysis factors. By utilizing our models can do not only long time but also short term flight delay predictions. The results demonstrate goodness of fit. Besides the theory part, it also presents a practical and beautiful web application for real time flight arrival prediction based on our second model.
Distributed Collaborative Machine Learning (DCML) has emerged in artificial intelligence-empowered edge computing environments, such as the Industrial Internet of Things (IIoT), to process tremendous data generated by...
详细信息
Distributed Collaborative Machine Learning (DCML) has emerged in artificial intelligence-empowered edge computing environments, such as the Industrial Internet of Things (IIoT), to process tremendous data generated by smart devices. However, parallel DCML frameworks require resource-constrained devices to update the entire Deep Neural Network (DNN) models and are vulnerable to reconstruction attacks. Concurrently, the serial DCML frameworks suffer from training efficiency problems due to their serial training nature. In this paper, we propose a Model Pruning-enabled Federated Split Learning framework (MP-FSL) to reduce resource consumption with a secure and efficient training scheme. Specifically, MP-FSL compresses DNN models by adaptive channel pruning and splits each compressed model into two parts that are assigned to the client and the server. Meanwhile, MP-FSL adopts a novel aggregation algorithm to aggregate the pruned heterogeneous models. We implement MP-FSL with a real FL platform to evaluate its performance. The experimental results show that MP-FSL outperforms the state-of-the-art frameworks in model accuracy by up to 1.35%, while concurrently reducing storage and computational resource consumption by up to 32.2% and 26.73%, respectively. These results demonstrate that MP-FSL is a comprehensive solution to the challenges faced by DCML, with superior performance in both reduced resource consumption and enhanced model performance.
暂无评论