检索结果-内蒙古大学图书馆

Enhancing Earth data analysis in 5G satellite networks: A novel lightweight approach integrating improved deep learning

引用

HELIYON 2024年第11期10卷 e32071页

作者： Yang, Yukun Ren, Kun Song, Jiong China Geol Survey Geophys Ctr 159 Fengsheng Rd Langfang 065000 Hebei Peoples R China China Geol Survey Technol Innovat Ctr Earth Near Surface Detect Langfang 065000 Hebei Peoples R China

Efficiently handling huge data amounts and enabling processing-intensive applications to run in faraway areas simultaneously is the ultimate objective of 5G networks. Currently, in order to distribute computing tasks, ongoing studies are exploring the incorporation of fog-cloud servers onto satellites, presenting a promising solution to enhance connectivity in remote areas. Nevertheless, analyzing the copious amounts of data produced by scattered sensors remains a challenging endeavor. The conventional strategy of transmitting this data to a central server for analysis can be costly. In contrast to centralized learning methods, distributed machine learning (ML) provides an alternative approach, albeit with notable drawbacks. This paper addresses the comparative learning expenses of centralized and distributed learning systems to tackle these challenges directly. It proposes the creation of an integrated system that harmoniously merges cloud servers with satellite network structures, leveraging the strengths of each system. This integration could represent a major breakthrough in satellite-based networking technology by streamlining data processing from remote nodes and cutting down on expenses. The core of this approach lies in the adaptive tailoring of learning techniques for individual entities based on their specific contextual nuances. The experimental findings underscore the prowess of the innovative lightweight strategy, LMAED2L (Enhanced Deep learning for Earth Data Analysis), across a spectrum of machine learning assignments, showcasing remarkable and consistent performance under diverse operational conditions. Through a strategic fusion of centralized and distributed learning frameworks, the LMAED2L method emerges as a dynamic and effective remedy for the intricate data analysis challenges encountered within satellite networks interfaced with cloud servers. The empirical findings reveal a significant performance boost of our novel approach over traditional

关键词： Processing -intensive applications Satellite networks Computation offloading distributed machine learning Centralized learning Hybrid learning

来源：评论

学校读者我要写书评

暂无评论

machine learning Analysis Using the Black Oil Model and Parallel Algorithms in Oil Recovery Forecasting

引用

ALGORITHMS 2024年第8期17卷

作者： Matkerim, Bazargul Mukhanbet, Aksultan Kassymbek, Nurislam Daribayev, Beimbet Mustafin, Maksat Imankulov, Timur Natl Engn Acad Republ Kazakhstan Alma Ata 050010 Kazakhstan Al Farabi Kazakh Natl Univ Dept Comp Sci Alma Ata 050040 Kazakhstan Joldasbekov Inst Mech & Engn Alma Ata 050000 Kazakhstan

The accurate forecasting of oil recovery factors is crucial for the effective management and optimization of oil production processes. This study explores the application of machine learning methods, specifically focusing on parallel algorithms, to enhance traditional reservoir simulation frameworks using black oil models. This research involves four main steps: collecting a synthetic dataset, preprocessing it, modeling and predicting the oil recovery factors with various machine learning techniques, and evaluating the model's performance. The analysis was carried out on a synthetic dataset containing parameters such as porosity, pressure, and the viscosity of oil and gas. By utilizing parallel computing, particularly GPUs, this study demonstrates significant improvements in processing efficiency and prediction accuracy. While maintaining the value of the R2 metric in the range of 0.97, using data parallelism sped up the learning process by, at best, 10.54 times. Neural network training was accelerated almost 8 times when running on a GPU. These findings underscore the potential of parallel machine learning algorithms to revolutionize the decision-making processes in reservoir management, offering faster and more precise predictive tools. This work not only contributes to computational sciences and reservoir engineering but also opens new avenues for the integration of advanced machine learning and parallel computing methods in optimizing oil recovery.

关键词： distributed machine learning HPC artificial intelligence cuML enhanced oil recovery

来源：评论

学校读者我要写书评

暂无评论

Resilient edge predictive analytics by enhancing local models

引用

OPEN COMPUTER SCIENCE 2024年第1期14卷

作者： Wang, Qiyuan AlFahad, Saleh Fornes, Jordi Mateo Anagnostopoulos, Christos Kolomvatsos, Kostas Univ Glasgow Sch Comp Sci Glasgow Scotland Univ Lleida Dept Comp Sci Lleida Spain Univ Thessaly Dept Informat & Telecommun Lamia Greece

In distributed computing environments, the collaboration of nodes for predictive analytics at the network edge plays a crucial role in supporting real-time services. When a node's service becomes unavailable for various reasons (e.g., service updates, node maintenance, or even node failure), the rest of the available nodes connot efficiently replace its service due to different data and predictive models (e.g., machine learning [ML] models). To address this, we propose decision-making strategies rooted in the statistical signatures of nodes' data. Specifically, these signatures refer to the unique patterns and behaviors within each node's data that can be leveraged to predict the suitability of potential surrogate nodes. Recognizing and acting on these statistical nuances ensures a more targeted and efficient response to node failures. Such strategies aim to identify surrogate nodes capable of substituting for failing nodes' services by building enhanced predictive models. Our resilient framework helps to guide the task requests from failing nodes to the most appropriate surrogate nodes. In this case, the surrogate nodes can use their enhanced models, which can produce equivalent and satisfactory results for the requested tasks. We provide experimental evaluations and comparative assessments with baseline approaches over real datasets. Our results showcase the capability of our framework to maintain the overall performance of predictive analytics under nodes' failures in edge computing environments.

关键词： edge computing model resilience predictive service model generalization distributed machine learning node failures

来源：评论

学校读者我要写书评

暂无评论

On the discardability of data in Support Vector Classification problems

On the discardability of data in Support Vector Classificati...

引用

IEEE Conference on Decision and Control

作者： Simone Del Favero Damiano Varagnolo Francesco Dinuzzo Luca Schenato Gianluigi Pillonetto Univ. of Padova Italy Max Planck Inst. for Intelligent Systems Germany

ISBN: (纸本)9781612848006

We analyze the problem of data sets reduction for support vector classification. The work is also motivated by distributed problems, where sensors collect binary measurements at different locations moving inside an environment that needs to be divided into a collection of regions labeled in two different ways. The scope is to let each agent retain and exchange only those measurements that are mostly informative for the collective reconstruction of the decision boundary. For the case of separable classes, we provide the exact conditions and an efficient algorithm to determine if an element in the training set can become a support vector when new data arrive. The analysis is then extended to the non-separable case deriving a sufficient discardability condition and a general data selection scheme for classification. Numerical experiments relative to the distributed problem show that the proposed procedure allows the agents to exchange a small amount of the collected data to obtain a highly predictive decision boundary.

关键词： distributed classification support vector machines model reduction distributed machine learning simplex convex analysis

来源：评论

学校读者我要写书评

暂无评论

A Hybrid Parallelization Approach Based on Workers Grouping Algorithm

A Hybrid Parallelization Approach Based on Workers Grouping ...

引用

作者： Ruoyu Lu SouthWest Jiaotong University

As the volume of model data increases,traditional machine learning is not able to train models efficiently,so distributed machine learning is gradually used in large-scale data ***,commonly used distributed machine learning algorithms are based on data parallelism,and often use an overall synchronous parallel strategy when passing data,but using this strategy makes the overall training speed limited by the computation speed of the slower workers in the *** the asynchronous parallel strategy maximizes the computational speed of the cluster,there is a delay in updating the parameters of the global model,which may lead to excessive computational errors or non-convergence of the *** this paper,the author combines these two data delivery methods by grouping workers together and using synchronous parallelism for the workers in the group and asynchronous parallelism for the components for *** experiment shows that the hybrid parallelism strategy can reduce the training time with guaranteed correctness.

关键词： Deep learning Data parallel distributed machine learning Synchronous parallel Asynchronous parallel

来源：评论

学校读者我要写书评

暂无评论

distributed Pattern Recognition within DHT-Based Networks for Imbalanced Datasets

Distributed Pattern Recognition within DHT-Based Networks fo...

引用

2016 First IEEE International Conference on Computer Communication and the Internet

作者： Amiza Amir Bala Srinivasan Asad I.Khan School of Computer and Communication Engineering Universiti Malaysia Perlis Faculty of Information Technology Monash University

ISBN: (纸本)9781467385169

This paper studies the accuracy of a fullydistributed pattern recognition algorithm,namely the P2P-GN,with imbalanced datasets problem which is often neglected by most of the available distributed algorithms.A major distinction of the P2P-GN compared to the other approaches is that it forms a single global classifier,instead of building many local classifiers(one at every site).Fine-granularity components of the classifier are distributed across the network by using distributed Hash Table(DHT)—which provides efficient linking to these components and ensures the system remains *** experimental results also show that the P2P-GN can produce highly accurate results despite imbalanced data distribution compared to other distributed algorithms(Ivote-DPV,ensemble ID3,ensemble k-NN,and ensemble na?ve Bayes).

关键词： P2P-based classification distributed machine learning efficient distributed algorithm distributed classification

来源：评论

学校读者我要写书评

暂无评论

A model-based approach to streamlining distributed training for asynchronous SGD

A model-based approach to streamlining distributed training ...

引用

IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems

作者： Sung-Han Lin Marco Paolieri Cheng-Fu Chou Leana Golubchik Department of Computer Science University of Southern California Department of Computer Science National Taiwan University

ISBN: (纸本)9781538668870

The success of Deep Neural Networks (DNNs) has created significant interest in the development of software tools, hardware architectures, and cloud systems to meet the huge computational demand of their training jobs. A common approach to speeding up an individual job is to distribute training data and computation among multiple nodes, periodically exchanging intermediate results. In this paper, we address two important problems for the application of this strategy to large-scale clusters and multiple, heterogeneous jobs. First, we propose and validate a queueing model to estimate the throughput of a training job as a function of the number of nodes assigned to the job;this model targets asynchronous Stochastic Gradient Descent (SGD), a popular strategy for distributed training, and requires only data from quick, two-node profiling in addition to job characteristics (number of requested training epochs, mini-batch size, size of DNN parameters, assigned bandwidth). Throughput estimations are then used to explore several classes of scheduling heuristics to reduce response time in a scenario where heterogeneous jobs are continuously submitted to a large-scale cluster. These scheduling algorithms dynamically select which jobs to run and how many nodes to assign to each job, based on different trade-offs between service time reduction and efficiency (e.g., speedup per additional node). Heuristics are evaluated through extensive simulations of realistic DNN workloads, also investigating the effects of early termination, a common scenario for DNN training jobs.

关键词： distributed machine learning Parallel Scheduling Queueing Networks TensorFlow

来源：评论

学校读者我要写书评

暂无评论

Research on distributed coding computation based on matrix multiplication

Research on distributed coding computation based on matrix m...

引用

作者： Lin Zhang Xi'an University of Technology

Matrix multiplication is used in machine learning to train larger,more accurate models with massive data more *** distributed machine learning paradigm necessitates the distributed computing of many matrix multiplications once it is *** execution time of distributed machine learning algorithms increases on dropout nodes,which are computational nodes in distributed clusters that randomly slow down computation due to a variety of factors(e.g.,node failure,system failure,communication bottlenecks,etc.),becoming a significant bottleneck in distributed computing *** has been discovered that coded computation is less expensive than replica methods and can more effectively reduce the effects of dropped *** this study,we present theoretical insights on how encoded solutions might produce significant gains over unencoded solutions.

关键词： coding computing distributed machine learning matrix multiplication

来源：评论

学校读者我要写书评

暂无评论

Classification algorithms and how to distribute them 17

Classification algorithms and how to distribute them

引用

Proceedings of the 27th Annual International Conference on Computer Science and Software Engineering

作者： Shadi Khalifa Patrick Martin Rebecca Young Queen's University Canada IBM Canada

The problem of how to adapt classification algorithms to handle the large volume of data associated with Big Data is commonly solved by rewriting the algorithms to run in a distributed fashion using a parallel programming language (e.g. OpenMP) or a parallel framework (e.g. Hadoop, Spark). While this approach can result in fast algorithms, it is time consuming and can be very challenging to implement for all algorithms. In this paper, we first categorize classification algorithms in terms of the difficulty to distribute them. Second, we propose the distributed Classifier Training (DCT) approach for distributing all types of classification algorithms that maintains the same prediction accuracy without having to rewriting them. Finally, we implement the DCT approach as a free open-source IBM SPSS Modeler plugin.

关键词： apache drill SPSS modeler weka distributed machine learning big data analytics

来源：评论

学校读者我要写书评

暂无评论

Privacy-Preserving Deep learning for the Detection of Protected Health Information in Real-World Data: Comparative Evaluation

引用

JMIR FORMATIVE RESEARCH 2020年第5期4卷 e14064页

作者： Festag, Sven Spreckelsen, Cord Rhein Westfal TH Aachen Med Fac Dept Med Informat Aachen Germany Jena Univ Hosp Inst Med Stat Comp & Data Sci Bachstr 18 Jena Germany

Background: Collaborative privacy-preserving training methods allow for the integration of locally stored private data sets into machine learning approaches while ensuring confidentiality and nondisclosure. Objective: In this work we assess the performance of a state-of-the-art neural network approach for the detection of protected health information in texts trained in a collaborative privacy-preserving way. Methods: The training adopts distributed selective stochastic gradient descent (ie, it works by exchanging local learning results achieved on private data sets). Five networks were trained on separated real-world clinical data sets by using the privacy-protecting protocol. In total, the data sets contain 1304 real longitudinal patient records for 296 patients. Results: These networks reached a mean F1 value of 0.955. The gold standard centralized training that is based on the union of all sets and does not take data security into consideration reaches a final value of 0.962. Conclusions: Using real-world clinical data, our study shows that detection of protected health information can be secured by collaborative privacy-preserving training. In general, the approach shows the feasibility of deep learning on distributed and confidential clinical data while ensuring data protection.

关键词： privacy-preserving protocols neural networks health informatics distributed machine learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：