Gradient compression is a promising approach to alleviating the communication bottleneck in data parallel deep neural network (DNN) training by significantly reducing the data volume of gradients for synchronization. ...
详细信息
Gradient compression is a promising approach to alleviating the communication bottleneck in data parallel deep neural network (DNN) training by significantly reducing the data volume of gradients for synchronization. While gradient compression is being actively adopted by the industry (e.g., Facebook and AWS), our study reveals that there are two critical but often overlooked challenges: 1) inefficient coordination between compression and communication during gradient synchronization incurs substantial overheads, and 2) developing, optimizing, and integrating gradient compression algorithms into DNN systems imposes heavy burdens on DNN practitioners, and ad-hoc compression implementations often yield surprisingly poor system performance. In this paper, we propose a compression-aware gradient synchronization architecture, CaSync, which relies on flexible composition of basic computing and communication primitives. It is general and compatible with any gradient compression algorithms and gradient synchronization strategies and enables high-performance computation-communication pipelining. We further introduce a gradient compression toolkit, CompLL, to enable efficient development and automated integration of on-GPU compression algorithms into DNN systems with little programming burden. Lastly, we build a compression-aware DNN training framework HiPress with CaSync and CompLL. HiPress is open-sourced and runs on mainstream DNN systems such as MXNet, TensorFlow, and PyTorch. Evaluation via a 16-node cluster with 128 NVIDIA V100 GPUs and a 100 Gbps network shows that HiPress improves the training speed over current compression-enabled systems (e.g., BytePS-onebit, Ring-DGC and PyTorch-PowerSGD) by 9.8%-69.5% across six popular DNN models. IEEE
In real-world materials research,machine learning(ML)models are usually expected to predict and discover novel exceptional materials that deviate from the known *** is thus a pressing question to provide an objective ...
详细信息
In real-world materials research,machine learning(ML)models are usually expected to predict and discover novel exceptional materials that deviate from the known *** is thus a pressing question to provide an objective evaluation ofMLmodel performances in property prediction of out-ofdistribution(OOD)materials that are different fromthe training *** performance evaluation of materials property prediction models through the random splitting of the dataset frequently results in artificially high-performance assessments due to the inherent redundancy of typical material datasets.
The concept of the digital twin,also known colloquially as the DT,is a fundamental principle within Industry 4.0 *** recent years,the concept of digital siblings has generated considerable academic and practical ***,a...
详细信息
The concept of the digital twin,also known colloquially as the DT,is a fundamental principle within Industry 4.0 *** recent years,the concept of digital siblings has generated considerable academic and practical ***,academia and industry have used a variety of interpretations,and the scientific literature lacks a unified and consistent definition of this *** purpose of this study is to systematically examine the definitional landscape of the digital twin concept as outlined in scholarly literature,beginning with its origins in the aerospace domain and extending to its contemporary interpretations in the manufacturing ***,this investigationwill focus on the research conducted on Industry 4.0 and smartmanufacturing,elucidating the diverse applications of digital twins in fields including aerospace,intelligentmanufacturing,intelligent transportation,and intelligent cities,among others.
Cloud computing has emerged as a viable alternative to traditional computing infrastructures,offering various ***,the adoption of cloud storage poses significant risks to data secrecy and *** article presents an effec...
详细信息
Cloud computing has emerged as a viable alternative to traditional computing infrastructures,offering various ***,the adoption of cloud storage poses significant risks to data secrecy and *** article presents an effective mechanism to preserve the secrecy and integrity of data stored on the public cloud by leveraging blockchain technology,smart contracts,and cryptographic *** proposed approach utilizes a Solidity-based smart contract as an auditor for maintaining and verifying the integrity of outsourced *** preserve data secrecy,symmetric encryption systems are employed to encrypt user data before outsourcing *** extensive performance analysis is conducted to illustrate the efficiency of the proposed ***,a rigorous assessment is conducted to ensure that the developed smart contract is free from vulnerabilities and to measure its associated running *** security analysis of the proposed system confirms that our approach can securely maintain the confidentiality and integrity of cloud storage,even in the presence of malicious *** proposed mechanism contributes to enhancing data security in cloud computing environments and can be used as a foundation for developing more secure cloud storage systems.
The advances from the last few decades in the fields of ML (Machine Learning), DL (Deep Learning), and semantic computing are now changing the shape of the healthcare system. But, unlike physical health problems, diag...
详细信息
- Distributed denial-of-service (DDoS) attacks are the major threat that disrupts the services in the computer system and networks using traffic and targeted sources. So, real-world attack detection techniques are con...
详细信息
作者：
Baba, AbdullatifAlothman, Basil
Computer Science and Engineering Department Kuwait
Computer Engineering Department Ankara Turkey
This paper explores essential aspects of autonomous underwater vehicle (AUV) design, focusing on hull structure, hydrodynamics, propulsion systems, and sensor integration. It also examines the role of underwater Simul...
详细信息
Representation of compound information in a truthful, coarse way forms the layout of the granular computing paradigm. In granular computing, the continuous variables are mapped into intervals to be utilized in the ext...
详细信息
A deep fusion model is proposed for facial expression-based human-computer Interaction ***,image preprocessing,i.e.,the extraction of the facial region from the input image is ***,the extraction of more discriminative...
详细信息
A deep fusion model is proposed for facial expression-based human-computer Interaction ***,image preprocessing,i.e.,the extraction of the facial region from the input image is ***,the extraction of more discriminative and distinctive deep learning features is achieved using extracted facial *** prevent overfitting,in-depth features of facial images are extracted and assigned to the proposed convolutional neural network(CNN)*** CNN models are then ***,the performance of each CNN model is fused to obtain the final decision for the seven basic classes of facial expressions,i.e.,fear,disgust,anger,surprise,sadness,happiness,*** experimental purposes,three benchmark datasets,i.e.,SFEW,CK+,and KDEF are *** performance of the proposed systemis compared with some state-of-the-artmethods concerning each *** performance analysis reveals that the proposed system outperforms the competitive methods in terms of various performance ***,the proposed deep fusion model is being utilized to control a music player using the recognized emotions of the users.
In the data retrieval process of the Data recommendation system,the matching prediction and similarity identification take place a major role in the *** that,there are several methods to improve the retrieving process...
详细信息
In the data retrieval process of the Data recommendation system,the matching prediction and similarity identification take place a major role in the *** that,there are several methods to improve the retrieving process with improved accuracy and to reduce the searching ***,in the data recommendation system,this type of data searching becomes complex to search for the best matching for given query data and fails in the accuracy of the query recommendation *** improve the performance of data validation,this paper proposed a novel model of data similarity estimation and clustering method to retrieve the relevant data with the best matching in the big data *** this paper advanced model of the Logarithmic Directionality Texture Pattern(LDTP)method with a Metaheuristic Pattern Searching(MPS)system was used to estimate the similarity between the query data in the entire *** overall work was implemented for the application of the data recommendation *** are all indexed and grouped as a cluster to form a paged format of database structure which can reduce the computation time while at the searching ***,with the help of a neural network,the relevancies of feature attributes in the database are predicted,and the matching index was sorted to provide the recommended data for given query *** was achieved by using the Distributional Recurrent Neural Network(DRNN).This is an enhanced model of Neural Network technology to find the relevancy based on the correlation factor of the feature *** training process of the DRNN classifier was carried out by estimating the correlation factor of the attributes of the *** are formed as clusters and paged with proper indexing based on the MPS parameter of similarity *** overall performance of the proposed work can be evaluated by varying the size of the training database by 60%,70%,and 80%.The parameters that are considered for performance analysis are Precision
暂无评论