As we can see, in the era of high-speed internet and the increasing use of technologies, the proper and safe management of IP Addresses is of paramount importance. De- centralized networks' older methods of distri...
详细信息
Artificial Intelligence (AI) and Deep Learning in particular have increasing computational requirements, with a corresponding increase in energy consumption. There is a tremendous opportunity to reduce the computation...
详细信息
ISBN:
(纸本)9781665497473
Artificial Intelligence (AI) and Deep Learning in particular have increasing computational requirements, with a corresponding increase in energy consumption. There is a tremendous opportunity to reduce the computational cost and environmental impact of deep learning by accelerating neural network architecture search and hyperparameter optimization, as well as explicitly designing neural architectures that optimize for both energy efficiency and performance. Here, we introduce a framework called training performance estimation (TPE), which builds upon existing techniques for training speed estimation in order to monitor energy consumption and rank model performance-without training models to convergence-saving up to 90% of time and energy of the full training budget. We benchmark TPE in the computationally intensive, well-studied domain of computer vision and in the emerging field of graph neural networks for machine-learned inter-atomic potentials, an important domain for scientific discovery with heavy computational demands. We propose variants of early stopping that generalize this common regularization technique to account for energy costs and study the energy costs of deploying increasingly complex, knowledge-informed architectures for AI-accelerated molecular dynamics and image classification. Our work enables immediate, significant energy savings across the entire pipeline of model development and deployment and suggests new research directions for energy-aware, knowledge-informed model architecture development.
The profound impact of recent developments in artificial intelligence is unquestionable. The applications of deep learning models are everywhere, from advanced natural language processing to highly accurate prediction...
详细信息
ISBN:
(数字)9781665451550
ISBN:
(纸本)9781665451550
The profound impact of recent developments in artificial intelligence is unquestionable. The applications of deep learning models are everywhere, from advanced natural language processing to highly accurate prediction of extreme weather. Those models have been continuously increasing in complexity, becoming much more powerful than their original versions. In addition, data to train the models is becoming more available as technological infrastructures sense and collect more readings. Consequently, distributed deep learning training is often times necessary to handle intricate models and massive datasets. Running a distributed training strategy on a supercomputer exposes the models to all the considerations of a large-scale machine;reliability is one of them. As supercomputers integrate a colossal number of components, each fabricated on an ever decreasing feature-size, faults are common during execution of programs. A particular type of fault, silent data corruption, is troublesome because the system does not crash and does not immediately give an evident sign of an error. We set out to explore the effects of that type of faults by inspecting how distributed deep learning training strategies cope with bit-flips that affect their internal data structures. We used checkpoint alteration, a technique that permits the study of this phenomenon on different distributed training platforms and with different deep learning frameworks. We evaluated two distributed learning libraries (distributed Data Parallel and Horovod) and found out Horovod is slightly more resilient to SDCs. However, fault propagation is similar in both cases, and the model is more sensitive to SDCs than the optimizer.
In vehicular networks, vehicle in the platooning relies on dissemination of beacons to perceive the status of neighbor vehicles and then take control low to maintain a constant inter-vehicle distance. Vehicle platooni...
详细信息
This paper proposes a novel offshore grid architecture that interconnects wind turbines, hybrid storage systems consisting of fuel cells, lithium-ion batteries, and an HVDC onshore grid to power subsea electrical load...
详细信息
This research explores the impact of photo distortion on face recognition algorithms performance, focusing on convolutional neural networks (CNNs) due to their superior efficiency. Through a comprehensive review of cu...
详细信息
Membrane proteins make up around 30% of all proteins in a cell. These proteins are difficult to evaluate due to their hydrophobic surface and dependence on their original in vivo environment. There is a tremendous dem...
详细信息
With the continuous progress of big data, Internet of Things, artificial intelligence and other technologies, we are living in an era of information explosion. A large amount of data is constantly recorded, and this d...
详细信息
The performance of P300-based Brain-computer Interface (BCI) applications is highly dependent on both the quality and quantity of the recorded Electroencephalography (EEG) signals. As recording extended datasets from ...
详细信息
ISBN:
(数字)9781665467704
ISBN:
(纸本)9781665467704
The performance of P300-based Brain-computer Interface (BCI) applications is highly dependent on both the quality and quantity of the recorded Electroencephalography (EEG) signals. As recording extended datasets from users for calibration is often a difficult and tedious task, data augmentation can be used to help supplement the training data for machine learning classifiers that are typically used in P300-based BCI applications. In this paper, we analyze and compare the performance of three different generative adversarial networks (GANs) as data augmentation techniques; namely, deep convolutional GAN (DCGAN), conditional GAN (cGAN), and the auxiliary classifier GAN (ACGAN). We first investigated the effect of increasing the training data size using each of these GANs on the performance of P300 classification. Our results revealed that the cGAN increased the classification accuracy by up to 18% relative to the baseline data under the best conditions. We also investigated the effect of decreasing the training data size and compensating for the reduced data size using data generated from the GANs. Our analysis indicated that the training data size could be reduced by ~30% while maintaining the accuracy on par with the baseline accuracy. These results demonstrate the utility of GANs in addressing the challenges associated with the limited data typically available for BCI applications.
Neural networks are being used successfully to solve classification problems, e.g., for detecting objects in images. It is well known that neural networks are susceptible if small changes applied to their input result...
详细信息
ISBN:
(纸本)9783031274800;9783031274817
Neural networks are being used successfully to solve classification problems, e.g., for detecting objects in images. It is well known that neural networks are susceptible if small changes applied to their input result in misclassification. Situations in which such a slight input change, often hardly noticeable by a human expert, results in a misclassification are called adversarial examples. If such inputs are used for adversarial attacks, they can be life-threatening if, for example, they occur in image classification systems used in autonomous cars or medical diagnosis. systems employing neural networks, e.g., for safety or security-critical functionality, are a particular challenge for formal verification, which usually expects a formal specification (e.g., given as source code in a programming language for which a formal semantics exists). Such a formal specification does, per se, not exist for neural networks. In this paper, we address this challenge by presenting a formal embedding of feedforward neural networks into Isabelle/HOL and discussing desirable properties for neural networks in critical applications. Our Isabelle-based prototype can import neural networks trained in TensorFlow, and we demonstrate our approach using a neural network trained for the classification of digits on a dot-matrix display.
暂无评论