Summary form only given, as follows. A new parallel distributedprocessing architecture called an entropy machine (EM) is proposed. This machine, which is based on an artificial neuralnetwork composed of massive neur...
详细信息
Summary form only given, as follows. A new parallel distributedprocessing architecture called an entropy machine (EM) is proposed. This machine, which is based on an artificial neuralnetwork composed of massive neurons and interconnections, is used for solving a variety of NP-complete optimization problems. The EM performs either the parallel distributed gradient descent method or gradient ascent method to search for minima or maxima.
Multichannel processing is widely used for speech enhancement but several limitations appear when trying to deploy these solutions in the real world. distributed sensor arrays that consider several devices with a few ...
详细信息
ISBN:
(纸本)9781509066315
Multichannel processing is widely used for speech enhancement but several limitations appear when trying to deploy these solutions in the real world. distributed sensor arrays that consider several devices with a few microphones is a viable solution which allows for exploiting the multiple devices equipped with microphones that we are using in our everyday life. In this context, we propose to extend the distributed adaptive node-specific signal estimation approach to a neuralnetwork framework. At each node, a local filtering is performed to send one signal to the other nodes where a mask is estimated by a neuralnetwork in order to compute a global multichannel Wiener filter. In an array of two nodes, we show that this additional signal can be leveraged to predict the masks and leads to better speech enhancement performance than when the mask estimation relies only on the local signals.
While Attention has come to be an important mechanism in deep learning, there remains limited intuition for why it works so well. Here, we show that Transformer Attention can be closely related under certain data cond...
详细信息
ISBN:
(纸本)9781713845393
While Attention has come to be an important mechanism in deep learning, there remains limited intuition for why it works so well. Here, we show that Transformer Attention can be closely related under certain data conditions to Kanerva's Sparse distributed Memory (SDM), a biologically plausible associative memory model. We confirm that these conditions are satisfied in pre-trained GPT2 Transformer models. We discuss the implications of the Attention-SDM map and provide new computational and biological interpretations of Attention.
Incorporating machine learning into automatic performance analysis and tuning tools is a promising path to tackle the increasing heterogeneity of current HPC applications. However, this introduces the need for generat...
详细信息
ISBN:
(纸本)9781665414555
Incorporating machine learning into automatic performance analysis and tuning tools is a promising path to tackle the increasing heterogeneity of current HPC applications. However, this introduces the need for generating balanced and representative datasets of parallel applications' executions. This work proposes a methodology for building datasets OpenMP parallel code regions patterns. It allows for determining whether a given code region covers a unique part of the pattern input space not covered by the patterns already included in the dataset. The proposed methodology uses hardware performance counter;to represent the execution of the region. which is referred to as the region signature for a given number of cores. Then, a complete representation of the region is built by joining the signatures for every different thread configuration in the system. Next, correlation analysis is performed between this representation and the representation of all the patterns already in the training set. Finally, H . this correlation is helm a given threshold. the region is considered to cover a unique part of the pattern input space and is subsequently added to the dataset. For validating this methodology, an example dataset, obtained from well known benchmarks, has been used to train a carefully designed neuralnetwork model to demonstrate that it is able to classify . different patterns of OpenMP parallel regions.
The increasing amount of data to be processed coming from multiple sources, as in the case of sensor networks, and the need to cope with constraints of security and privacy, make necessary the use of computationally e...
详细信息
ISBN:
(纸本)9781538618950
The increasing amount of data to be processed coming from multiple sources, as in the case of sensor networks, and the need to cope with constraints of security and privacy, make necessary the use of computationally efficient techniques on simple and cheap hardware architectures often distributed in pervasive scenarios. Random Vector Functional-Link is a neuralnetwork model usually adopted for processingdistributed big data, but no constraints have been considered so far to deal with limited hardware resources. This paper is focused on implementing a modified version of the Random Vector Functional-Link network with finite precision arithmetic, in order to make it suited to hardware architectures even based on a simple microcontroller. A genetic optimization is also proposed to ensure that the overall performance is comparable with standard software implementations. The numerical results prove the efficacy of the proposed approach.
distributed Denial-of-Service (DDoS) attacks are serious threats to a smart grid infrastructure services' availability, and can cause massive blackouts. This study describes an anomaly detection method for improvi...
详细信息
ISBN:
(纸本)9781538633601
distributed Denial-of-Service (DDoS) attacks are serious threats to a smart grid infrastructure services' availability, and can cause massive blackouts. This study describes an anomaly detection method for improving the detection rate of a DDoS attack in a smart grid. This improvement was achieved by increasing the classification of the training and testing phases in a convolutional neuralnetwork (CNN). An improved version of the variance fractal dimension trajectory (VFDTv2) was used to extract inherent features from the non-pure fractal input data. A discrete wavelet transform (DWT) was applied to the input data and the VFDTv2 to extract distinguishing features during data pre-processing. A support vector machine (SVM) was used for post data-processing. The implementation detected the DDoS attack with 87.35% accuracy.
We study acceleration for distributed sparse regression in high-dimensions, which allows the parameter size to exceed and grow faster than the sample size. When applicable, existing distributed algorithms employing ac...
详细信息
ISBN:
(纸本)9781713871088
We study acceleration for distributed sparse regression in high-dimensions, which allows the parameter size to exceed and grow faster than the sample size. When applicable, existing distributed algorithms employing acceleration perform poorly in this setting, theoretically and numerically. We propose a new accelerated distributed algorithm suitable for high-dimensions. The method couples a suitable instance of accelerated Nesterov's proximal gradient with consensus and gradient-tracking mechanisms, aiming at estimating locally the gradient of the empirical loss while enforcing agreement on the local estimates. Under standard assumptions on the statistical model and tuning parameters, the proposed method is proved to globally converge at linear rate to an estimate that is within the statistical precision of the model. The iteration complexity scales as O(root kappa), while the communications per iteration are at most (O) over tilde (log m/(1-rho)), where kappa is the restricted condition number of the empirical loss, m is the number of agents, and rho is an element of[0,1) measures the network connectivity. As by-product of our design, we also report an accelerated method for high-dimensional estimations over master-worker architectures, which is of independent interest and compares favorably with existing works.
In this paper we introduce federated learning (FL) based resource allocation (RA) for wireless communication networks, where users cooperatively train a RA policy in a distributed scenario. The RA policy for each user...
详细信息
ISBN:
(纸本)9789082797091
In this paper we introduce federated learning (FL) based resource allocation (RA) for wireless communication networks, where users cooperatively train a RA policy in a distributed scenario. The RA policy for each user is represented by a local deep neuralnetwork (DNN), which has the same structure for all users. Each DNN monitors local measurements and outputs a power allocation to the user. The proposed approach is model-free;each user is responsible for training its own DNN to maximize the sum rate (SR) and communicates with the server to aggregate its local DNN with other DNNs. More importantly, each user needs to probe only its own data rate as a distributed reward function and communications with the server once in a while. Simulations show that the proposed approach enables conventional deep learning (DL) based RA methods to not only use their policy in a distributed scenario, but also to (re)train their policy in time-varying environments in a model-free distributed manner without needing a computationally complex server.
Many belief networks have been proposed that are composed of binary units. However, for tasks such as object and speech recognition which produce real-valued data, binary network models are usually inadequate. Indepen...
详细信息
ISBN:
(纸本)0262112450
Many belief networks have been proposed that are composed of binary units. However, for tasks such as object and speech recognition which produce real-valued data, binary network models are usually inadequate. Independent component analysis (ICA) learns a model from real data, but the descriptive power of this model is severly limited. We begin by describing the independent factor analysis (IFA) technique, which overcomes some of the limitations of ICA. We then create a multilayer network by cascading singlelayer IFA models. At each level, the IFA network extracts real-valued latent variables that are non-linear functions of the input data with a highly adaptive functional form, resulting in a hierarchical distributed representation of these data. Whereas exact maximum-likelihood learning of the network is intractable, we derive an algorithm that maximizes a lower bound on the likelihood, based on a variational approach.
In wireless multi-hop networks, delay is an important metric for many applications. However, the max-weight scheduling algorithms in the literature typically focus on instantaneous optimality, in which the schedule is...
详细信息
ISBN:
(纸本)9781665405409
In wireless multi-hop networks, delay is an important metric for many applications. However, the max-weight scheduling algorithms in the literature typically focus on instantaneous optimality, in which the schedule is selected by solving a maximum weighted independent set (MWIS) problem on the interference graph at each time slot. These myopic policies perform poorly in delay-oriented scheduling, in which the dependency between the current backlogs of the network and the schedule of the previous time slot needs to be considered. To address this issue, we propose a delay-oriented distributed scheduler based on graph convolutional networks (GCNs). In a nutshell, a trainable GCN module generates node embeddings that capture the network topology as well as multi-step lookahead backlogs, before calling a distributed greedy MWIS solver. In small- to medium-sized wireless networks with heterogeneous transmit power, where a few central links have many interfering neighbors, our proposed distributed scheduler can outperform the myopic schedulers based on greedy and instantaneously optimal MWIS solvers, with good generalizability across graph models and minimal increase in communication complexity.
暂无评论