检索结果-内蒙古大学图书馆

IEEE International conference on Image processing (ICIP)

作者： Neumann, David Sattler, Felix Kirchhoffer, Heiner Wiedemann, Simon Mueller, Karsten Schwarz, Heiko Wiegand, Thomas Marpe, Detlev Samek, Wojciech Fraunhofer Heinrich Hertz Inst Berlin Germany Free Univ Berlin Inst Comp Sci Berlin Germany Tech Univ Berlin Dept Telecommun Syst Berlin Germany

ISBN: (数字)9781728163956

ISBN: (纸本)9781728163956

An increasing number of distributed machine learning applications require efficient communication of neural network parameterizations. DeepCABAC, an algorithm in the current working draft of the emerging MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis, has demonstrated high compression gains for a variety of neural network models. In this paper we propose a method for employing DeepCABAC in a Federated Learning scenario for the exchange of intermediate differential parameterizations. Furthermore, we discuss the efficiency of DeepCABAC when compressing trained neural networks. Our experiments on large neural networks show that in both scenarios, DeepCABAC achieves competitive compression rates, without degrading the network accuracy.

关键词： neural networks Federated Learning neural network Compression CABAC distributed Training

来源：评论

学校读者我要写书评

暂无评论

Study on distributed Representation of Words with Sparse neural network Language Model 3

Study on Distributed Representation of Words with Sparse Neu...

引用

3rd IIAI International conference on Advanced Applied Informatics (IIAI-AAI)

作者： Yanagimoto, Hidekazu Osaka Prefecture Univ Sch Engn Osaka 5998531 Japan

ISBN: (纸本)9781479941735

These days a neural network is paid attention to again since it is improved as deep learning. Deep learning achieves good data representation according to a data distribution and get over state of the art classifiers in computer vision and speech recognition. The representation captures abstracts of many data and is used as general features to solve many types of classification problems. A neural network is applied to natural language processing, too. In natural language processing neural networks achieve the best distributed representation of words and many researchers pay attention to the neural network language model. The distributed representation allocates words in a continuous feature space and there are semantically or syntactically similar words near area in the space. Hence, the distributed representation contributes to a solution of analogical reasoning tasks. In this paper a sparse neural network language model (SNNLM) is used, which achieves sparse active neurons in the hidden layer and a distributed representation of words is obtained. In evaluational experiments SNNLM selects words that do not occur in the same sentence at all as related words and it is confirmed that the word selection is appropriate manually.

关键词： Natural language process neural network

来源：评论

学校读者我要写书评

暂无评论

Evaluation of MPI Allreduce for distributed Training of Convolutional neural networks 29

Evaluation of MPI Allreduce for Distributed Training of Conv...

引用

29th Euromicro International conference on Parallel, distributed and network-Based processing (PDP)

作者： Castello, Adrian Catalan, Mar Dolz, Manuel F. Mestre, Jose, I Quintana-Orti, Enrique S. Duato, Jose Univ Jaume 1 Castellon de La Plana Spain Univ Politecn Valencia Valencia Spain

ISBN: (纸本)9781665414555

Training deep neural networks is a costly procedure, often performed via sophisticated deep learning frameworks on clusters of computers. As faster processor technologies are integrated into these cluster facilities (eg., NVIDIA's graphics accelerators or Google's tensor processing units), the communication component of the training process rapidly becomes a performance bottleneck. In this paper, we offer a complete analysis of the key collective communication primitive for the distributed data- parallel training of convolutional network networks (CNNs) focused on three relevant instances of the Message Passing Interface (MPI): MPICH, OpenMPI. and IntelMPI. In addition, our experimental evaluation is extended to expose the practical impact of this collective primitive when the training is performed using TensorFlow+Horovod on a 16-node cluster. Finally, the theoretical analysis is further relined to a number of accelerated cluster configurations that are emulated by adjusting the communication-arithmetic ratio of the training process.

关键词： Message Passing Interface (MPI) collective communication primitives Allreduce deep learning distributed training convolutional neural networks

来源：评论

学校读者我要写书评

暂无评论

Parallel training of neural networks for finite element mesh decomposition

引用

COMPUTERS & STRUCTURES 1997年第4期63卷 693-707页

作者： Topping, BHV Khan, AI Bahreininejad, A Department of Mechanical and Chemical Engineering Heriot-Watt University Riccarton Edinburgh EH14 4AS U.K.

This paper describes a parallel processing implementation for neural computing and its application to finite element mesh decomposition. The parallelized neural network software developed is based on the public domain NASA developed program NETS 2.01, which is based on the back propagation algorithm of Rumelhart et al. [Learning internal representation by error propagation. In Parallel distributed processing: Explorations in the Microstructure of Cognition (Edited by D. E. Rummelhart and J. L. McClelland), Vol. 1: Foundations. MIT Press, MA (1986)]. The principal focus of this research concerns the parallel implementation. Comparisons between sequential and parallel versions are given. Finally a structural design problem concerned with finite element mesh generation is solved using the parallel neural network software. (C) 1997 Civil-Comp Ltd and Elsevier Science Ltd.

关键词： neural networks

来源：评论

学校读者我要写书评

暂无评论

dsODENet: neural ODE and Depthwise Separable Convolution for Domain Adaptation on FPGAs 30

dsODENet: Neural ODE and Depthwise Separable Convolution for...

引用

30th Euromicro International conference on Parallel, distributed and network-Based processing (PDP)

作者： Kawakami, Hiroki Watanabe, Hirohisa Sugiura, Keisuke Matsutani, Hiroki Keio Univ Dept ICS Kohoku Ku 3-14-1 Hiyoshi Yokohama Kanagawa 2238522 Japan

ISBN: (纸本)9781665469586

High-performance deep neural network (DNN)-based systems are in high demand in edge environments. Due to its high computational complexity, it is challenging to deploy DNNs on edge devices with strict limitations on computational resources. In this paper, we derive a compact while highly-accurate DNN model, termed dsODENet. by combining recently-proposed parameter reduction techniques: neural ODE (Ordinary Differential Equation) and DSC (Depthwise Separable Convolution). neural ODE exploits a similarity between ResNet and ODE, and shares most of weight parameters among multiple layers, which greatly reduces the memory consumption. We apply dsODENet to a domain adaptation as a practical use case with image classification datasets. We also propose a resource-efficient FPGA-based design for dsODENet, where all the parameters and feature maps except for pre- and post-processing layers can be mapped onto on-chip memories. It is implemented on Xffinx ZCII104 board and evaluated in terms of domain adaptation accuracy, training speed, FPGA resource utilization, and speedup rate compared to a software counterpart. The results demonstrate that dsODENet achieves comparable or slightly better domain adaptation accuracy compared to our baseline neural ODE implementation, while the total parameter size without pre- and post-processing layers is reduced by 54.2% to 79.8%. Our FPGA implementation accelerates the inference speed by 27.9 times.

关键词： Training Convolution neural networks Memory management Ordinary differential equations Software Mathematical models

来源：评论

学校读者我要写书评

暂无评论

Gaussian and exponential lateral connectivity on distributed spiking neural network simulation 26

Gaussian and exponential lateral connectivity on distributed...

引用

26th Euromicro International conference on Parallel, distributed, and network-Based processing (PDP)

作者： Pastorelli, Elena Paolucci, Pier Stanislao Simula, Francesco Biagioni, Andrea Capuani, Fabrizio Cretaro, Paolo De Bonis, Giulia Lo Cicero, Francesca Lonardo, Alessandro Martinelli, Michele Pontisso, Luca Vicini, Piero Ammendola, Roberto INFN Sez Roma Rome Italy Sapienza Univ Rome PhD Program Behav Neurosci Rome Italy INFN Sez Tor Vergata Rome Italy Univ Roma Tor Vergata Elect Engn Dept Rome Italy

ISBN: (纸本)9781538649756

We measured the impact of long-range exponentially decaying intra-areal lateral connectivity on the scaling and memory occupation of a distributed spiking neural network simulator compared to that of short-range Gaussian decays. While previous studies adopted short-range connectivity, recent experimental neurosciences studies are pointing out the role of longer-range intra-areal connectivity with implications on neural simulation platforms. Two-dimensional grids of cortical columns composed by up to 11 M point-like spiking neurons with spike frequency adaption were connected by up to 30 G synapses using short-and long-range connectivity models. The MPI processes composing the distributed simulator were run on up to 1024 hardware cores, hosted on a 64 nodes server platform. The hardware platform was a cluster of IBM NX360 M5 16-core compute nodes, each one containing two Intel Xeon Haswell 8-core E5-2630 v3 processors, with a clock of 2.40 G Hz, interconnected through an InfiniBand network, equipped with 4x QDR switches.

关键词： cortical simulation distributed computing spiking neural network lateral synaptic connectivity hardware/software co-design

来源：评论

学校读者我要写书评

暂无评论

Topic Classification Based on distributed Document Representation and Latent Topic Information 9

Topic Classification Based on Distributed Document Represent...

引用

9th Annual Summit and conference of the Asia-Pacific-Signal-and-Information-processing-Association (APSIPA ASC)

作者： Chen, Peixin Guo, Wu Wang, Qingnan Song, Yan Univ Sci & Technol China Natl Engn Lab Speech & Language Informat Proc Hefei Anhui Peoples R China

ISBN: (纸本)9781538615423

The classical bag-of-words and probabilistic topic models are widely used on topic classification tasks. Recently, neural networks have achieved remarkable performance and formed the mainstream, due to their ability to encode distributed semantic features of documents based on word embeddings. To demonstrate the superiority of neural networks, this paper compares Latent Dirichlet Allocation (LDA) with Convolutional neural network (CNN), Recurrent neural network (RNN) and Recurrent Convolutional neural network (RCNN), which are the mainstream neural network architectures. Beyond this, we combine the latent topic information inferred by LDA and distributed semantic information learned by neural networks to generate a better document representation for topic classification. The experimental results show that the proposed representation outperforms individual systems and can achieve excellent performance on topic classification tasks.

关键词： Task analysis neural networks Semantics Support vector machines Logistics Computer architecture Feature extraction

来源：评论

学校读者我要写书评

暂无评论

Intelligent Multi-Agent based Back-Propagation neural network forecasting model for statistical database anomaly prevention system

Intelligent Multi-Agent based Back-Propagation Neural Networ...

引用

International conference on Intelligent Sensing and Information processing

作者： Ramasubramian, P Kannan, A Anna Univ Dept Comp Engn & Sci Madras 25 Tamil Nadu India

ISBN: (纸本)0780382439

This paper describes a framework for highly distributed real-time monitoring approach to, Database Security using Intelligent Multi-Agents. The Statistical Anomaly Prevention system employs Back-Propagation neural network forecasting model, which predicts unauthorized invasions of user based on previous observations and takes further action before intrusion occurs. Our Back-Propagation neural network model makes periodic short-term forecasts, since long-term forecasts cannot accurately predict an intrusion. We use a multivariate time series technique to forecast the hacker's behavior effectively. Our Back-Propagation neural network model results have been compared with traditional statistical forecasting models, and a better prediction accuracy has been observed. In order to reduce single point of failures in centralized security system, a dynamic distributed system has been designed in which the security management task is distributed across the network using Intelligent Multi-Agents.

关键词： Intelligent Multi-Agents database security Back-Propagation neural network forecasting technique statistical database anomaly prevention

来源：评论

学校读者我要写书评

暂无评论

Consensus propagation

引用

IEEE TRANSACTIONS ON INFORMATION THEORY 2006年第11期52卷 4753-4766页

作者： Moallemi, Ciamac C. Van Roy, Benjamin Stanford Univ Dept Elect Engn Stanford CA 94305 USA Stanford Univ Dept Management Sci & Engn Stanford CA 94305 USA

We propose consensus propagation, an asynchronous distributed protocol for averaging numbers across a network. We establish convergence, characterize the convergence rate for regular graphs, and demonstrate that the protocol exhibits better scaling properties than painvise averaging, an alternative that has received much recent attention. Consensus propagation can be viewed as a special case of belief propagation, and our results contribute to the belief propagation literature. In particular, beyond singly-connected graphs, there are very few classes of relevant problems for which belief propagation is known to converge.

关键词： belief propagation distributed averaging distributed consensus distributed signal processing Gaussian Markov random fields max-product algorithm message-passing algorithms min-sum algorithm sum-product algorithm

来源：评论

学校读者我要写书评

暂无评论

Predicting the Price of Bitcoin Using Machine Learning 26

Predicting the Price of Bitcoin Using Machine Learning

引用

26th Euromicro International conference on Parallel, distributed, and network-Based processing (PDP)

作者： McNally, Sean Roche, Jason Caton, Simon Natl Coll Ireland Sch Comp Dublin 1 Ireland Dublin Business Sch Dublin 2 Ireland

ISBN: (纸本)9781538649756

The goal of this paper is to ascertain with what accuracy the direction of Bitcoin price in USD can be predicted. The price data is sourced from the Bitcoin Price Index. The task is achieved with varying degrees of success through the implementation of a Bayesian optimised recurrent neural network (RNN) and a Long Short Term Memory (LSTM) network. The LSTM achieves the highest classification accuracy of 52% and a RMSE of 8%. The popular ARIMA model for time series forecasting is implemented as a comparison to the deep learning models. As expected, the non-linear deep learning methods outperform the ARIMA forecast which performs poorly. Finally, both deep learning models are benchmarked on both a GPU and a CPU with the training time on the GPU outperforming the CPU implementation by 67.7%.

关键词： Bitcoin Deep Learning GPU Recurrent neural network Long Short Term Memory ARIMA

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：