检索结果-内蒙古大学图书馆

distributed Deep Learning With GPU-FPGA Heterogeneous Computing

IEEE MICRO 2021年第1期41卷 15-22页

作者： Tanaka, Kenji Arikawa, Yuki Ito, Tsuyoshi Morita, Kazutaka Nemoto, Naru Terada, Kazuhiko Teramoto, Junji Sakamoto, Takeshi NTT Corp NTT Device Technol Labs Atsugi Kanagawa 2430198 Japan NTT Corp NTT Software Innovat Ctr Tokyo 1808585 Japan

In distributed deep learning (DL), collective communication algorithms, such as Allreduce, used to share training results between graphical processing units (GPUs) are an inevitable bottleneck. We hypothesize that a cache access latency occurred at every Allreduce is a significant bottleneck in the current computational systems with high-bandwidth interconnects for distributed DL. To reduce this frequency of latency, it is important to aggregate data at the network interfaces. We implement a data aggregation circuit in a field-programmable gate array (FPGA). Using this FPGA, we proposed novel Allreduce architecture and training strategy without accuracy degradation. Results of the measurement show Allreduce latency reduction to 1/4. Our system can also conceal about 90% of the communication overhead and improve scalability by 20%. The end-to-end time consumed for training in distributed DL with ResNet-50 and ImageNet is reduced to 87.3% without any degradation in validation accuracy.

关键词： Cache Storage Data Aggregation Deep Learning Artificial Intelligence Field Programmable Gate Arrays Graphics processing Units neural Chips distributed Deep Learning GPU FPGA Heterogeneous Computing Collective Communication Algorithms Graphical processing Units Cache Access Latency High Bandwidth Interconnects network Interfaces Field Programmable Gate Array Novel Allreduce Architecture Communication Overhead distributed DL Training Strategy Allreduce Latency Reduction Res Net 50 Image Net Field Programmable Gate Arrays Random Access Memory Graphics processing Units Training Data distributed Databases Bandwidth Data Aggregation Deep Learning

来源：评论

学校读者我要写书评

暂无评论

Power Microservices Troubleshooting by Pretrained Language Model with Multi-source Data 22

Power Microservices Troubleshooting by Pretrained Language M...

引用

22nd IEEE International Symposium on Parallel and distributed processing with Applications, ISPA 2024

作者： Ding, Shuai Xu, Yifei Lu, Zhuang Tang, Fan Li, Tong Ge, Jingguo Chinese Academy of Sciences Key Laboratory of Cyberspace Security Defense Institute of Information Engineering Beijing China University of Chinese Academy of Sciences School of Cyber Security Beijing China Information Center Aier Eye Hospital Hunan China

ISBN: (纸本)9798331509712

Microservice has become the mainstream paradigm for developing cloud-native applications, but the intricate interdependencies between microservices and the vast amount of heterogeneous observable data (i.e. metrics, logs and traces) pose challenges for rapid troubleshooting. Several anomaly detection and root cause localization approaches that integrate multi-source data have been proposed. However, they are plagued with issues such as scarcity of high-quality data and insufficient model generalization. This is particularly evident when domain-specific models are trained from scratch for specific tasks. Recently, Large Language Models (LLMs) have shown outstanding capabilities in time series analysis, due to multi-source data generated by distributed microservices exhibit intrinsic spatio-temporal characteristics. In view of this, we propose LLM4MST, an LLM-empowered microservice troubleshooting model. We first unify and represent multi-source data by extracting service invocation graphs, and model dependencies between microservices by using a message-passing based graph neural network to generate graph-level sequences. The graph-level representation is then aligned with the LLM, and the LLM is fine-tuned to capture complex spatio-temporal patterns, generating a global vector that represents the state of microservice system within a timeslot. LLM4MST achieves accurate anomaly detection and root cause localization by jointly training the end-to-end model. Experiments on real datasets show that LLM4MST exhibits excellent performance in both full-sample and few-shot scenarios, demonstrating the powerful ability of LLMs in cross-domain knowledge transfer and few-shot learning. © 2024 IEEE.

关键词： Graph neural networks

来源：评论

学校读者我要写书评

暂无评论

Hardware-aware Partitioning of Convolutional neural network Inference for Embedded AI Applications 18

Hardware-aware Partitioning of Convolutional Neural Network ...

引用

18th Annual International Conference on distributed Computing in Sensor Systems (DCOSS)

作者： Kress, Fabian Hoefer, Julian Hotfilter, Tim Walter, Iris Sidorenko, Vladimir Harbaum, Tanja Becker, Juergen Karlsruhe Inst Technol Karlsruhe Germany

ISBN: (纸本)9781665495127

Embedded image processing applications like multicamera-based object detection or semantic segmentation are often based on Convolutional neural networks (CNNs) to provide precise and reliable results. The deployment of CNNs in embedded systems, however, imposes additional constraints such as latency restrictions and limited energy consumption in the sensor platform. These requirements have to be considered during hardware/software co-design of embedded Artifical Intelligence (AI) applications. In addition, the transmission of uncompressed image data from the sensor to a central edge node requires large bandwidth on the link, which must also be taken into account during the design phase. Therefore, we present a simulation toolchain for fast evaluation of hardware-aware CNN partitioning for embedded AI applications. This approach explores an efficient workload distribution between sensor nodes and a central edge node. Neither processing all layers close to the sensor nor transmitting all uncompressed raw data to the edge node is an optimal solution for each use case. Hence, our proposed simulation toolchain evaluates power and performance metrics for each reasonable partitioning point in a CNN. In contrast to the state of the art, our approach does not only consider the neural network architecture. In the evaluation, our simulation toolchain additionally takes into account hardware components such as special accelerators and memories that are implemented in the sensor node. Exemplary, we show the simulation results for three commonly used CNNs in embedded systems. Thereby, we identify advantageous partitioning points regarding inference latency and energy consumption. With the support of the toolchain, we are able to identify three beneficial partitioning points for FCN ResNet-50 and two for GoogLeNet as well as for SqueezeNet V1.1.

关键词： convolutional neural networks embedded systems hardware accelerator simulation toolchain hardware/software co-design

来源：评论

学校读者我要写书评

暂无评论

1-ADM-CNN: A Lightweight In-field Compression Method for Seismic Data

引用

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS 2022年第12期69卷 5164-5168页

作者： Iqbal, Naveed King Fand Univ Petr & Minerals Dept Elect Engn Dhahran 31261 Saudi Arabia King Fand Univ Petr & Minerals Ctr Energy & Geo Proc Dhahran 31261 Saudi Arabia

Large-scale seismic acquisition, versatility, flexibility, automation, and scalability are the objectives of future oil and gas exploration technology. An example of emerging technology for seismic monitoring is distributed acoustic sensing (DAS). The significant amount of data produced by DAS is a challenge that necessitates the development of new technologies for its efficient handling and processing. A typical seismic survey, on the one hand, can generate hundreds of terabytes of raw seismic data per day. The demand for wireless seismic data transmission, on the other hand, remains enormous. The massive amount of data transmission from geophones to the on-site data collection center and its storage poses significant challenges. A lightweight compression procedure is required in order to reduce the data traffic and the storage size at the data center without putting an extra burden on a geophone. In this brief, an efficient implementation of a 1D convolutional neural network (CNN) together with 1-bit adaptive delta modulation is presented for in-field seismic data compression. It is worth mentioning here that the training of CNN is done offline on synthetic data set and hence, the proposed approach has potential for real-time implementation. Furthermore, no assumption on the underlying statistics of noise or the seismic signal is imposed and consequently, the proposed method is suitable for a wide range of seismic data. Furthermore, the proposed method works in the time-domain, unlike existing transform-domain methods, making it suitable for quick diagnosis of bad traces at the data center. Simulation results with real data set reveal that the proposed approach achieves a signal-to-noise ratio (SNR) of approximately 30 dB with a compression gain of 35: 1. Finally, significant superiority in terms of compression gain and reconstruction quality is demonstrated when compared to the existing methods.

关键词： Delta modulation convolutional neural network compression distributed acoustic sensing

来源：评论

学校读者我要写书评

暂无评论

Power Line Partial Discharge Detection Using Multi-Scale 1D Convolutional neural networks 5

Power Line Partial Discharge Detection Using Multi-Scale 1D ...

引用

5th International Conference on Power and Energy Technology, ICPET 2023

作者： Jiang, Yuliang Xu, Zhenhuan Fang, Dan Zhang, Guoliang Zhi, Bihen Wang, Bing State Grid Shanghai Electric Power Corporation Training Center Shanghai China Nanjing Haoqing Information Technology Co. Ltd Nanjing China

ISBN: (纸本)9798350339673

In recent years, due to the widespread access of distributed power supply, the topology of electrical network becomes more and more complex such that it is difficult to analyze the fault causes of transmission lines, and partial discharge (PD) is a type of failure that often occurs. An automated transmission lines PD detection system requires automated feature extraction and online identification of PD. Firstly, A PD detection method based on multi-scale one dimensional convolutional neural network (MS-Conv1D) is proposed in this paper, which takes an arbitrary-long time series of fault signals as input. Furthermore, the single-phase and three-phase electrical signals can be dealt with an unified structure of neural network. Secondly, a complete flow of training MS-Conv1D and data processing is given. Finally, simulations on a real PD dataset shows that the detection accuracy of MS-Conv1D reaches 90% with 20MHz sampling rate, and the receiver operating characteristic (ROC) points indicate the robustness of MS-Conv1D at different frequencies. © 2023 IEEE.

关键词： automated feature extraction convolutional neural network fault diagnosis partial discharge

来源：评论

学校读者我要写书评

暂无评论

BIRDSONG OF COMMON BIRDS IN AN URBAN SOUNDSCAPE AS EVALUATED WITH RECURRENT neural networkS 10

BIRDSONG OF COMMON BIRDS IN AN URBAN SOUNDSCAPE AS EVALUATED...

引用

10th Convention of the European Acoustics Association, EAA 2023

作者： Devos, Paul Department of Information Technology WAVES Research Group Ghent University Belgium

ISBN: (纸本)9788888942674

Birds are among the most prominent vocalizing animals, making most of them easy to observe and identify. In urban areas the adaptation of wildlife to human presence results from feeding and safety opportunities arising from different human activities. In the rich bird community, common birds are a subset of birds species that are strongly adapted to human activity and are regularly observed in urban areas. In studying the presence of these birds machinal song recognition is a desired tool for automatic processing of outdoor recordings. Improved neural network architectures offer flexible architectures for detecting temporal sequences as is the case in stereotype song patterns. Recurrent neural networks (RNNs) are a group of networks that can focus on these temporal patterns. The use of a LSTM network and other RNN variants, working in a supervised learning paradigm, is successfuly trained on environmental recordings to detect the singing of a target bird species. The use of such lightweight models is expected to provide presence and activity information of common bird species. © 2023 Paul Devos This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

关键词： Birds

来源：评论

学校读者我要写书评

暂无评论

Quality evaluation of airfoil hybrid mesh based on graph neural network 19th

Quality evaluation of airfoil hybrid mesh based on graph neu...

引用

19th Chinese Intelligent Systems Conference, CISC 2023

作者： Wang, Huaiqing Pang, Yufei Xiao, Sumei Wang, Zhichao School of Manufacturing Science and Engineering Southwest University of Science and Technology Mianyang621010 China Computational Aerodynamics Istitute China Aerodynamics Research and Development Center Mianyang621010 China Science and Technology on Parallel and Distributed Processing Laboratory National University of Defense Technology Changsha410073 China

ISBN: (纸本)9789819968817

In airfoil numerical simulation, the mesh quality has an important influence on the accuracy and error of numerical simulation. The existing mesh quality evaluation requires a lot of manual interaction, which greatly reduces the efficiency of mesh generation and necessitates the implementation of intelligent mesh evaluation methods. Graph neural networks can extract features from graph data, possess self-adaptability and generalization ability, and have been successfully applied in many industries. In this paper, we propose a deep graph neural network, SDeepNet, to evaluate mesh quality and construct a large-scale mixed mesh dataset, MixSet, for training and validating the model. We test and compare the performance of the mesh quality evaluation models GridNet, GMeshNet, and SDeepNet on the mesh dataset MixSet. The experimental results show that the SDeepNet model can achieve high accuracy and recall in the mixed mesh quality evaluation task. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023.

关键词： Mesh generation

来源：评论

学校读者我要写书评

暂无评论

Delta Activation Layer exploits temporal sparsity for efficient embedded video processing

Delta Activation Layer exploits temporal sparsity for effici...

引用

IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) / IEEE World Congress on Computational Intelligence (IEEE WCCI) / International Joint Conference on neural networks (IJCNN) / IEEE Congress on Evolutionary Computation (IEEE CEC)

作者： Yousefzadeh, Amirreza Sifalakis, Manolis IMEC Eindhoven Netherlands

ISBN: (数字)9781728186719

ISBN: (纸本)9781728186719

Activation sparsity can improve compute efficiency and resource utilization in sparsity-aware neural network accelerators. While spatial sparsification of activations is a popular topic in DNN literature, introducing and exploiting spatio-temporal sparsity is a topic much less explored in DNN literature. However, it is in perfect resonance with the trend in DNNs, to shift from static-data signal processing (e.g., image processing) to stream-data real-time signal processing (e.g., video and audio) in embedded edge devices. Towards the goal of exploiting temporal sparsity, in this paper, we introduce a new DNN layer (called Delta Activation Layer), whose sole purpose is to promote both spatial as well as temporal sparsity of activations during training with time-distributed data. One may use the Delta Activation Layer either during vanilla training or during a refinement phase. We have implemented the Delta Activation Layer as an extension of the standard TensorFlow-Keras (2.0) library and applied it on training deep neural networks with several datasets including the Human Action Recognition (UCF101) dataset. We report an almost 3x improvement of activation sparsity, with recoverable loss of model accuracy after prolonged training. All source code for this project is available upon request for academic purposes.

关键词： Training Image processing neural networks Streaming media Signal processing Market research Real-time systems

来源：评论

学校读者我要写书评

暂无评论

Clustering and Evolution of International Sports Field Based on Multi-sensor Fusion Technology

Clustering and Evolution of International Sports Field Based...

引用

作者： Yuan, Jun JiangXi Police Institute JiangXi China

Based on 549 literatures with the theme of "sports artificial intelligence" and other keywords in web of science database since 1995, this paper uses CiteSpace V software for visualization processing and analysis, and combs the country, discipline distribution, research hotspots and evolution trend of sports artificial intelligence research in recent 25 years by means of visual knowledge mapping, and discusses its research progress and development direction. 1) The research area of sports artificial intelligence is widely distributed, among which the United States, China and Germany are in the leading position. 2) Sports artificial intelligence research involves many disciplines, mainly using and learning from the research methods and theoretical perspectives of computer science, engineering, sports science and other disciplines. 3) The frequency and centrality of keywords confirm that machine learning is the main direction in the field of sports artificial intelligence, artificial neural network is the main algorithm, and data mining is the basis of practice and research. 4) Research hotspots include simple activity recognition and energy consumption research based on wearable accelerometer technology;action analysis and damage prevention and control research based on wearable sensor;computer vision scene classification research based on convolution neural network algorithm;analysis and prediction of physical fitness and technology and tactics based on computer vision;human posture recognition technology based on computer Deep learning. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： neural networks

来源：评论

学校读者我要写书评

暂无评论

Uncovering the encoding and transmission of behaviourally relevant information in neural activity

Uncovering the encoding and transmission of behaviourally re...

引用

作者： Celotto, Marco University of Bologna

Most cognitive functions require the encoding and routing of information across distributed networks of brain regions. Information propagation is typically attributed to physical connections existing between brain regions, and contributes to the formation of spatially correlated activity patterns, known as functional connectivity. While structural connectivity provides the anatomical foundation for neural interactions, the exact manner in which it shapes functional connectivity is complex and not yet fully understood. Additionally, traditional measures of directed functional connectivity only capture the overall correlation between neural activity, and provide no insight on the content of transmitted information, limiting their ability in understanding neural computations underlying the distributed processing of behaviorally-relevant variables. In this work, we first study the relationship between structural and functional connectivity in simulated recurrent spiking neural networks with spike timing dependent plasticity. We use established measures of time-lagged correlation and overall information propagation to infer the temporal evolution of synaptic weights, showing that measures of dynamic functional connectivity can be used to reliably reconstruct the evolution of structural properties of the network. Then, we extend current methods of directed causal communication between brain areas, by deriving an information-theoretic measure of Feature-specific Information Transfer (FIT) quantifying the amount, content and direction of information flow. We test FIT on simulated data, showing its key properties and advantages over traditional measures of overall propagated information. We show applications of FIT to several neural datasets obtained with different recording methods (magneto and electro-encephalography, spiking activity, local field potentials) during various cognitive functions, ranging from sensory perception to decision making and motor learning. Overall,

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：