检索结果-内蒙古大学图书馆

A Dynamic Sliding Window Based Tensor Communication Scheduling Framework for distributed Deep Learning

IEEE TRANSACTIONS ON network SCIENCE AND ENGINEERING 2025年第2期12卷 1080-1095页

作者： Gao, Yunqi Hu, Bing Mashhadi, Mahdi Boloursaz Wang, Wei Tafazolli, Rahim Debbah, Merouane Zhejiang Univ Hangzhou 310027 Peoples R China Univ Surrey 5GIC& 6G Inst Commun Syst ICS Guildford GU2 7XH England Khalifa Univ KU 6G Res Ctr Dept Comp & Informat Engn Abu Dhabi 127788 U Arab Emirates

Simultaneous tensor communication can effectively improve the scalability of distributed deep learning on large clusters. However, a fixed number of tensor blocks communicated concurrently violates the priority-based scheduling strategy and cannot minimize communication overheads. In this paper, we propose a novel simultaneous tensor communication framework, namely D-Credit, which transmits tensor blocks based on dynamic sliding windows to minimize per-iteration time in distributed DNN training. We build the mathematical model of D-Credit in two phases: (1) the overlap of gradient communication and backward propagation, and (2) the overlap of gradient communication and forward computation. We drive the optimal window sizes for the second phase analytically, and develop a greedy algorithm to efficiently determine the dynamic window sizes for the first phase of D-Credit. We implement the D-Credit architecture on PyTorch framework. Experimental results on two different GPU clusters demonstrate that at training speed, D-Credit can achieve up to 1.26x, 1.21x, 1.48x and 1.53x speedup compared to ByteScheduler, DeAR, PyTorch-DDP and WFBP, respectively. At energy consumption, D-Credit saves up to 17.8% and 25.1% of the training energy consumption compared to ByteScheduler and WFBP, respectively.

关键词： Tensors Training Processor scheduling Parallel processing Dynamic scheduling Computational modeling Artificial neural networks Computer architecture Mathematical models Energy consumption distributed deep learning data parallelism communication scheduling tensor partitioning generative pre-trained transformer (GPT)

来源：评论

学校读者我要写书评

暂无评论

UMPIPE: Unequal Microbatches-Based Pipeline Parallelism for Deep neural network Training

引用

IEEE TRANSACTIONS ON PARALLEL AND distributed SYSTEMS 2025年第2期36卷 293-307页

作者： Zhou, Guangyao Tian, Wenhong Buyya, Rajkumar Wu, Kui Southwest Jiaotong Univ Sch Comp & Artificial Intelligence Chengdu 610032 Sichuan Peoples R China Univ Elect Sci & Technol China Sch Informat & Software Engn Chengdu 610056 Sichuan Peoples R China Univ Melbourne Dept Comp & Informat Syst Cloud Comp & Distributed Syst CLOUDS Lab Melbourne Vic 3052 Australia Univ Victoria Dept Comp Sci Victoria BC V8P 5C2 Canada

The increasing need for large-scale deep neural networks (DNN) has made parallel training an area of intensive focus. One effective method, microbatch-based pipeline parallelism (notably GPipe), accelerates parallel training in various architectures. However, existing parallel training architectures normally use equal data partitioning (EDP), where each layer's process maintains identical microbatch-sizes. EDP may hinder training speed because different processes often require varying optimal microbatch-sizes. To address this, we introduce UMPIPE, a novel framework for unequal microbatches-based pipeline parallelism. UMPIPE enables unequal data partitions (UEDP) across processes to optimize resource utilization. We develop a recurrence formula to calculate the time cost in UMPIPE by considering both computation and communication processes. To further enhance UMPIPE's efficiency, we propose the Dual-Chromosome Genetic Algorithm for UMPIPE (DGAP) that accounts for the independent time costs of forward and backward propagation. Furthermore, we present TiDGAP, a two-level improvement on DGAP. TiDGAP accelerates the process by simultaneously calculating the end time for multiple individuals and microbatches using matrix operations. Our extensive experiments validate the dual-chromosome strategy's optimization benefits and TiDGAP's acceleration capabilities. TiDGAP can achieve better training schemes than baselines, such as the local greedy algorithm and the global greedy-based dynamic programming. Compared to (GPipe, PipeDream), UMPIPE achieves increases in training speed: (13.89,11.09)% for GPT1-14, (17.11,7.96)% for VGG16 and >=(170,100)% for simulation networks.

关键词： Training Parallel processing Pipelines Data models Computational modeling Genetic algorithms Costs Pipeline processing Tensors Transformers Parallel training data parallelism unequal data partitions pipeline parallelism genetic algorithm

来源：评论

学校读者我要写书评

暂无评论

Next-generation IIoT security: Comprehensive comparative analysis of CNN-based approaches

引用

KNOWLEDGE-BASED SYSTEMS 2025年 316卷

作者： Dong, Huiyao Kotenko, Igor Levshun, Dmitry ITMO Univ Kronverksky Pr 49Bldg A St Petersburg 197101 Russia Russian Acad Sci SPC RAS St Petersburg Fed Res Ctr 3914th Liniya St Petersburg 199178 Russia

Industrial Internet of Things (IIoT) presents a range of benefits but also introduces security vulnerabilities. This paper systematically compares different deep learning model structures for IIoT intrusion detection. Four Convolutional neural network (CNN) classifiers are implemented, including hybrid CNN-GRU, 1D Xception, and 1D Resnet models. The evaluation focuses on robustness and generalizability across two datasets to assess overall performance and detection rates. To handle imbalanced data, preprocessing involves PCA feature selection and hybrid resampling techniques. Furthermore, we design a distributed training process for massive datasets and continuous learning, enabling efficient large-scale processing. For the experimental evaluation, we make use of two datasets with multi-label classification tasks and analyze performance metrics of different models in terms of their detection abilities and efficiency. The proposed methods demonstrate strong performance, successfully identifying even rare attacks. In addition, we conduct a performance comparison with existing publications that utilize the same datasets, confirming that our models are on par with state-of-the-art IIoT models.

关键词： Information security Artificial intelligence Deep learning State assessment Attack detection network traffic Industrial ioT

来源：评论

学校读者我要写书评

暂无评论

Microwave Photonics-Based Small Hydrostatic Pressure Sensing Assisted by Convolutional neural network

引用

IEEE Sensors Journal 2025年第11期25卷 20948-20955页

作者： Li, Songlin Xue, Ting Wu, Yan Li, Zhuping Wu, Bin Tianjin University Tianjin Key Laboratory of Process Measurement and Control School of Electrical and Information Engineering Tianjin300072 China Tianjin University State Key Laboratory of Precision Measuring Technology and Instruments Tianjin300072 China

Hydrostatic pressure measurement is essential in many industries, such as oil and gas production, chemical processing, and environmental monitoring. Due to the minimal impact of small hydrostatic pressure on ordinary silica optical fibers, research on its measurement utilizing optical fiber sensing technology remains limited. In this paper, a novel microwave photonics technique, termed optical carrier-based microwave interferometry (OCMI), is utilized for small hydrostatic pressure sensing with the assistance of a convolutional neural network (CNN). The theory of OCMI-based phase demodulation is established, and numerical simulations are conducted to investigate the factors affecting the axial displacement of the fiber core. In practical experiments, the phase demodulation method is applied to small hydrostatic pressure measurements;however, the results are suboptimal. Therefore, the CNN is developed to assist in the implementation of accurate small hydrostatic pressure sensing. The small hydrostatic pressures predicted by the well-trained CNN model are in good agreement with the actual values, with an error of less than 0.25 kPa. In addition, the prediction results from multiple Fabry-Perot interferometers demonstrate the feasibility and effectiveness of utilizing CNN for OCMI-based small hydrostatic pressure sensing. The introduction of machine learning broadens the application scope of the OCMI technique, allowing it to be employed for distributed sensing of a wider range of physical, chemical, and biological quantities. © 2001-2012 IEEE.

关键词： Interferometry

来源：评论

学校读者我要写书评

暂无评论

Enhancing μNAS for 1D CNNs on Microcontrollers 55th

Enhancing μNAS for 1D CNNs on Microcontrollers

引用

55th Meeting of the Italian Electronics Society

作者： Capello, Alessio Berta, Riccardo Ballout, Hadi Fresta, Matteo Soltanmuradov, Vafali Bellotti, Francesco Univ Genoa Dept Elect Elect & Telecommun Engn DITEN Via Opera Pia 11a I-16145 Genoa Italy

ISBN: (纸本)9783031715204;9783031715181;9783031715174

Deep Learning (DL) has proved effective in a variety of application domains. However, the computational and memory demand posed by deep neural models limits the spread of DL on resource-constrained devices such as microcontrollers. An opportunity to tailor DL models to specific hardware constraints is given by neural Architecture Search (NAS), which automatically traverses a large search space, seeking for optimal architectures both in terms of hardware and performance, based on user specifications. State of the art open-source NAS tools for microcontrollers only support 2D Convolutional neural network (CNN) and Multi Layer Perceptron (MLP), but do not consider 1D convolution, which is key for time series analysis and signal processing. This study focuses on enhancing the state-of-the-art mu NAS framework, by adding support for 1D CNN. Preliminary tests on a dummy dataset consisting of simple gaussian-distributed waveforms, demonstrate the system ability to find appropriate architectures to satisfy the specified constraints.

关键词： neural Architecture Search (NAS) Microcontrollers Optimization 1D Convolutional neural network

来源：评论

学校读者我要写书评

暂无评论

Early Diagnosis and Severity Assessment of Weligama Coconut Leaf Wilt Disease and Coconut Caterpillar Infestation Using Deep Learning-Based Image processing Techniques

引用

IEEE ACCESS 2025年 13卷 24463-24477页

作者： Vidhanaarachchi, Samitha Wijekoon, Janaka L. Abeysiriwardhana, W. A. Shanaka P. Wijesundara, Malitha Sri Lanka Inst Informat Technol Malabe 10115 Sri Lanka Victorian Inst Technol Adelaide Campus Adelaide SA 5000 Australia Keio Univ Dept Syst Design Engn Yokohama 1080073 Japan ACSL Ltd Tokyo 1340086 Japan

Global Coconut (Cocos nucifera (L.)) cultivation faces significant challenges, including yield loss, due to pest and disease outbreaks. In particular, Weligama Coconut Leaf Wilt Disease (WCWLD) and Coconut Caterpillar Infestation (CCI) damage coconut trees, causing severe coconut production loss in Sri Lanka and nearby coconut-producing countries. Currently, both WCWLD and CCI are detected through on-field human observations, a process that is not only time-consuming but also limits the early detection of infections. This paper presents a study conducted in Sri Lanka, demonstrating the effectiveness of employing transfer learning-based Convolutional neural network (CNN) and Mask Region-based-CNN (Mask R-CNN) to identify WCWLD and CCI at their early stages and to assess disease progression. Further, this paper presents the use of the You Only Look Once (YOLO) object detection model to count the number of caterpillars distributed on leaves with CCI. The introduced methods were tested and validated using datasets collected from Matara, Puttalam, and Makandura, Sri Lanka. The results show that the proposed methods identify WCWLD and CCI with an accuracy of 90% and 95%, respectively. In addition, the proposed WCWLD disease severity identification method classifies the severity with an accuracy of 97%. Furthermore, the accuracies of the object detection models for calculating the number of caterpillars in the leaflets were: YOLOv5-96.87%, YOLOv8-96.1%, and YOLO11-95.9%.

关键词： Diseases YOLO Accuracy Crops Cameras Plant diseases Nutrients Insects Convolutional neural networks Predictive models Coconut pest control transfer learning deep learning image processing mask R-CNN YOLOv5 YOLOv8 YOLO11

来源：评论

学校读者我要写书评

暂无评论

Evaluation of effect of optimizers and loss functions on prediction accuracy of brain tumor type using a Light neural network

引用

BIOMEDICAL SIGNAL processing AND CONTROL 2025年 103卷

作者： Barati, Barat Erfaninejad, Maryam Khanbabaei, Hashem Shoushtar Fac Med Sci Dept Radiol Technol Shoushtar Iran Shoushtar Fac Med Sci Dept Basic Med Sci Shoushtar Iran Kerman Univ Med Sci Fac Allied Med Dept Radiol Technol Kerman Iran

Accurate identification of brain tumor categories plays a crucial role in therapeutic management of brain tumors. However, due to the complexity and wide variety of information, this method considers the use of advanced and efficient techniques to be a fundamental requirement. The main objective of this study was to evaluate the effectiveness of various types of loss functions including BCE, CCE, and MSE along with different optimizers such as SGD, Adam, RMSprop, Nadam, and Adagrad on predicting brain tumor classification using a deep learning model with limited layers. To choose the best possible scenarios between optimizers and loss functions, 7,125 MRI images were distributed across four categories of glioma, meningioma, pituitary, and no-tumor, with appropriate training parameters and the same amount of information. The model was trained and tested for 12 courses, and its performance was evaluated using accuracy, precision, recall, F1 score, confusion matrices, and ROC curve. The results showed that the Adam optimizer paired with either CCE or BCE loss function performs better than other combinations with a test validation accuracy of 97.89% and 97.40%, respectively. Thereafter, Nadam and RMSprop outperform the other optimizers. Altogether, the results of this study suggest that integration of the most efficient optimizer and loss function into CNN architecture is a crucial step towards improving tumor classification accuracy.

关键词： Convolutional neural network Optimizer Loss Function Deep Learning Tumor Type Classification

来源：评论

学校读者我要写书评

暂无评论

Robust stability in Cohen-Grossberg neural network with both time-varying and distributed delays

引用

neural processing LETTERS 2008年第2期27卷 179-196页

作者： Song, Qian-Kun Cao, Jin-De Chongqing Jiaotong Univ Dept Math Chongqing 400074 Peoples R China Southeast Univ Dept Math Nanjing 210096 Peoples R China

In this article, the global exponential robust stability is investigated for Cohen-Grossberg neural network with both time-varying and distributed delays. The parameter uncertainties are assumed to be time-invariant and bounded, and belong to given compact sets. Applying the idea of vector Lyapunov function, M-matrix theory and analysis techniques, several sufficient conditions are obtained to ensure the existence, uniqueness, and global exponential robust stability of the equilibrium point for the neural network. The methodology developed in this article is shown to be simple and effective for the exponential robust stability analysis of neural networks with time-varying delays and distributed delays. The results obtained in this article extend and improve a few recently known results and remove some restrictions on the neural networks. Three examples are given to show the usefulness of the obtained results that are less restrictive than recently known criteria.

关键词： global exponential robust stability Cohen-Grossberg neural network time-varying delays distributed delays

来源：评论

学校读者我要写书评

暂无评论

Blind signal extraction of arbitrarily distributed, but temporally correlated signals - A neural network approach

引用

IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES 1999年第9期E82A卷 1834-1844页

作者： Thawonmas, R Cichocki, A Kochi Univ Technol Dept Informat Syst Engn Kochi 7828502 Japan RIKEN Brain Sci Inst Lab Open Informat Syst Wako Saitama 3510198 Japan

In this paper, we discuss a neural network approach for blind signal extraction of temporally correlated sources. Assuming autoregressive models of source signals, we propose a very simple neural network model and an efficient online adaptive algorithm that extract, from linear mixtures, a temporally correlated source with an arbitrary distribution, including a colored Gaussian source and a source with extremely low value (or even zero) of kurtosis. We then combine these extraction processing units with deflation processing units to extract such sources sequentially in a cascade fashion. Theory and simulations show that the proposed neural network successfully extracts all arbitrarily distributed, but temporally correlated source signals from linear mixtures.

关键词： blind source separation and extraction neural networks on-line adaptive algorithms

来源：评论

学校读者我要写书评

暂无评论

Document Relevance Filtering by Natural Language processing and Machine Learning: A Multidisciplinary Case Study of Patents

引用

APPLIED SCIENCES-BASEL 2025年第5期15卷 2357-2357页

作者： Bridgelall, Raj North Dakota State Univ Coll Business Dept Transportat & Supply Chain POB 6050 Fargo ND 58108 USA

The exponential growth of patent datasets poses a significant challenge in filtering relevant documents for research and innovation. Traditional semantic search methods based on keywords often fail to capture the complexity and variability in multidisciplinary terminology, leading to inefficiencies. This study addresses the problem by systematically evaluating supervised and unsupervised machine learning (ML) techniques for document relevance filtering across five technology domains: solid-state batteries, electric vehicle chargers, connected vehicles, electric vertical takeoff and landing aircraft, and light detecting and ranging (LiDAR) sensors. The contributions include benchmarking the performance of 10 classical models. These models include extreme gradient boosting, random forest, and support vector machines;a deep artificial neural network;and three natural language processing methods: latent Dirichlet allocation, non-negative matrix factorization, and k-means clustering of a manifold-learned reduced feature dimension. Applying these methods to more than 4200 patents filtered from a database of 9.6 million patents revealed that most supervised ML models outperform the unsupervised methods. An average of seven supervised ML models achieved significantly higher precision, recall, and F1-scores across all technology domains, while unsupervised methods show variability depending on domain characteristics. These results offer a practical framework for optimizing document relevance filtering, enabling researchers and practitioners to efficiently manage large datasets and enhance innovation.

关键词： document search supervised machine learning unsupervised machine learning natural language processing latent Dirichlet allocation non-negative matrix factorization manifold learning t-distributed stochastic neighbor embedding term co-occurrence networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：