—Neural networks (NNs) have been driving machine learning progress in recent years, but their larger models present challenges in resource-limited environments. Weight pruning reduces the computational demand, often ...
详细信息
—Neural networks (NNs) have been driving machine learning progress in recent years, but their larger models present challenges in resource-limited environments. Weight pruning reduces the computational demand, often with performance degradation and long training procedures. This work introduces distilled gradual pruning with pruned fine-tuning (DG2PF), a comprehensive algorithm that iteratively prunes pretrained NNs using knowledge distillation. We employ a magnitude-based unstructured pruning function that selectively removes a specified proportion of unimportant weights from the network. This function also leads to an efficient compression of the model size while minimizing classification accuracy loss. Additionally, we introduce a simulated pruning strategy with the same effects of weight recovery but while maintaining stable convergence. Furthermore, we propose a multistep self-knowledge distillation strategy to effectively transfer the knowledge of the full, unpruned network to the pruned counterpart. We validate the performance of our algorithm through extensive experimentation on diverse benchmark datasets, including CIFAR-10 and ImageNet, as well as a set of model architectures. The results highlight how our algorithm prunes and optimizes pretrained NNs without substantially degrading their classification accuracy while delivering significantly faster and more compact models. Impact Statement—In recent times, NNs have demonstrated remarkable outcomes in various tasks. Some of the most advanced possess billions of trainable parameters, making their training and inference both energy intensive and costly. As a result, the focus on pruning is growing in response to the escalating demand for NNs. However, most current pruning techniques involve training a model from scratch or with a lengthy training process leading to a significant increase in carbon footprint, and some experience a notable drop in performance. In this article, we introduce DG2PF. This unstruct
Nowadays, high energy amount is being wasted by computing servers and personal electronic devices, which produce a high amount of carbon dioxide. Thus, it is required to decrease energy usage and pollution. Many appli...
详细信息
Vehicular Ad-Hoc Networks (VANETs) are studied wireless networks that enable communication among vehicles and roadside infrastructure. The role a vital play in improving on-road safety, efficacy, and convenience by en...
详细信息
Robust fake speech detection systems are crucial in an era where audio recordings can be easily altered or developed due to advancements in technology. The potential impact of this technology could be devastating due ...
详细信息
Many people all around the world suffer from heart disease, which is regarded as a severe illness. In healthcare, especially cardiology, it is crucial to accurately and quickly diagnose cardiac problems. In this resea...
详细信息
This article proposes three-level (TL) buck-boost direct ac-ac converters based on switching-cell configuration with coupled magnetics. The proposed converters use only six active switches and can produce noninverting...
详细信息
Due to the dynamic nature and node mobility,assuring the security of Mobile Ad-hoc Networks(MANET)is one of the difficult and challenging tasks *** MANET,the Intrusion Detection System(IDS)is crucial because it aids i...
详细信息
Due to the dynamic nature and node mobility,assuring the security of Mobile Ad-hoc Networks(MANET)is one of the difficult and challenging tasks *** MANET,the Intrusion Detection System(IDS)is crucial because it aids in the identification and detection of malicious attacks that impair the network’s regular *** machine learning and deep learning methodologies are used for this purpose in the conventional works to ensure increased security of ***,it still has significant flaws,including increased algorithmic complexity,lower system performance,and a higher rate of ***,the goal of this paper is to create an intelligent IDS framework for significantly enhancing MANET security through the use of deep learning ***,the min-max normalization model is applied to preprocess the given cyber-attack datasets for normalizing the attributes or fields,which increases the overall intrusion detection performance of ***,a novel Adaptive Marine Predator Optimization Algorithm(AOMA)is implemented to choose the optimal features for improving the speed and intrusion detection performance of ***,the Deep Supervise Learning Classification(DSLC)mechanism is utilized to predict and categorize the type of intrusion based on proper learning and training *** evaluation,the performance and results of the proposed AOMA-DSLC based IDS methodology is validated and compared using various performance measures and benchmarking datasets.
The capacity to generalize to future unseen data stands as one of the utmost crucial attributes of deep neural networks. Sharpness-Aware Minimization (SAM) aims to enhance the generalizability by minimizing worst-case...
详细信息
In the ever-evolving landscape of optimization algorithms for healthcare datasets, this study introduces an innovative fusion of the gannet optimization algorithm (GOA) with advanced opposition-based learning (OBL) te...
详细信息
暂无评论