The adam algorithm is a common choice for optimizing neural network models. However, its application often brings challenges, such as susceptibility to local optima, overfitting and convergence problems caused by unst...
详细信息
The adam algorithm is a common choice for optimizing neural network models. However, its application often brings challenges, such as susceptibility to local optima, overfitting and convergence problems caused by unstable learning rate behavior. In this article, we introduce an enhanced adam optimization algorithm that integrates Warmup and cosine annealing techniques to alleviate these challenges. By integrating preheating technology into traditional adam algorithms, we systematically improved the learning rate during the initial training phase, effectively avoiding instability issues. In addition, we adopt a dynamic cosine annealing strategy to adaptively adjust the learning rate, improve local optimization problems and enhance the model's generalization ability. To validate the effectiveness of our proposed method, extensive experiments were conducted on various standard datasets and compared with traditional adam and other optimization methods. Multiple comparative experiments were conducted using multiple optimization algorithms and the improved algorithm proposed in this paper on multiple datasets. On the MNIST, CIFAR10 and CIFAR100 datasets, the improved algorithm proposed in this paper achieved accuracies of 98.87%, 87.67% and 58.88%, respectively, with significant improvements compared to other algorithms. The experimental results clearly indicate that our joint enhancement of the adam algorithm has resulted in significant improvements in model convergence speed and generalization performance. These promising results emphasize the potential of our enhanced adam algorithm in a wide range of deep learning tasks.
Distributed Optical Fiber Sensing Technology, with advantages such as long detection distance, resistance to electromagnetic interference, and easy maintenance, is widely applied in various fields including security m...
详细信息
Distributed Optical Fiber Sensing Technology, with advantages such as long detection distance, resistance to electromagnetic interference, and easy maintenance, is widely applied in various fields including security monitoring, electrical safety, and aerospace. However, the current Distributed Optical Fiber Sensing Technology faces challenges such as short localization distance, limited range for stress measurement, low accuracy, and time-consuming processes. This study proposes a novel approach for extracting Brillouin frequency shift signals using an adaptive gradient descent algorithm (adam algorithm). A Brillouin strain testing system based on heterodyne coherent detection is also constructed. Experimental results show that the distributed strain testing system using the adam algorithm can achieve accurate and fast measurement of maximum strain up to 9500 mu epsilon within a range of 10 kilometers. The average strain measurement deviation is 32.88 mu epsilon, and the time required for frequency shift extraction is less than 18.5 ms. This method provides a theoretical and experimental basis for the application of BOTDR Distributed Optical Fiber Sensing Technology.
Current stomach disease detection and diagnosis is challenged by data complexity and high dimensionality and requires effective deep learning algorithms to improve diagnostic accuracy. To address this challenge, in th...
详细信息
Current stomach disease detection and diagnosis is challenged by data complexity and high dimensionality and requires effective deep learning algorithms to improve diagnostic accuracy. To address this challenge, in this paper, an improved strategy based on the adam algorithm is proposed, which aims to alleviate the influence of local optimal solutions, overfitting, and slow convergence rates by controlling the restart strategy and the gradient norm joint clipping technique. This improved algorithm is abbreviated as the CG-adam algorithm. The control restart strategy performs a restart operation by periodically checking the number of steps and once the number of steps reaches a preset restart period. After the restart is completed, the algorithm will restart the optimization process. It helps the algorithm avoid falling into the local optimum and maintain convergence stability. Meanwhile, gradient norm joint clipping combines both gradient clipping and norm clipping techniques, which can avoid gradient explosion and gradient vanishing problems and help accelerate the convergence of the optimization process by restricting the gradient and norm to a suitable range. In order to verify the effectiveness of the CG-adam algorithm, experimental validation is carried out on the MNIST, CIFAR10, and Stomach datasets and compared with the adam algorithm as well as the current popular optimization algorithms. The experimental results demonstrate that the improved algorithm proposed in this paper achieves an accuracy of 98.59%, 70.7%, and 73.2% on the MNIST, CIFAR10, and Stomach datasets, respectively, surpassing the adam algorithm. The experimental results not only prove the significant effect of the CG-adam algorithm in accelerating the model convergence and improving generalization performance but also demonstrate its wide potential and practical application value in the field of medical image recognition.
Stability assessment of anti-dip bedding rock slopes (ABRSs) remains a troublesome issue especially when there are hundreds or thousands of rock layers. The adaptive moment estimation (adam) algorithm proposed in rece...
详细信息
Stability assessment of anti-dip bedding rock slopes (ABRSs) remains a troublesome issue especially when there are hundreds or thousands of rock layers. The adaptive moment estimation (adam) algorithm proposed in recent years has been widely used in artificial intelligence research and is a powerful tool for solving the above-mentioned problem. In this work, the limit equilibrium method (LEM) is combined with the adam algorithm (referred to as the LEM-adam method) to determine the failure surface of an ABRS undergoing shearing-flexural toppling failure. Two tests reported in previous studies (a 1 g model and a centrifuge model) were analyzed to validate the LEM-adam method. The critical failure surface and the safety factor agree well with the experimental observations, validating the LEM-adam method in the stability analysis of ABRSs. Finally, the LEM-adam method was compared with the LEM combined with a genetic algorithm (GA). The results show that the LEM-adam method has higher computational efficiency and eliminates the randomness of the solution. The LEM-adam method proposed in this work offers a practical and fast approach for the stability analysis and design of ABRSs.
A neural network model consisting of a fixed network and a flexible network and optimized by adam algorithm is designed to calculate the utilization factor when the floor reflectance ratio is 0.2 and the correction fa...
详细信息
The high-precision short-arc measurement is a huge challenge in scientific research and engineering practice. The popular traditional least square fitting (TLSF) however fails to achieve the precise fitting parameters...
详细信息
The high-precision short-arc measurement is a huge challenge in scientific research and engineering practice. The popular traditional least square fitting (TLSF) however fails to achieve the precise fitting parameters when the arc gets shorter. In this work, an inequation constrained fitting (ICF) method based on four-parameter circle equation is presented. Lagrangian multiplier and Karush-Kuhn-Tucker criteria are used to redefine the objective function. After that, adam algorithm is utilized to solve the objective function in iterative way. adam algorithm has a strong ability to resist noise pollution by virtue of modifying continually the first-order momentum and second-order momentum with average of gradients during the course of iteration. Finally, simulation and experimental results show that our ICF method is more robust and high-precision than TLSF and Hyper method, so it is very competent to measure short arcs with noise even their central angles are close to 5°.
In this paper, a clear underwater image is attained by a fusion process using Transfer Learning (TL). Two images are selected from the underwater colour image dataset and those images are allowed to Discrete Wavelet T...
详细信息
In this paper, a clear underwater image is attained by a fusion process using Transfer Learning (TL). Two images are selected from the underwater colour image dataset and those images are allowed to Discrete Wavelet Transform (DWT), Tetrolet transform and Saliency maps. Here, the outputs gained from images by the Tetrolet transform are fused and allowed for inverse Tetrolet transform. Moreover, the DWT process done with two images is fused and the output gained is allowed for inverse DWT. Similarly, the same fusion process is carried out with image outputs from Saliency maps. Finally, three image outputs that are considered as input to TL with newly devised optimization. Here, Convolutional Neural Network (CNN) is used with hyperparameters from trained models, such as SqueezeNet and AlexNet, where weights are updated using adam Based Bald Eagle algorithm (ABBEA). This ABBEA is obtained by combining the Bald Eagle Search (BES) algorithm and adam algorithm. Further, the ABBEA has Peak Signal-to-Noise Ratio (PSNR) with maximal of 38.95, Mean Squared Error (MSE) with lesser value of 20.14, Structural Similarity Index Measure (SSIM) with maximal value of 0.92, Mutual Information (MI) with maximal value of 0.86, Signal-to-Noise Ratio (SNR) with lesser value of 0.38.
Choosing a suitable optimization algorithm in deep learning is essential for effective model development as it significantly influences convergence speed, model performance, and the success of the training process. Op...
详细信息
ISBN:
(纸本)9783031829307;9783031829314
Choosing a suitable optimization algorithm in deep learning is essential for effective model development as it significantly influences convergence speed, model performance, and the success of the training process. Optimizers play an essential role in adjusting the model's parameters to minimize errors, assisting the learning process during the model development. With various optimization algorithms available, choosing the one that best suits the deep learning model and dataset can make a substantial difference in achieving optimal results. Adaptive Moment Estimation (adam) and Adaptive Nesterov Accelerated Gradient (Adan), two well-known optimizers, are widely used in deep learning for their ability to handle large-scale data and complex models efficiently. While adam is known for its balance between speed and reliability, Adan builds on this by incorporating momentum and lookahead mechanisms to enhance the model's performance. However, choosing the right optimizer for different tasks can be challenging, as each optimizer offers various advantages and disadvantages. This paper, therefore, explores the comparative effectiveness of adam and Adan optimizers, analyzing their impact on convergence speed, model performance, and overall training success on different classifications tasks, which are image and text classifications. The results show that adam performs better initially, but prone to overfitting. On the other hand, for image classification tasks, Adan provides more consistent optimisation across extended training periods. Based on these results, this paper aims to provide insights into the strengths and limitations of each optimizer, highlighting the importance of considering task-specific requirements when selecting an optimization algorithm for deep learning models.
Support vector regression (SVR) encounters challenges when confronted with outliers and noise, primarily due to the limitations of the traditional E-insensitive loss function. To address this, bounded loss functions h...
详细信息
Support vector regression (SVR) encounters challenges when confronted with outliers and noise, primarily due to the limitations of the traditional E-insensitive loss function. To address this, bounded loss functions have gained traction for their robustness and improved generalization. More recent advancements, BLINEX and bounded least square loss, focus on smooth bounded loss functions that enable efficient gradient based optimization. However, these approaches lack an insensitive zone, which is crucial for mitigating deviations and noise. The challenge of designing a loss function that combines boundedness, smoothness, an insensitive zone remains unresolved in the current literature. To address this issue, we develop the HawkEye loss, a novel formulation that integrates boundedness, smoothness, and the presence of an insensitive This unique combination enhances the robustness and generalization capabilities of SVR models, particularly the presence of noise and outliers. Notably, the HawkEye loss is the first in SVR literature to simultaneously incorporate boundedness, smoothness, and an insensitive zone. Leveraging this breakthrough, we integrate the HawkEye loss into the least squares framework of SVR and yield a new robust and scalable termed HE-LSSVR. The optimization problem inherent to HE-LSSVR is addressed by harnessing the adaptive moment estimation (adam) algorithm, known for its adaptive learning rate and efficacy in handling scale problems. To our knowledge, this is the first time adam has been employed to solve an SVR problem. empirically validate the proposed HE-LSSVR model, we evaluate it on UCI, synthetic, time series, and age datasets. The experimental outcomes unequivocally reveal the superiority of the HE-LSSVR model terms of its remarkable generalization performance and its efficiency in training time. The code of the proposed model is publicly available at https://***/mtanveer1/HawkEye.
The present research introduces the best architecture model for predicting the unsoaked California bearing ratio (CBRu) of soil by comparing the models based on the least square support vector machine (LSSVM), long- s...
详细信息
The present research introduces the best architecture model for predicting the unsoaked California bearing ratio (CBRu) of soil by comparing the models based on the least square support vector machine (LSSVM), long- short-term memory (LSTM), and artificial neural network (ANN) approach. The two kernel functions, linear and polynomial, have been selected to create LSSVM models. The developed LSTM models have been optimized by the adam algorithm. In the employed ANN models, the Levenberg-Marquardt (LM), BFGS Quasi-Newton (BFG), scaled conjugate gradient (SCG), gradient descent with momentum (GDM), gradient descent (GD), and gradient descent with adaptive learning (GDA) algorithms have been used in the backpropagation process. For this purpose, three databases, such as training, testing and validation, have been compiled from the published research. A laboratory database has been developed by performing laboratory experiments for soil samples collected from and around Kota, Rajasthan, used for cross-validation of the best architecture model. The statistical tools, such as root means square error (RMSE), mean absolute error (MAE), correlation coefficient (R), mean absolute percentage error (MAPE), variance accounted for (VAF), weighted mean absolute percentage error (WMAPE), Nash-Sutcliffe efficiency (NS), normalized mean bias error (NMBE), Legate and McCabe's index (LMI), root mean square error to observation's standard deviation ratio (RSR), a20-index, index of agreement (IOA) and index of scatter (IOS) have been used to measure the performance of the models. The LSTM model MD 14 has achieved higher performance and accuracy (RMSE = 0.9127%, MAE = 0.8114%, R = 0.9863%, MAPE = 9.0772%, VAF = 97.26, WMAPE = 0.0669%, NS = 0.9708, NMBE = 0.0687%, LMI = 0.1926, RSR = 0.1708, a20-index = 93.88, IOA = 0.9037 and IOS = 0.0752) in testing phase. For the performance validation, model (MD) 14 has predicted the CBRu of the validation database. Also, model MD 14 has attained higher
暂无评论