检索结果-内蒙古大学图书馆

Reproducible hyperparameter optimization

JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS 2022年第1期31卷 84-99页

作者： Hertel, Lars Baldi, Pierre Gillen, Daniel L. Univ Calif Irvine Dept Stat Irvine CA 92697 USA Univ Calif Irvine Dept Comp Sci Irvine CA 92697 USA

A key issue in machine learning research is the lack of reproducibility. We illustrate what role hyperparameter search plays in this problem and how regular hyperparameter search methods can lead to a large variance in outcomes due to nondeterministic model training during hyperparameter optimization. The variation in outcomes poses a problem both for reproducibility of the hyperparameter search itself and comparisons of different methods each optimized using hyperparameter search. In addition, the fact that hyperparameter search may result in nonoptimal hyperparameter settings may affect other studies, since hyperparameter settings are often copied from previously published research. To remedy this issue, we define the mean prediction error across model training runs as the objective for the hyperparameter search. We then propose a hypothesis testing procedure that makes inference on the mean performance of each hyperparameter setting and results in an equivalence class of hyperparameter settings that are not distinguishable in performance. We further embed this procedure into a group sequential testing framework to increase efficiency in terms of the average number of model training replicates required. Empirical results on machine learning benchmarks show that at equal computation the proposed method reduces the variation in hyperparameter search outcomes by up to 90% while resulting in equal or lower mean prediction errors when compared to standard random search and Bayesian optimization. Moreover, the sequential testing framework successfully reduces computation while preserving performance of the method. The code to reproduce the results is available online and in the .

关键词： ANOVA hyperparameter optimization Machine learning Sequential testing

来源：评论

学校读者我要写书评

暂无评论

hyperparameter optimization based deep convolution neural network model for automated bone age assessment and classification

引用

DISPLAYS 2022年 73卷

作者： Palaniswamy, Thangam King Abdulaziz Univ Dept Elect & Comp Engn Jeddah Saudi Arabia

Bone age assessment (BAA) is widely employed in therapeutic investigation of endocrinology issues in children. BAA is usually carried out by radiological investigation of the left hand. Since manual BAA is prone to observer variations, it is needed to design automated BAA approaches. Recently developed deep learning (DL) models have demonstrated fascinating outcomes in automatic BAA. This paper presents a novel hyperparameter optimization based deep learning model for automated bone age assessment and classification (HPTDL-BAAC). The proposed HPTDL-BAAC technique aims to predict the bone age and classify it into several stages using X-ray images. The proposed model involves data normalization for pre-processing. For feature extraction, a Regional Convolutional Neural Network (RCNN) based mask with SqueezeNet is employed as a baseline model. For boosting the performance of SqueezeNet method, swallow swarm optimization (SSO) algorithm is used for hyperparameter optimization. Finally, SoftMax classifier-based age prediction and weighted extreme learning machine (WELM) based stage classification models are applied to determine the proper bone age and class label. A wide range of simulations were performed on Digital Hand Atlas (DHA) Database and the outcomes are examined with respect to several measures. The experimental outcomes highlighted the supremacy of HPTDLBAAC technique over the other existing techniques.

关键词： Bone age assessment Deep learning hyperparameter optimization Radiography Convolutional neural network Swallow swarm optimization

来源：评论

学校读者我要写书评

暂无评论

hyperparameter optimization of tapping center machines model using robust whale optimization algorithm

引用

ADVANCES IN MECHANICAL ENGINEERING 2023年第5期15卷

作者： Yang, Po-Yuan Chang, Ping-Yueh Natl Pingtung Univ Dept Intelligent Robot Pingtung Taiwan Natl Kaohsiung Univ Sci & Technol Dept Elect Engn Kaohsiung Taiwan Natl Pingtung Univ Dept Intelligent Robot 51 Ming Sheng East Rd Pingtung 900 Taiwan

To build a synchronization error prediction model for the machine tool efficiently, a robust whale optimization algorithm (RWOA) method proposed in this study is applied to the hyperparameter optimization of its model. The proposed RWOA method integrated a non-linear time-invariant inertia weighting (NTIW) method and a Taguchi-based adaptive parameter exploration (ATPE) to improve the performance of WOA and promote robustness. The NTIW method can improve the performance of other algorithms, so this study used the NTIW method in the WOA. In addition, the Taguchi method can get an excellent combination of variables with optimal values and stable performance, making the WOA robust. First, to verify the validity of the proposed RWOA method, 13 benchmark functions were used in this study. The results of the benchmark function tests include the mean, standard deviation, and p-value of the t-distribution test. The results show that 11 of the 13 functions differ significantly. In other non-significant difference functions, the means and standard deviations obtained by the proposed RWOA are considerably better than those obtained by WOA. Since the product cost of machine tools is higher, if a prediction model can be built effectively, it can reduce the cost. Therefore, in this study, the proposed RWOA was used to explore the best hyperparameter combination for the model. From the results, the model's average MAPE (mean absolute percentage error) was 7.2604% for training data and 9.2603% for the testing data under 30 modeling runs. For the best one in 30 models, the MAPE was 6.8384% for the training data and 6.7372% for the testing data. This model was also introduced into the actual machine in this study, and the experimental results showed the MAPE 6.3447%. The proposed RWOA method effectively explores a suitable synchronization error model for the tool machine.

关键词： Robust whale optimization algorithm hyperparameter optimization machine tool rigid tapping synchronization error model

来源：评论

学校读者我要写书评

暂无评论

Dynamic Fidelity Selection for hyperparameter optimization

Dynamic Fidelity Selection for Hyperparameter Optimization

引用

Genetic and Evolutionary Computation Conference (GECCO)

作者： Takenaga, Shintaro Ozaki, Yoshihiko Onishi, Masaki Univ Tsukuba Ibaraki Japan AIST Ibaraki Japan AIST Tokyo Japan GREE Inc Tokyo Japan

ISBN: (纸本)9798400701207

The dramatic growth of deep learning over the past decade has increased the demand for effective hyperparameter optimization (HPO). At the moment, evolutionary algorithms such as the covariance matrix adaptation evolution strategy (CMA-ES) are recognized as one of the most promising approaches for HPO. However, it is often problematic for practitioners that HPO is a time-consuming task because of its computationally expensive objective even if evaluations were parallelized in each generation of an evolutionary algorithm. To address the problem, multi-fidelity optimization that exploits cheap-to-evaluate lower-fidelity alternatives instead of the true maximum-fidelity objective can be utilized for faster optimization. In this paper, we introduce a new fidelity-selecting strategy designed to solve HPO problems with an evolutionary algorithm. Then, we demonstrate that the CMA-ES with the proposed strategy accelerates the search by about 8.5%-15% compared with the vanilla CMA-ES while keeping the quality of the solutions obtained.

关键词： Multi-Fidelity optimization hyperparameter optimization Evolution Strategy

来源：评论

学校读者我要写书评

暂无评论

NCQS: Nonlinear Convex Quadrature Surrogate hyperparameter optimization

NCQS: Nonlinear Convex Quadrature Surrogate Hyperparameter O...

引用

IEEE/CVF International Conference on Computer Vision (ICCV)

作者： Abraham, Sophia Maduranga, Kehelwala Dewage Gayan Kinnison, Jeffery Hauenstein, Jonathan Scheirer, Walter Univ Notre Dame Notre Dame IN 46556 USA Tennessee Technol Univ Cookeville TN 38505 USA

ISBN: (纸本)9798350307443

Deep learning has revolutionized artificial intelligence and enabled breakthroughs across various domains. However, as deep learning models continue to grow in scale and complexity, optimizing their hyperparameters for efficient resource utilization becomes a critical challenge. Traditional optimization techniques often assume smooth and continuous loss functions, limiting their effectiveness in this context. In this work, we propose a novel data-driven approach to hyperparameter optimization using a convex quadrature surrogate. By leveraging a set of sampled hyperparameters and their corresponding performance, our method fits a multivariate quadratic surrogate model to identify the optimal hyperparameters. We demonstrate the practicality and effectiveness of our approach by improving the efficiency and performance of various hyperparameter strategies on both closed and open set benchmarks across diverse vision and tabular datasets. Additionally, we showcase its applicability in automatic target recognition tasks. This research contributes to the broader objective of resource-efficient deep learning for computer vision, fostering advancements in model efficiency, computational memory constraints, and latency considerations. Code available here.

关键词： computer vision deep learning hyperparameter optimization

来源：评论

学校读者我要写书评

暂无评论

Air Quality Index Prediction Model Based on Multiple Attention Mechanisms and hyperparameter optimization 3

Air Quality Index Prediction Model Based on Multiple Attenti...

引用

3rd International Conference on Electronic Information Engineering and Computer Science, EIECS 2023

作者： Yang, Fan Jiang, Xiaoming Chen, Zhanfang Changchun University of Science and Technology School of Computer Science and Technology Changchun China

ISBN: (纸本)9798350316674

Air pollution poses extensive threats to human health and the environment. Air quality forecasting helps people take preventive measures in advance, reducing unnecessary losses. This paper introduces a new Air Quality Index(AQI) prediction model named SSA-CCBA, which is based on a Convolutional Neural Network Bidirectional Long Short-Term Memory network (CNN-BiLSTM) foundational model, combined with multiple attention mechanisms and hyperparameter optimization. Firstly, on a conventional CNN network, the Convolutional Block Attention Module (CBAM) is incorporated to enhance the network's feature extraction capability. Subsequently, an attention mechanism is introduced at the output of the BiLSTM network to enhance the temporal features extracted by BiLSTM, reinforcing the influence of crucial historical information. Lastly, the Sparrow Search Algorithm (SSA) is employed for hyperparameter optimization, selecting the optimal hyperparameters to unleash the model's best performance. Simultaneously, experimental validation was conducted on real-world datasets, demonstrating that the model exhibits higher prediction accuracy. © 2023 IEEE.

关键词： Air quality index prediction Attention mechanism Deep learning hyperparameter optimization

来源：评论

学校读者我要写书评

暂无评论

Fault diagnosis of photovoltaic array with multi-module fusion under hyperparameter optimization

引用

ENERGY CONVERSION AND MANAGEMENT 2024年 319卷

作者： Gong, Bin An, Aimin Shi, Yaoke Jia, Wenchao Lanzhou Univ Technol Coll Elect & Informat Engn Lanzhou 730050 Peoples R China Lanzhou Univ Technol Sch Comp & Commun Lanzhou 730050 Peoples R China Lanzhou Univ Technol Key Lab Gansu Adv Control Ind Proc Lanzhou 730050 Peoples R China

Photovoltaic (PV) arrays' random and intermittent output characteristics impact power system safety. To improve the performance of the PV array fault diagnosis model, a novel online fault monitoring technique is introduced. (1) Fault diagnostic model construction: Significant differences in PV arrays' I-V and P-V curves under various fault conditions led to constructing a 3D channel feature map based on I, V, and P features. (2) Multi-source information fusion network (MSIFN): this multi-module fusion model includes a time-frequency domain fusion module (TDFM), a multi-feature shuffle expansion convolution module (MSECM), a parameterfree parallel hybrid attention enhancement module, and a multi-scale mixed pooling fusion classification module (MMPCM). (3) Multi-strategy fusion whale optimization algorithm (MSFWOA): addressing the original WOA's deficiencies, we designed time control, parameter modification, and greedy control strategies based on lens imaging to optimize MSIFN's hyper-parameters. Experimental results show that the MSFWOA-MSIFN model excels in PV array fault diagnosis (Paccuracy=Pprecision=Precall = 99.92 %). In three types of noise experiments with 15 dB, 25 dB, and 30 dB, the average performance index remained above 99 %. In practical experiments, the average performance indices werePaccuracy = 97.53 %, Pprecision = 97.32 %, andPrecall = 97.41 %, further demonstrating its excellent diagnostic performance. This model effectively diagnoses various faults in PV arrays, providing scientific and theoretical support for PV system operations.

关键词： Photovoltaic array Fault diagnosis Multi-source information fusion hyperparameter optimization Multi-strategy fusion

来源：评论

学校读者我要写书评

暂无评论

XF-OPT/META: A hyperparameter optimization Framework Applied to the H -SPPBO Metaheuristic for the Dynamic TSP

XF-OPT/META: A Hyperparameter Optimization Framework Applied...

引用

2023 IEEE Symposium Series on Computational Intelligence, SSCI 2023

作者： Werner, Daniel Turna, Fatma Le, Hoang Thanh Middendorf, Martin Leipzig University Department of Computer Science Germany Leipzig University Swarm Intelligence and Complex Systems Group Department of Computer Science Germany

ISBN: (纸本)9781665430654

This paper has two objectives. Firstly, to introduce a new framework XF -OPTIMETA for testing and comparing hyperparameter optimization (HPO) methods. The framework supports model-free methods, e.g., Random Search (RS), as well as model-based methods, such as Bayesian optimization (BO), with various surrogate models. Due to the generalized and modular structure of the XF-OPTIMETA framework, it can be easily extended to other optimization methods for dif-ferent optimization problems. The second objective is to empir-ically compare the performance of various HPO methods for population-based metaheuristics. For that the XF -OPTIMETA framework is used to apply HPO methods to the Hierarchical Simple Probabilistic Population-Based optimization (H-SPPBO) metaheuristic for the Dynamic Traveling Salesperson Problem (DTSP) and to calculate high-performing parameter values for H-SPPBO. Promising results are obtained using the parameter values found by BO. In particular, a parameter set obtained with Gradient-Boosted Regression Trees (GBRT) outperforms a reference parameter set for H-SPPBO from an existing study. © 2023 IEEE.

关键词： Bayesian optimization Dynamic Traveling Salesperson Problem Hierarchical Simple Probabilistic Population-Based optimization (H-SPPBO) hyperparameter optimization Metaheuristics Parameter Tuning

来源：评论

学校读者我要写书评

暂无评论

Advanced hyperparameter optimization of deep learning models for wind power prediction

引用

RENEWABLE ENERGY 2024年 221卷

作者： Hanifi, Shahram Cammarono, Andrea Zare-Behtash, Hossein Univ Glasgow James Watt Sch Engn Glasgow G12 8QQ Scotland Sch Engn R703C Level 7James Watt South Bldg Glasgow G12 8QQ Scotland

The uncertainty of wind power as the main obstacle of its integration into the power grid can be addressed by an accurate and efficient wind power forecast. Among the various wind power forecasting methods, machine learning (ML) algorithms, are recognized as a powerful wind power forecasting tool, however, their performance is highly dependent on the proper tuning of their hyperparameters. Common hyperparameter tuning methods such as grid search or random search are time-consuming, computationally expensive, and unreliable for complex models such as deep learning neural networks. Therefore, there is an urgent need for automatic methods to discover optimal hyperparameters for higher accuracy and efficiency of prediction models. In this study, a novel investigation is contributed to the field of wind power forecasting by a comprehensive comparison of three advanced techniques - Scikit-opt, Optuna, and Hyperopt - for hyperparameter optimization of Convolutional Neural Network (CNN) and Long Short-Term Memory Network (LSTM) models, a facet that, to our knowledge, has not been systematically explored in existing literature. The impact of these optimization techniques on the accuracy and efficiency of the CNN and LSTM models are assessed by comparing the root mean square error (RMSE) of the predictions and the required time to tune the models. The results show that the Optuna algorithm, using a Tree-structured Parzen Estimator (TPE) search method and Expected Improvement (EI) acquisition function, has the best efficiency for both CNN and LSTM models. In terms of accuracy, it is demonstrated that while for the CNN model all the optimization methods achieve similar performances, the LSTM model optimized by the Hyperopt algorithm, based on the annealing search method, results in the highest accuracy. In addition, for the first time in this research, the impact of the random initialization features on the performance of the forecasting models with neural networks is investiga

关键词： hyperparameter optimization Bayesian optimization Gaussian process Acquisition function Wind power forecasting Random initialization

来源：评论

学校读者我要写书评

暂无评论

Identification of multiclass tympanic membranes by using deep feature transfer learning and hyperparameter optimization

引用

MEASUREMENT 2024年 229卷

作者： Kilicarslan, Serhat Diker, Aykut Kozkurt, Cemil Donmez, Emrah Demir, Fahrettin Burak Elen, Abdullah Bandirma Onyedi Eylul Univ Fac Engn & Nat Sci Dept Software Engn TR-10200 Balikesir Turkiye Bandirma Onyedi Eylul Univ Fac Engn & Nat Sci Dept Transportat Engn TR-10200 Balikesir Turkiye

Middle ear health is a process that generally depends on eardrum health. Middle ear disorders are more common during childhood. Permanent damage may occur in bacterial or viral infections in this region if an early diagnosis is not made. Infectious ear disease, especially known as Otitis Media, is one of the diseases. In this study, morphological features of images are obtained by using various feature extraction methods. Deep feature-based transfer learning and hyperparameter optimization methods were used to detect the presence and type of otitis media. While the EfficientNet convolutional neural network (CNN) model was used to extract deep features, KNN, SVM, and Ensemble classifiers were used as classifiers. Bayesian, Grid Search, and Random Search were used for hyperparameter optimization. As a result of the experiments carried out, it was observed that the classification performance was 99.1%.

关键词： Tympanic membrane Deep learning hyperparameter optimization Classification

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：