检索结果-内蒙古大学图书馆

Distributed hyperparameter optimization based multivariate time series forecasting

MULTIMEDIA TOOLS AND APPLICATIONS 2023年第2期83卷 5031-5053页

作者： Singh, Tinku Sinha, Ayush Singh, Satakshi Vyas, O. P. Kumar, Manish Indian Inst Informat Technol Allahabad Dept IT Prayagraj Uttar Pradesh India SHAUTS Dept Maths Prayagraj Uttar Pradesh India

Various domains generate continuous data in the form of multivariate time series (MTS), including electricity and stock trading. It is difficult to maintain a low variance in forecasting error while forecasting multi-step MTS. Deep learning models can forecast large and complex multivariate time series with high accuracy, but simulating the hyperparameters for a model to achieve high accuracy is time-consuming. Figuring out the appropriate hyperparameters becomes increasingly stagnant and resource-intensive as the dataset size increases. In this study, we propose a distributed random parameter optimization model to deal with short-term multivariate time series forecasting problems. It utilizes the deep neural network for forecasting in synchronization with the hyperparameter tuning optimization in a distributed manner to exploit the resources and speed up the computation. The proposed model has been validated against a real-world multivariate dataset of electricity consumption and weather conditions for the Canadian province of Ontario from 2010 to 2019. Extensive experiments on the Ontario region demonstrate that the proposed model is scalable and outperforms traditional models such as vector autoregression(VAR), Multilayer Perceptron(MLP), Hidden Markov Model (HMM), and Long Short Term Memory (LSTM). Furthermore, the model outperformed all others in terms of accuracy on a benchmark Spanish electricity dataset.

关键词： Recurrent neural network Information retrieval LSTM HMM hyperparameter optimization

来源：评论

学校读者我要写书评

暂无评论

Efficient hyperparameter optimization through model-based reinforcement learning

引用

NEUROCOMPUTING 2020年 409卷 381-393页

作者： Wu, Jia Chen, SenPeng Liu, XiYuan Univ Elect Sci & Technol China Sch Informat & Software Engn Chengdu Peoples R China

hyperparameter tuning is critical for the performance of machine learning algorithms. However, a noticeable limitation is the high computational cost of algorithm evaluation for complex models or for large datasets, which makes the tuning process highly inefficient. In this paper, we propose a novel model-based method for efficient hyperparameter optimization. Firstly, we frame this optimization process as a reinforcement learning problem and then employ an agent to tune hyperparameters sequentially. In addition, a model that learns how to evaluate an algorithm is used to speed up the training. However, model inaccuracy is further exacerbated by long-term use, resulting in collapse performance. We propose a novel method for controlling the model use by measuring the impact of the model on the policy and limiting it to a proper range. Thus, the horizon of the model use can be dynamically adjusted. We apply the proposed method to tune the hyperparameters of the extreme gradient boosting and convolutional neural networks on 101 tasks. The experimental results verify that the proposed method achieves the highest accuracy on 86.1% of the tasks, compared with other state-of-the-art methods and the average ranking of runtime is significant lower than all methods by using the predictive model. (C) 2020 Elsevier B.V. All rights reserved.

关键词： hyperparameter optimization Machine learning Reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

Predicting health indicators for open source projects (using hyperparameter optimization)

引用

EMPIRICAL SOFTWARE ENGINEERING 2022年第6期27卷 1-31页

作者： Xia, Tianpei Fu, Wei Shu, Rui Agrawal, Rishabh Menzies, Tim North Carolina State Univ Dept Comp Sci Raleigh NC 27695 USA

Software developed on public platform is a source of data that can be used to make predictions about those projects. While the individual developing activity may be random and hard to predict, the developing behavior on project level can be predicted with good accuracy when large groups of developers work together on software projects. To demonstrate this, we use 64,181 months of data from 1,159 GitHub projects to make various predictions about the recent status of those projects (as of April 2020). We find that traditional estimation algorithms make many mistakes. Algorithms like k-nearest neighbors (KNN), support vector regression (SVR), random forest (RFT), linear regression (LNR), and regression trees (CART) have high error rates. But that error rate can be greatly reduced using hyperparameter optimization. To the best of our knowledge, this is the largest study yet conducted, using recent data for predicting multiple health indicators of open-source projects. To facilitate open science (and replications and extensions of this work), all our materials are available online at https://***/arennax/Health_Indicator_Prediction..

关键词： hyperparameter optimization Project health Machine learning

来源：评论

学校读者我要写书评

暂无评论

An automatic hyperparameter optimization DNN model for precipitation prediction

引用

APPLIED INTELLIGENCE 2022年第3期52卷 2703-2719页

作者： Peng, Yuzhong Gong, Daoqing Deng, Chuyan Li, Hongya Cai, Hongguo Zhang, Hao Nanning Normal Univ Sch Comp & Informat Engn Nanning 530001 Peoples R China Fudan Univ Coll Comp Sci & Technol Shanghai 200433 Peoples R China Shangqiu Univ Appl Sci & Technol Coll Dept Sci Kaifeng 475000 Peoples R China Guangxi Coll Educ Dept Math & Comp Sci Nanning 530023 Peoples R China

Deep neural networks (DNN) have gained remarkable success on many rainfall predictions tasks in recent years. However, the performance of DNN highly relies upon the hyperparameter setting. In order to design DNNs with the best performance, extensive expertise in both the DNN and the problem domain under investigation is required. But many DNN users have not met this requirement. Therefore, it is difficult for the users who have no extensive expertise in DNN to design optimal DNN architectures for their rainfall prediction problems that is to solve. In this paper, we proposed a novel automatic hyperparameters optimization method for DNN by using an improved Gene Expression Programming. The proposed method can automatically optimize the hyperparameters of DNN for precipitation modeling and prediction. Extensive experiments are conducted with three real precipitation datasets to verify the performance of the proposed algorithm in terms of four metrics, including MAE, MSE, RMSE, and R-Squared. The results show that: 1) the DNN optimized by the proposed method outperforms the existing precipitation prediction methods including Multiple Linear Regression (MLR), Back Propagation (BP), Support Vector Machine (SVM), Random Forest (RF) and DNN;2) the proposed DNN hyperparameter optimization method outperforms state-of-the-art DNN hyperparameter optimization methods, including Genetic Algorithm, Bayes Search, Grid Search, Randomized Search, and Quasi Random Search.

关键词： Deep neural networks Precipitation prediction Neural structure optimization Neural architecture search hyperparameter optimization Gene expression programming

来源：评论

学校读者我要写书评

暂无评论

A short-term photovoltaic output power forecasting based on ensemble algorithms using hyperparameter optimization

引用

ELECTRICAL ENGINEERING 2024年第5期106卷 5319-5337页

作者： Basaran, Kivanc Celikten, Azer Bulut, Hasan Manisa Celal Bayar Univ Dept Elect Engn Manisa Turkiye Ege Univ Dept Comp Engn Izmir Turkiye

The stochastic and intermittent nature of solar energy presents the power grid with the challenge of providing a stable, secure, and economical power supply, especially in the case of large-scale penetration. The prerequisite for addressing these challenges is accurate power output estimation from PV systems. In addition, accurate power estimation also ensures the correct sizing of PV systems for investors. In this study, the PV output prediction model has been developed based on ensemble algorithms using two years of real power and meteorological data from grid-connected PV systems. Grid search, random search, and Bayesian optimization were used to determine the optimal hyperparameters for ensemble algorithms. The originality of this study is that (i) the use of hyperparameter optimization for ensemble algorithms in predicting PV performance, (ii) the degradation rate of PV panels by ensemble algorithms using the first two years' data, and (iii) the performance comparison of ensemble algorithms using the hyperparameter optimization technique. The accuracy and precision of the prediction model are determined by the relative root mean square error (RMSE), mean absolute error (MAE), mean bias error (MBE), mean scaled error (MSE), coefficient of determination (R2), mean absolute percentage error (MAPE), and maximum absolute error (MaxAE). To the best of our knowledge, this is one of the first studies to address the optimization of all hyperparameters to find the best parameters for ensemble algorithms and PV panel degradation rates. The results show that the CatBoost algorithm has better performance than the other algorithms used. The performance metrics of the CatBoost algorithm were determined to be 0.9327 R2, 0.047 MSE, 0.0388 MAE, 0.0003 MBE, 0.069 RMSE, 18.7 MAPE, and 0.79 MaxAE.

关键词： PV output power Forecasting Degradation rate Ensemble algorithms hyperparameter optimization

来源：评论

学校读者我要写书评

暂无评论

Static facial expression recognition using convolutional neural networks based on transfer learning and hyperparameter optimization

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2020年第35-36期79卷 26587-26604页

作者： Ozcan, Tayyip Basturk, Alper Erciyes Univ Dept Comp Engn TR-38039 Kayseri Turkey

Expression recognition (ER), which has been frequently used in human-computer interaction, uses visual data such as video and static images or sensor-based data for recognizing. Facial expression recognition (FER) is a visual data based ER. Since videos have sequential images, it can be easier to recognize emotion in video signals rather than static images which consist of a single plain image. Therefore, FER on static images is a relatively tough task. Recently, deep learning methods have introduced increased success in classification problems. Accordingly, these methods are also used for FER in the literature. Data preparation and hyperparameter optimization can be utilized to increase the success of deep learning methods. With the preparation of data, the features become more pronounced. Increasing the number of training samples directly also generally affects the success rate. Tuning the hyperparameters of deep learning is another factor that increases the performance of the models. In this study, a classification method including data preparation, hyperparameter optimization, and a transfer learning aided convolutional neural network is proposed. Through the study, a new dataset, named ERUFER, was created by using static images. The newly introduced dataset ERUFER and a popular public dataset JAFFE were classified by the proposed method. To the extent of our knowledge, the best result in the literature is achieved by the proposed method for the JAFFE dataset using a 10-fold cross-validation test technique. On the other hand, a success rate with 92.56 % is achieved for the ERUFER dataset.

关键词： ERUFER dataset Static facial expression recognition Emotion analysis Deep learning Image processing hyperparameter optimization

来源：评论

学校读者我要写书评

暂无评论

Simpler hyperparameter optimization for Software Analytics: Why, How, When?

引用

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 2022年第8期48卷 2939-2954页

作者： Agrawal, Amritanshu Yang, Xueqi Agrawal, Rishabh Yedida, Rahul Shen, Xipeng Menzies, Tim Wayfair Boston MA 02116 USA North Carolina State Univ Dept Comp Sci Raleigh NC 27695 USA

How can we make software analytics simpler and faster? One method is to match the complexity of analysis to the intrinsic complexity of the data being explored. For example, hyperparameter optimizers find the control settings for data miners that improve the predictions generated via software analytics. Sometimes, very fast hyperparameter optimization can be achieved by "DODGE-ing";i.e., simply steering way from settings that lead to similar conclusions. But when is it wise to use that simple approach and when must we use more complex (and much slower) optimizers? To answer this, we applied hyperparameter optimization to 120 SE data sets that explored bad smell detection, predicting Github issue close time, bug report analysis, defect prediction, and dozens of other non-SE problems. We find that the simple DODGE works best for data sets with low "intrinsic dimensionality" (mu(D) approximate to 3) and very poorly for higher-dimensional data (mu(D) > 8). Nearly all the SE data seen here was intrinsically low-dimensional, indicating that DODGE is applicable for many SE analytics tasks.

关键词： Software analytics hyperparameter optimization defect prediction bad smell detection issue close time bug reports

来源：评论

学校读者我要写书评

暂无评论

Deep Learning Algorithms and Parallel Distributed Computing Techniques for High-Resolution Load Forecasting Applying hyperparameter optimization

引用

IEEE SYSTEMS JOURNAL 2022年第3期16卷 3758-3769页

作者： Parizad, Ali Hatziadoniu, Constantine Southern Illinois Univ Dept Elect & Comp Engn Carbondale IL 62901 USA Virginia Tech Natl Capital Region Adv Res Inst ARI 900 North Glebe Rd Arlington VA 22203 USA

Electrical load forecasting is one of the critical tasks that helps power utility companies in planning and operation as well as the energy management system (EMS) in controlling and optimizing the power grid's performance. It guarantees system adequacy and reliability by reducing the gap between supply and demand, and it bolsters cybersecurity by enabling false data detection and system security by predicting system peak load. This article aims to build an accurate long-term load forecasting model based on deep learning (DL) algorithms. Initially, big-data analytics and feature engineering are applied to the power load and weather historical data. Next, three different load forecasting models a re developed based on three corresponding deep neural network models (DNN): the simple RNN, the long short term memory (LSTM), and the gated recurrent unit (GRU), respectively. To find an accurate DNN model, a two-step hyperparameter optimization is performed consisting of cross-validation. In the first step, the random search (RS) algorithm is employed to narrow the parameter search space to a smaller local one. In the second step, the grid search (GS) algorithm is used to find the best hyperparameters in the local area located by RS. However, the computational burden of the DNN training process is too high and may lead to out-of-memory (OOM) issues, especially when we are dealing with a big volume of data. To overcome this problem, we developed our Python program based on Tensorflow framework and implemented the framework on a high-performance computer cluster (HPCC). Then, different jobs are defined to employ parallel distributed computing (PDC) capability on the SIU BigDawg HPCC. To this end, all DNN hyperparameters (e.g., number of layers, neurons, dropout, batch size, epoch, learning rate, etc.) can be optimized and result in a comprehensive and accurate load forecasting model. Our proposed method, along with the programming framework, is evaluated on a real-world data

关键词： Big data analytics deep learning (DL) algorithms gated recurrent unit (GRU) hyperparameter optimization load forecasting long short term memory (LSTM) parallel distributed computing (PDC) sequence prediction

来源：评论

学校读者我要写书评

暂无评论

Convolutional Neural Networks-Based Lung Nodule Classification: A Surrogate-Assisted Evolutionary Algorithm for hyperparameter optimization

引用

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 2021年第5期25卷 869-882页

作者： Zhang, Miao Li, Huiqi Pan, Shirui Lyu, Juan Ling, Steve Su, Steven Beijing Inst Technol Sch Informat & Elect Beijing 100081 Peoples R China Univ Technol Sydney Fac Engn & Informat Technol Ultimo NSW 2007 Australia Monash Univ Fac Informat Technol Melbourne Vic 3800 Australia Harbin Engn Univ Informat & Commun Engn Harbin 150001 Peoples R China

This article investigates deep neural networks (DNNs)-based lung nodule classification with hyperparameter optimization. hyperparameter optimization in DNNs is a computationally expensive problem, and a surrogate-assisted evolutionary algorithm has been recently introduced to automatically search for optimal hyperparameter configurations of DNNs, by applying computationally efficient surrogate models to approximate the validation error function of hyperparameter configurations. Different from existing surrogate models adopting stationary covariance functions (kernels) to measure the difference between hyperparameter points, this article proposes a nonstationary kernel that allows the surrogate model to adapt to functions whose smoothness varies with the spatial location of inputs. A multilevel convolutional neural network (ML-CNN) is built for lung nodule classification, and the hyperparameter configuration is optimized by the proposed nonstationary kernel-based Gaussian surrogate model. Our algorithm searches with a surrogate for optimal setting via a hyperparameter importance-based evolutionary strategy, and the experiments demonstrate our algorithm outperforms manual tuning and several well-established hyperparameter optimization methods, including random search, grid search, the tree-structured parzen estimator (TPE) approach, Gaussian processes (GP) with stationary kernels, and the recently proposed hyperparameter optimization via RBF and dynamic (HORD) coordinate search.

关键词： optimization Kernel Neural networks Lung Feature extraction Gaussian processes Convolutional neural networks AutoML evolutionary algorithm hyperparameter optimization lung nodule classification nonstationary kernel

来源：评论

学校读者我要写书评

暂无评论

hyperparameter optimization of Deep Learning Networks for Classification of Breast Histopathology Images

引用

SENSORS AND MATERIALS 2021年第1期33卷 315-325页

作者： Lin, Cheng-Jian Jeng, Shiou-Yun Lee, Chin-Ling Natl Chin Yi Univ Technol Dept Comp Sci & Informat Engn 57Sec 2Zhongshan Rd Taichung 411 Taiwan Natl Taichung Univ Sci & Technol Coll Intelligence 129Sec 3Sanmin Rd Taichung 404 Taiwan Natl Taichung Univ Sci & Technol Dept Int Business 129Sec 3Sanmin Rd Taichung 404 Taiwan

After tumor detection in their breasts, women typically fear mastectomy;this affects curative care outcomes. Most tumors are benign. After resection and pathological examination, because of advances in medicine and treatment, the success rate of early breast cancer treatment can reach 60 to 90%. An accurate assessment of tumor extent is essential. In this study, a novel method of hyperparameter optimization of deep learning networks was proposed to classify tumors as malignant or benign. When setting hyperparameters in deep learning networks, most users use trial and error to determine them. In our experiments, the Taguchi method was used to select the impact factors. The orthogonal table design was used to conduct experiments. Then, the best combination of parameters was determined and significant impact factors were analyzed. The Breast Cancer Histopathological Database was used for analysis. This database was built in collaboration with the MD Laboratory and contains 2480 benign and 5429 malignant samples. The experimental results showed that the proposed method obtained a high accuracy of 83.19%.

关键词： breast cancer deep learning histopathology hyperparameter optimization Taguchi method

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：