检索结果-内蒙古大学图书馆

Rethinking density ratio estimation based hyper-parameter optimization

NEURAL NETWORKS 2025年 182卷 106917页

作者： Fan, Zi-En Lian, Feng Li, Xin-Ran Xi An Jiao Tong Univ Sch Automat Sci & Technol 28 West Xianning Rd Xian 710049 Shaanxi Peoples R China

hyper-parameter optimization (HPO) aims to improve the performance of machine learning algorithms by identifying appropriate hyper-parameters. By converting the computation of expected improvement into density-ratio estimation problems, existing works use binary classifiers to estimate these ratio and determine the next point by maximizing the class posterior probabilities. However, these methods tend to treat different points equally and ignore some important regions, because binary classifiers are unable to capture more information about search spaces and highlight important regions. In this work, we propose a hyper-parameter optimization method by estimating ratios and selecting the next point using multi-class classifiers. First, we divide all samples into multiple classes and train multi-class classifiers. The decision boundaries of the trained classifiers allow fora finer partitioning of search spaces, offering richer insights into the distribution of hyper- parameters within search spaces. We then define an acquisition function as a weighted sum of multi-class classifiers' outputs, with these weights determined by samples in each class. By assigning different weights to each class posterior probability in our acquisition function, points within search spaces are no longer treated equally. Experimental results on three representative tasks demonstrate that our method achieves a significant improvement in immediate regrets and convergence speed.

关键词： hyper-parameter optimization Density ratio estimation Neural architecture search

来源：评论

学校读者我要写书评

暂无评论

Is hyper-parameter optimization Different for Software Analytics?

引用

IEEE Transactions on Software Engineering 2025年

作者： Yedida, Rahul Menzies, Tim The Department of Computer Science North Carolina State University Raleigh United States

Yes. SE data can have "smoother" boundaries between classes (compared to traditional AI data sets). To be more precise, the magnitude of the second derivative of the loss function found in SE data is typically much smaller. A new hyper-parameter optimizer, called SMOOTHIE, can exploit this idiosyncrasy of SE data. We compare SMOOTHIE and a state-of-the-art AI hyper-parameter optimizer on three tasks: (a) GitHub issue lifetime prediction (b) detecting static code warnings false alarm;(c) defect prediction. For completeness, we also show experiments on some standard AI datasets. SMOOTHIE runs faster and predicts better on the SE data–but ties on non-SE data with the AI tool. Hence we conclude that SE data can be different to other kinds of data;and those differences mean that we should use different kinds of algorithms for our data. © 2025 IEEE. All rights reserve.

关键词： hyper-parameter optimization smoothness software analytics

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization of multiple machine learning algorithms for molecular property prediction using hyperopt library

引用

Chinese Journal of Chemical Engineering 2022年第12期52卷 115-125页

作者： Jun Zhang Qin Wang Weifeng Shen School of Chemistry and Chemical Engineering Chongqing UniversityChongqing 401331China School of Chemistry and Chemical Engineering Chongqing University of Science&TechnologyChongqing 401331China Chongqing Key Laboratory of Theoretical and Computational Chemistry Chongqing 400044China

Due to outstanding performance in cheminformatics,machine learning algorithms have been increasingly used to mine molecular properties and biomedical big *** performance of machine learning models is known to critically depend on the selection of the hyper-parameter ***,many studies either explored the optimal hyper-parameters per the grid searching method or employed arbitrarily selected hyper-parameters,which can easily lead to achieving a suboptimal hyper-parameter *** this study,hyperopt library embedding with the Bayesian optimization is employed to find optimal hyper-parameters for different machine learning *** drug discovery datasets,including solubility,probe-likeness,h ERG,Chagas disease,tuberculosis,and malaria,are used to compare different machine learning algorithms with ECFP6 *** contribution aims to evaluate whether the Bernoulli Na?ve Bayes,logistic linear regression,Ada Boost decision tree,random forest,support vector machine,and deep neural networks algorithms with optimized hyper-parameters can offer any improvement in testing as compared with the referenced models assessed by an array of metrics including AUC,F1-score,Cohen’s kappa,Matthews correlation coefficient,recall,precision,and *** on the rank normalized score approach,the hyperopt models achieve better or comparable performance on 33 out 36 models for different drug discovery datasets,showing significant improvement achieved by employing the hyperopt *** open-source code of all the 6 machine learning frameworks employed in the hyperopt python package is provided to make this approach accessible to more scientists,who are not familiar with writing code.

关键词： Machine learning Prediction Optimal design hyper-parameter optimization hyperopt library

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization of Classifiers, Using an Artificial Immune Network and Its Application to Software Bug Prediction

引用

IEEE ACCESS 2020年 8卷 20954-20964页

作者： Khan, Faiza Kanwal, Summrina Alamri, Sultan Mumtaz, Bushra Riphah Int Univ Fac Comp Islamabad 45211 Pakistan Saudi Elect Univ Dept Comp & Informat Riyadh 11673 Saudi Arabia

Software testing is an important task in software development activities, and it requires most of the resources, namely, time, cost and effort. To minimize this fatigue, software bug prediction (SBP) models are applied to improve the software quality assurance (SQA) processes by predicting buggy components. The bug prediction models use machine learning classifiers so that bugs can be predicted in software components in some software metrics. These classifiers are characterized by some configurable parameters, called hyper-parameters that need to be optimized to ensure better performance. Many methods have been proposed by researchers to predict the defective components but these classifiers sometimes not perform well when default settings are used for machine learning classifiers. In this paper, software bug prediction model is proposed which uses machine learning classifiers in conjunction with the Artificial Immune Network (AIN) to improve bug prediction accuracy through its hyper-parameter optimization. For this purpose, seven machine learning classifiers, such as support vector machine Radial base function (SVM-RBF), K-nearest neighbor (KNN) (Minkowski metric), KNN (Euclidean metric), Naive Bayes (NB), Decision Tree (DT), Linear discriminate analysis (LDA), Random forest (RF) and adaptive boosting (AdaBoost), were used. The experiment was carried out on bug prediction dataset. The results showed that hyper-parameter optimization of machine learning classifiers, using AIN and its applications for software bug prediction, performed better than when classifiers with their default hyper-parameters were used.

关键词： Artificial immune network (AIN) artificial immune system (AIS) hyper-parameter optimization optimized artificial immune network (opt-aiNet) software bug prediction (SBP)

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization of Stacked Asymmetric Auto-Encoders for Automatic Personality Traits Perception

引用

SENSORS 2022年第16期22卷 6206页

作者： Zaferani, Effat Jalaeian Teshnehlab, Mohammad Khodadadian, Amirreza Heitzinger, Clemens Vali, Mansour Noii, Nima Wick, Thomas KN Toosi Univ Technol Elect & Comp Engn Fac Tehran *** Iran Leibniz Univ Hannover Inst Appl Math D-30167 Hannover Germany TU Wien Inst Anal & Sci Comp A-1040 Vienna Austria TU Wien Ctr Artificial Intelligence & Machine Learning CA A-1040 Vienna Austria Leibniz Univ Hannover Inst Continuum Mech D-30823 Hannover Germany

In this work, a method for automatic hyper-parameter tuning of the stacked asymmetric auto-encoder is proposed. In previous work, the deep learning ability to extract personality perception from speech was shown, but hyper-parameter tuning was attained by trial-and-error, which is time-consuming and requires machine learning knowledge. Therefore, obtaining hyper-parameter values is challenging and places limits on deep learning usage. To address this challenge, researchers have applied optimization methods. Although there were successes, the search space is very large due to the large number of deep learning hyper-parameters, which increases the probability of getting stuck in local optima. Researchers have also focused on improving global optimization methods. In this regard, we suggest a novel global optimization method based on the cultural algorithm, multi-island and the concept of parallelism to search this large space smartly. At first, we evaluated our method on three well-known optimization benchmarks and compared the results with recently published papers. Results indicate that the convergence of the proposed method speeds up due to the ability to escape from local optima, and the precision of the results improves dramatically. Afterward, we applied our method to optimize five hyper-parameters of an asymmetric auto-encoder for automatic personality perception. Since inappropriate hyper-parameters lead the network to over-fitting and under-fitting, we used a novel cost function to prevent over-fitting and under-fitting. As observed, the unweighted average recall (accuracy) was improved by 6.52% (9.54%) compared to our previous work and had remarkable outcomes compared to other published personality perception works.

关键词： big five personality traits cultural algorithm deep learning hyper-parameter optimization personality perception

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization-based multi-source fusion for remote sensing inversion of non-photosensitive water quality parameters

引用

INTERNATIONAL JOURNAL OF REMOTE SENSING 2024年第18期45卷 6838-6859页

作者： Yuan, Yuhao Lin, Zhiping Jiang, Xinhao Fan, Zhongmou Fujian Agr & Forestry Univ Sch Transportat & Civil Engn Comprehens Bldg 50763 Xiyuangong Rd Fuzhou 350100 Fujian Peoples R China

The constraints of spatiotemporal heterogeneity and spatial resolution constitute two crucial challenges in the establishment of remote sensing inversion models. Spatiotemporal heterogeneity gives rise to an inadequate generalization capacity of remote sensing models, demanding extensive manual parameter adjustment for each model construction. This not only escalates the task's work intensity but also leads to unstable performance. The limited spatial resolution of remote sensing images leads to suboptimal inversion accuracy for sampling points influenced by mixed pixel effects. To tackle these problems, we take the case of non-photosensitive water quality parameter inversion in the narrow rivers of Longnan area. By integrating advanced hyper-parameter optimization (HPO) techniques, such as Optuna from machine learning, an inversion model was developed, incorporating the bands of Sentinel-2 and Sentinel-3 as model features. Among these features, bands with lower spatial resolution are employed to furnish surrounding information, thereby enhancing the inversion accuracy. The research outcomes demonstrate that: 1) The model constructed based on the HPO method, Optuna, attained favourable inversion results, with R2 values of 0.68, 0.77, 0.35, and 0.60 for Total Nitrogen (TN), Total Phosphorus (TP), Ammonia Nitrogen (NH3-N), and Chemical Oxygen Demand (COD), respectively. 2) The fusion of Sentinel-2 and Sentinel-3 data enhanced the inversion accuracy compared to using them separately, highlighting the considerable significance of multi-source data fusion methods in improving inversion accuracy. This research fills a void in the remote sensing inversion domain and lays the groundwork for future endeavours.

关键词： Remote sensing inverse problems hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization in classification: To-do or not-to-do

引用

PATTERN RECOGNITION 2020年 103卷 107245-107245页

作者： Ngoc Tran Schneider, Jean-Guy Weber, Ingo Qin, A. K. Swinburne Univ Technol Dept Comp Sci & Software Engn Hawthorn Vic 3122 Australia Deakin Univ Sch Informat Technol Geelong Vic Australia CSIRO Data61 Eveleigh NSW 2015 Australia Tech Univ Berlin Chair Software & Business Engn Berlin 10587 Germany Swinburne Univ Technol Hawthorn Vic Australia

hyper-parameter optimization is a process to find suitable hyper-parameters for predictive models. It typically incurs highly demanding computational costs due to the need of the time-consuming model training process to determine the effectiveness of each set of candidate hyper-parameter values. A priori, there is no guarantee that hyper-parameter optimization leads to improved performance. In this work, we propose a framework to address the problem of whether one should apply hyper-parameter optimization or use the default hyper-parameter settings for traditional classification algorithms. We implemented a prototype of the framework, which we use a basis for a three-fold evaluation with 486 datasets and 4 algorithms. The results indicate that our framework is effective at supporting modeling tasks in avoiding adverse effects of using ineffective optimizations. The results also demonstrate that incrementally adding training datasets improves the predictive performance of framework instantiations and hence enables "life-long learning." (C) 2020 Elsevier Ltd. All rights reserved.

关键词： hyper-parameter optimization Framework Bayesian optimization Machine learning Incremental learning

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization Using MARS Surrogate for Machine-Learning Algorithms

引用

IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE 2020年第3期4卷 287-297页

作者： Li, Yangyang Liu, Guangyuan Lu, Gao Jiao, Licheng Marturi, Naresh Shang, Ronghua Xidian Univ Int Res Ctr Intelligent Percept & Computat Sch Artificial IntelligenceJoint Int Res Lab Int Minist EducKey Lab Intelligent Percept & Image U Xian 710071 Peoples R China Univ Birmingham Extreme Robot Lab Edgbaston B15 2TT England

Automatically searching for optimal hyper parameters is of crucial importance for applying machine learning algorithms in practice. However, there are concerns regarding the tradeoff between efficiency and effectiveness of current approaches when faced with the expensive function evaluations. In this paper, a novel efficient hyper-parameter optimization algorithm is proposed (called MARSAOP), in which multivariate spline functions are used as surrogate and dynamic coordinate search approach is employed to generate the candidate points. Empirical studies on benchmark problems and machine-learning models (e.g.,SVM, RE, and NN) demonstrate that the proposed algorithm is able to find relatively high-quality solutions for benchmark problems and excellent hyper-parameter configurations for machine-learning models using a limited computational budget (few function evaluations).

关键词： hyper-parameter optimization MARS dynamic coordinate search machine learning

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization by Using the Genetic Algorithm for Upper Limb Activities Recognition Based on Neural Networks

引用

IEEE SENSORS JOURNAL 2021年第2期21卷 1877-1884页

作者： Zhang, Junjie Sun, Guangmin Sun, Yuge Dou, Huijing Bilal, Anas Beijing Univ Technol Fac Informat Technol Beijing 100124 Peoples R China Univ Manchester Sch Elect & Elect Engn Manchester M13 9PL Lancs England

A method of Upper Limb Activities Recognition (UPLA) based on Neural Networks is presented. The accuracy of activity recognition will be influenced by the size of sliding window, the overlapping of adjacent sequences and the number of neurons for neural networks. Whereas, there is less work in hyper parameters optimization of neural networks automatically. It is very time-consuming to optimize hyper parameters by experts through an experience and error approach. In the paper, Genetic algorithm is used to find the best hyper parameters automatically: the size of sliding window, the overlapping of adjacent sequences and the number of neurons for neural networks. The basic genetic algorithm has a slow convergence problem and it is very easy to fall into a local optimum. To solve the problem, the population selection mechanism is improved. A comparison is made for the improving method with seven traditional classification algorithms and convolutional neural network, an accuracy of 97.9% is reached by using the new method. Finally, an App is developed that can collect and recognize upper limb activity in real time.

关键词： Sensors Activity recognition Accelerometers Gyroscopes Feature extraction Real-time systems Elbow Activity recognition neural networks genetic algorithm hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

hyper-parameter optimization of deep learning architectures using artificial bee colony (ABC) algorithm for high performance real-time automatic colorectal cancer (CRC) polyp detection

引用

APPLIED INTELLIGENCE 2023年第12期53卷 15603-15620页

作者： Karaman, Ahmet Karaboga, Dervis Pacal, Ishak Akay, Bahriye Basturk, Alper Nalbantoglu, Ufuk Coskun, Seymanur Sahin, Omur Acibadem Hosp Dept Gastroenterol Kayseri Turkey Erciyes Univ Engn Fac Dept Comp Engn Kayseri Turkey Erciyes Univ Artificial Intelligence & Big Data Applicat & Res Kayseri Turkey Igdir Univ Engn Fac Dept Comp Engn Igdir Turkey

Colorectal cancer (CRC) is one of the most common and malignant types of cancer worldwide. Colonoscopy, considered the gold standard for CRC screening, allows immediate removal of polyps, which are precursors to CRC. Many computer-aided diagnosis systems (CADs) have been proposed for automatic polyp detection. Most of these systems are based on traditional machine learning algorithms and their generalization ability, sensitivity and specificity are limited. On the other hand, with the widespread use of deep learning algorithms in medical image analysis and the successful results in the analysis of colonoscopy images, especially in the early and accurate detection of polyps, these problems are eliminated in recent years. In short, deep learning algorithms and applications have gained a critical role in CAD systems for real-time autonomous polyp detection. Here, we make significant improvements to object detection algorithms to improve the performance of CAD-based real-time polyp detection systems. We integrate the artificial bee colony algorithm (ABC) into the YOLO algorithm to optimize the hyper-parameters of YOLO-based algorithms. The proposed method can be easily integrated into all YOLO algorithms such as YOLOv3, YOLOv4, Scaled-YOLOv4, YOLOv5, YOLOR and YOLOv7. The proposed method improves the performance of the Scaled-YOLOv4 algorithm with an average of more than 3% increase in mAP and a more than 2% improvement in F1 value. In addition, the most comprehensive study is conducted by evaluating the performance of all existing models in the Scaled-YOLOv4 algorithm (YOLOv4s, YOLOv4m, YOLOV4-CSP, YOLOv4-P5, YOLOV4-P6 and YOLOv4-P7) on the novel SUN and PICCOLO polyp datasets. The proposed method is the first study for the optimization of YOLO-based algorithms in the literature and makes a significant contribution to the detection accuracy.

关键词： Colorectal cancer Deep learning hyper-parameter optimization Artificial bee colony Real-time polyp detection YOLOv4

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：