检索结果-内蒙古大学图书馆

Efficient benchmarking of algorithm configurators via model-based surrogates

MACHINE LEARNING 2018年第1期107卷 15-41页

作者： Eggensperger, Katharina Lindauer, Marius Hoos, Holger H. Hutter, Frank Leyton-Brown, Kevin Univ Freiburg Freiburg Germany Univ British Columbia Vancouver BC Canada

The optimization of algorithm (hyper-)parameters is crucial for achieving peak performance across a wide range of domains, ranging from deep neural networks to solvers for hard combinatorial problems. However, the proper evaluation of new algorithm configuration (AC) procedures (or configurators) is hindered by two key hurdles. First, AC scenarios are hard to set up, including the target algorithm to be optimized and the problem instances to be solved. Second, and even more significantly, they are computationally expensive: a single configurator run involves many costly runs of the target algorithm. Here, we propose a benchmarking approach that uses surrogate scenarios, which are computationally cheap while remaining close to the original AC scenarios. These surrogate scenarios approximate the response surface corresponding to true target algorithm performance using a regression model. In our experiments, we construct and evaluate surrogate scenarios for hyperparameter optimization as well as for AC problems that involve performance optimization of solvers for hard combinatorial problems. We generalize previous work by building surrogates for AC scenarios with multiple problem instances, stochastic target algorithms and censored running time observations. We show that our surrogate scenarios capture overall important characteristics of the original AC scenarios from which they were derived, while being much easier to use and orders of magnitude cheaper to evaluate.

关键词： Algorithm configuration hyper-parameter optimization Empirical performance model

来源：评论

学校读者我要写书评

暂无评论

Reliable adaptive distributed hyperparameter optimization (RadHPO) for deep learning training and uncertainty estimation

引用

JOURNAL OF SUPERCOMPUTING 2023年第10期79卷 10677-10690页

作者： Li, John Pantoja, Maria Fernandez-Escribano, Gerardo Univ Calif San Diego Math & Comp Sci 9500 Gilman Dr San Diego CA 92093 USA CalPoly Comp Sci & Software Engn 1 Grand Ave San Luis Obispo CA 93407 USA Univ Castilla La Mancha CLM Comp Syst Ave Espana S-N Albacete 02071 Spain

Training and validation of Neural Networks (NN) are very computationally intensive. In this paper, we propose a distributed system based NN infrastructure that achieves two goals: to accelerate model training, specifically for hyperparameter optimization, and to re-use some of these intermediate models to evaluate the uncertainty of the model. By accelerating model training, we can obtain a large set of potential models and compare them in a shorter amount of time. Automating this process reduces development time and provides an easy way to compare different models. Our application runs different models on distinct servers with a single training data set, each with tweaked hyperparameters. By adding uncertainty to our results, our framework provides not just a single prediction but a distribution over predictions. Adding uncertainty is essential to some NN applications since most models assume that the input data distributions are identical between test and validation. However, in reality, they are producing some catastrophic mistakes. Since our solution is a distributed system, we make our implementation robust to common distributed system failures (servers going down, loss of communication among some nodes, and others). Furthermore, we use a gossip-style heartbeat protocol for failure detection and recovery. Finally, some preliminary results using a black-box approach to generate the training models show that our infrastructure scales well in different hardware platforms.

关键词： Distributed systems Resiliency Machine learning hyper-parameter optimization Uncertainty

来源：评论

学校读者我要写书评

暂无评论

Demystifying Impact of Key hyper-parameters in Federated Learning: A Case Study on CIFAR-10 and FashionMNIST

引用

IEEE ACCESS 2024年 12卷 120570-120583页

作者： Kundroo, Majid Kim, Taehong Chungbuk Natl Univ Sch Informat & Commun Engn Cheongju 28644 South Korea

Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving distributed Machine Learning (ML), enabling model training across distributed devices without compromising data privacy. However, the impact of hyper-parameters on FL model performance remains understudied and most of the existing FL studies rely on default or out-of-the-box hyper-parameters, often leading to suboptimal convergence. This study specifically investigates the intricate relationship between key hyper-parameters-learning rate, epochs per round, batch size, and client participation ratio (CPR)-and the performance of FL models on two distinct datasets: CIFAR-10 using ResNet-18 and FashionMNIST using a simple CNN model. Through systematic exploration on these datasets, employing a centralized server and 200 clients, we elucidate the significant impact of varying hyper-parameters. Our findings underscore the importance of dataset-specific hyper-parameter optimization, revealing contrasting optimal configurations for the complex CIFAR-10 dataset and the simpler FashionMNIST dataset. Additionally, the correlation analysis offers a deep understanding of hyper-parameter inter-dependencies, essential for effective optimization. This study provides valuable insights for practitioners to customize hyper-parameter configurations, ensuring optimal performance for FL models trained on different types of datasets and provides a foundation for future exploration in hyper-parameter optimization within the FL domain.

关键词： Training Convergence Data models optimization Data privacy Servers Complexity theory Communication systems Costs Federated learning hyperparameter optimization Communication cost federated learning hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

A kMap optimized VMD-SVM model for milling chatter detection with an industrial robot

引用

JOURNAL OF INTELLIGENT MANUFACTURING 2022年第5期33卷 1483-1502页

作者： Wang, Yu Zhang, Mingkai Tang, Xiaowei Peng, Fangyu Yan, Rong Huazhong Univ Sci & Technol Sch Mech Sci & Engn Natl NC Syst Engn Res Ctr Wuhan 430074 Peoples R China

Industrial robots play an important role in the milling of large complex parts. However, the robot is less rigid and prone to vibration-related problems;chatter, which affects machining quality and efficiency, is more complex and difficult to monitor. In this paper, a variational mode decomposition-support vector machine (VMD-SVM) model based on information entropy (IE) is built to detect chatter in robotic milling. Significantly, the vibration signals are classified into four states for the first time: stable, transition, regular chatter, and irregular chatter. To improve the accuracy of the identification model based on VMD-SVM, a novel hyper-parameter optimization strategy-the kMap method-is proposed in this paper for optimizing three-dimensional hyper-parameters in the VMD-SVM model. The hyper-parameters of VMD-SVM are jointly optimized by the kMap method, with constant step sizes. As an improved grid search (GS), kMap reduces the operation time to the same order of magnitude as the heuristic algorithm (HA) [comprising particle swarm optimization (PSO) and genetic algorithm (GA)]. The VMD-SVM model with the hyper-parameters optimized by kMap exhibits higher accuracy and better stability than the hyper-parameters optimized by PSO and GA. The results of the validation experiments show that the kMap-optimized identification model is effective in industrial robotic milling.

关键词： VMD-SVM Industrial robot Chatter identification hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

An EnKF-based scheme to optimize hyper-parameters and features for SVM classifier

引用

PATTERN RECOGNITION 2017年第0期62卷 202-213页

作者： Ji, Yingsheng Chen, Yushu Fu, Haohuan Yang, Guangwen Tsinghua Univ Key Lab Earth Syst Modeling Minist Educ Beijing Peoples R China Tsinghua Univ Ctr Earth Syst Sci Beijing Peoples R China Tsinghua Univ Dept Comp Sci & Technol Beijing Peoples R China

The quality of models built by machine learning algorithms mostly depends on the careful tuning of hyper-parameters and feature weights. This paper introduces a novel scheme to optimize hyper-parameters and features by using the Ensemble Kalman Filter (EnKF), which is an iterative optimization algorithm used for high-dimensional nonlinear systems. We build a framework for applying the EnKF method on parameter optimization problems. We propose ensemble evolution to converge to the global optimum. We also optimize the EnKF calculation for large datasets by using the computationally efficient UR decomposition. To demonstrate the performance of our proposed design, we apply our approach for the tuning problem of Support Vector Machines. Experimental results show that the better global optima can be identified by our approach with acceptable computation cost compared to three state-of-the-art Bayesian optimization methods (SMAC, TPE and SPEARMINT). (C) 2016 Elsevier Ltd. All rights reserved.

关键词： EnKF SVM hyper-parameter optimization Feature weighting Feature selection

来源：评论

学校读者我要写书评

暂无评论

Comparative Research of hyper-parameters Mathematical optimization Algorithms for Automatic Machine Learning in New Generation Mobile Network

引用

MOBILE NETWORKS & APPLICATIONS 2022年第3期27卷 928-935页

作者： Zhang, Xiaohang Li, Yuqi Li, Zhengren Beijing Univ Posts & Telecommun Sch Econ & Management Beijing 100876 Peoples R China Beijing Univ Posts & Telecommun Sch Modern Post Beijing 100876 Peoples R China

Under the configuration of the new generation communication network, the algorithm based on machine learning has been widely used in network optimization and mobile user behavior prediction. Therefore, the optimization method with hyper-parameters will have a huge development space in the field of mobile communication network. However, for non-professionals, the bottleneck that restricts the further development and application of the whole machine learning is the selection of suitable machine learning algorithm and the determination of suitable algorithm hyper-parameters. Researchers have proposed to use automatic machine learning algorithm to solve this remarkable problem. This article forms a technical manual that can be easily searched by researchers with summarizing related hyper-parameter optimization methods and proposing the corresponding algorithm framework. Moreover, through the comparison of related optimization methods, we highlight the characteristics and deficiencies of related algorithms in the new generation of mobile networks, and put forward suggestions for future improvement.

关键词： Machine learning Automatic algorithm selection hyper-parameter optimization 5G New generation mobile network Information technology

来源：评论

学校读者我要写书评

暂无评论

A General Descent Aggregation Framework for Gradient-Based Bi-Level optimization

引用

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2023年第1期45卷 38-57页

作者： Liu, Risheng Mu, Pan Yuan, Xiaoming Zeng, Shangzhi Zhang, Jin Dalian Univ Technol DUT RU Int Sch Informat Sci & Engn Dalian 116024 Peoples R China Zhejiang Univ Technol Coll Comp Sci & Technol Hangzhou 310023 Peoples R China Dalian Univ Technol Sch Math Sci Dalian 116024 Peoples R China Univ Hong Kong Dept Math Hong Kang Peoples R China Univ Victoria Dept Math & Stat Victoria BC V8P 5C2 Canada Southern Univ Sci & Technol SUSTech Int Ctr Math Natl Ctr Appl Math Shenzhen Dept Math Shenzhen 518055 Guangdong Peoples R China

In recent years, a variety of gradient-based methods have been developed to solve Bi-Level optimization (BLO) problems in machine learning and computer vision areas. However, the theoretical correctness and practical effectiveness of these existing approaches always rely on some restrictive conditions (e.g., Lower-Level Singleton, LLS), which could hardly be satisfied in real-world applications. Moreover, previous literature only proves theoretical results based on their specific iteration strategies, thus lack a general recipe to uniformly analyze the convergence behaviors of different gradient-based BLOs. In this work, we formulate BLOs from an optimistic bi-level viewpoint and establish a new gradient-based algorithmic framework, named Bi-level Descent Aggregation (BDA), to partially address the above issues. Specifically, BDA provides a modularized structure to hierarchically aggregate both the upper- and lower-level subproblems to generate our bi-level iterative dynamics. Theoretically, we establish a general convergence analysis template and derive a new proof recipe to investigate the essential theoretical properties of gradient-based BLO methods. Furthermore, this work systematically explores the convergence behavior of BDA in different optimization scenarios, i.e., considering various solution qualities (i.e., global/local/stationary solution) returned from solving approximation subproblems. Extensive experiments justify our theoretical results and demonstrate the superiority of the proposed algorithm for hyper-parameter optimization and meta-learning tasks. Source code is available at https://***/vis-opt-group/BDA.

关键词： Bi-level optimization gradient-based method descent aggregation hyper-parameter optimization meta-learning

来源：评论

学校读者我要写书评

暂无评论

Optimized Task Scheduling and Virtual Object Management Based on Digital Twin for Distributed Edge Computing Networks

引用

IEEE ACCESS 2023年 11卷 114790-114810页

作者： Xu, Rongxu Park, Chan-Won Khan, Salabat Jin, Wenquan Moe, Sa Jim Soe Kim, Do Hyeun Jeju Natl Univ Big Data Res Ctr Jeju 63243 South Korea Elect & Telecommun Res Inst Autonomous IoT Res Sect Daejeon 34129 South Korea Elect & Telecommun Res Inst Intelligent Convergence Res Lab Daejeon 34129 South Korea COMSATS Univ Islamabad Dept Comp Sci Attock Campus Attock 43600 Pakistan Yanbian Univ Dept Elect & Commun Engn Yanji 133002 Jilin Peoples R China Jeju Natl Univ Dept Comp Engn Jeju 63243 South Korea

In this paper, we address the challenge of limited resources in Internet of Things (IoT) devices by proposing a solution based on digital twin in distributed edge computing networks. Edge computing is a promising approach that moves computing resources closer to the network's edge to reduce response times in IoT applications. However, simply offloading tasks from IoT devices to edge computing does not accelerate user control. To enhance task performance and improve user management experience, we introduce optimized task scheduling and virtual object management based on a digital twin concept. Our system incorporates virtualization, synchronization, visualization, and simulation functionalities to provide digital twin capabilities. Additionally, we develop a user-friendly web application with a graphical user interface (GUI) for intuitive management of edge computing services. To support our approach, we implement an edge computing supervisor that generates virtualized objects such as edge gateways, IoT devices, and services. These virtual objects serve as resources for creating tasks. Using our proposed digital twin platform, users can dynamically create new tasks based on demand, easily deploy and execute tasks in specific locations, and dynamically allocate edge network resources according to task requirements. An optimized task scheduling mathematical model is presented to compare task scheduling done with and without optimization. Further, the edge computing and digital twin based optimized task scheduling method is integrated with Federated Learning for collaborative learning and privacy preserved computation of sensors sensitive data. We demonstrate the effectiveness of our system by generating tasks for data collection related to indoor environment for prediction of Predicted Mean Vote (PMV) for thermal comfort index of smart homes occupants using HTTP and IoTivity-based devices in distributed edge computing networks. These tasks are properly delivered and ex

关键词： Internet of Things edge computing federated learning digital twin task management hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

An Easy, Simple, and Accessible Web-based Machine Learning Platform, SimPL-ML

引用

INTEGRATING MATERIALS AND MANUFACTURING INNOVATION 2022年第1期11卷 85-94页

作者： Jang, Seunghun Na, Gyoung S. Lee, Jungho Shin, Jung Ho Kim, Hyun Woo Chang, Hyunju Korea Res Inst Chem Technol KRICT Chem Data Driven Res Ctr Gajeong Ro 141 Daejeon 34114 South Korea Virtual Lab 49Achasan Ro 17 Gil Seoul 04799 South Korea

Most machine learning (ML) platforms used in materials science provide prediction models built using a computational database. However, to provide more practical and accurate material property predictions, it is advantageous to build a prediction model with user's own data. Here, we present a web-based ML platform, SimPL-ML (https://***- ***) that enables the user to build an ML prediction model through a simple process using their own data. Our platform, SimPL-ML, comprises four main parts: a dataset editor for dataset preprocessing, a model generator to perform actual model training, a predictor to provide a predicted target value for an arbitrary input, and a band-gap predictor (as an example case study) to predict the band gap of inorganic materials through several optimized band gap prediction models. In addition to its core functions, SimPL-ML provides additional functions such as atomic feature generation and hyper-parameter optimization for efficient ML research. We expect our platform to facilitate more accurate and efficient materials research through ML.

关键词： Machine learning Web-based platform Band gap Inorganic materials Atomic feature generation hyper-parameter optimization

来源：评论

学校读者我要写书评

暂无评论

Dynamic Stacking ensemble monitoring model of dam displacement based on the feature selection with PCA-RF

引用

JOURNAL OF CIVIL STRUCTURAL HEALTH MONITORING 2022年第3期12卷 557-578页

作者： Lei, Wei Wang, Jian Hohai Univ Coll Water Conservancy & Hydropower Engn Nanjing 210098 Peoples R China

The factors that affect the prediction accuracy of monitoring models are the selection of environmental features, the optimization of model hyper-parameters, and the robustness of the model itself. For dam systems affected by multiple factors, there is usually a multi-collinear relationship between features. Eliminating the collinearity problem and selecting the optimal feature subset are the essential methods to affect the performance of the monitoring model. The super-parameters of the model do not depend on data-driven but need to be set artificially before training. Choosing an appropriate hyper-parameter optimization algorithm can optimize the training effect of the model. In this paper, a feature selection method with PCA-RF is proposed, which employs principal component analysis (PCA) to reconstruct the feature with equal dimensional mapping to eliminate the collinearity of the feature set. Then, based on the feature importance evaluation method of random forest (RF) to eliminate the interference factors in the new feature set to improve the model's prediction accuracy. The Bayesian optimization algorithm is introduced to optimize the model hyper-parameters. Secondly, based on the idea of Stacking ensemble, the dam displacement monitoring models of support vector regression (SVR), MLP neural network (MLP), and Gaussian process regression (GPR) are integrated, and static and dynamic monitoring Stacking heterogeneous ensemble models are proposed, respectively. Based on the dynamic monitoring model, the dynamic monitoring index of the dam service status is realized by the confidence interval estimation method. Finally, an engineering example is given to verify the prediction ability of the method. The results show that the feature selection method with PCA-RF has good adaptability to deal with collinear feature selection, the Bayesian optimization algorithm can optimize parameters efficiently. The prediction performance of the dynamic Stacking ensemble monitorin

关键词： Dam monitoring Multi-collinearity hyper-parameter optimization Stacking ensemble Dynamic monitoring

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：