检索结果-内蒙古大学图书馆

Prediction compressive strength of cement-based mortar containing metakaolin using explainable Categorical Gradient Boosting model

引用

ENGINEERING STRUCTURES 2022年 269卷

作者： Nguyen, Ngoc-Hien Tong, Kien T. Lee, Seunghye Karamanli, Armagan Vo, Thuc P. HUTECH Univ CIRTech Inst Ho Chi Minh City Vietnam Ha Noi Univ Civil Engn Fac Bldg Mat 55 Giai Phong Hanoi Vietnam Sejong Univ Deep Learning Architecture Res Ctr Dept Architectural Engn 209 Neungdong Ro Seoul 05006 South Korea Istinye Univ Fac Engn & Nat Sci Mech Engn Istanbul Turkey La Trobe Univ Sch Comp Engn & Math Sci Bundoora Vic 3086 Australia

Although machine learning models have been employed for the compressive strength (CS) of cement-based mortar containing metakaolin, it is difficult to understand how they work due to "black-box "nature. In order to explain the involved mechanism, Categorical Gradient Boosting (CatBoost) model with feature importance, feature interaction, partial dependence plot (PDP) and SHapley Additive exPlanations (SHAP) is proposed in this paper. A dataset consisting of 424 samples with six input variables is used to build the CatBoost model, which has optimal performance by tuning a set of seven hyper-parameters using sequential model -based optimization. Five quantitative measures (R-2, MAE, RMSE, a10-, a20-index) are employed to evaluate the accuracy and the obtained results are superior to the previous study. It is from feature importance that the most significant input variable involving the CS is water-to-binder ratio, followed by age of specimen and cement grade. The strongest feature interaction is between water-to-binder ratio and metakaolin. A comprehensive parametric study is carried out via SHAP and PDP to investigate the effects of all input variables on the CS of cement-based mortar.

关键词： Cement-based mortar CatBoost sequential model-based optimization Partial dependence plot Shapley Additive exPlanations

来源：评论

学校读者我要写书评

暂无评论

Virtual metrology of semiconductor PVD process based on combination of tree-based ensemble model

引用

ISA TRANSACTIONS 2020年 103卷 192-202页

作者： Chen, Ching-Hsien Zhao, Wei-Dong Pang, Timothy Lin, Yi-Zheng Tongji Univ Elect & Informat Engn Shanghai 201804 Peoples R China Semicond Mfg Int Corp It Dept Shanghai 201203 Peoples R China

In order to improve the accuracy of semiconductor wafer virtual metrology, and overcome the physical metrology delay of wafer acceptance test, a virtual physical vapor deposition metrology method based on combination of tree-based ensemble models is proposed to conduct online virtual metrology on semiconductor wafer electrical parameters, and use hyperparameter optimization technique to perform model optimization and to achieve real-time alarm on process deviation. This combination of tree-based ensemble model combines Bagging, Boosting, and Stacking techniques. First, based on 4 types of base learner, Random Forest, Extra-Trees, XGBoost, and lightGBM, preliminary virtual metrology is performed on wafer PVD process, and then transforms the predict results of the 4 base learners into meta feature vector as the input of meta learner lightGBM to perform further virtual metrology. The sequential model-based optimization algorithm is used to improve the accuracy of virtual metrology. First, the initial hyperparameter of the sequential model-based optimization is initialized by using random sampling, then the combination model is approximated by the surrogate model of tree-structured Parzen estimator, and the recommended hyperparameters is obtained by using EI (Expected Improvement), and then the optimized combination model is obtained. Finally, the superiority of the method proposed in this paper is verified by studying the results comparing to the common virtual metrology methods on the PVD process. The experiment shows the result of resistivity metrology using the combination of tree-based ensemble models in the PVD process is significantly better than LASSO regression, partial least squares regression(PLSR), support vector machine(SVR), Gaussian process regression(GPR) and artificial neural network regression(ANN). (C) 2020 ISA. Published by Elsevier Ltd. All rights reserved.

关键词： Semiconductor sequential model-based optimization Combination of tree-based ensemble models Virtual metrology

来源：评论

学校读者我要写书评

暂无评论

Continuous control with Stacked Deep Dynamic Recurrent Reinforcement Learning for portfolio optimization

引用

EXPERT SYSTEMS WITH APPLICATIONS 2020年 140卷 112891-112891页

作者： Aboussalah, Amine Mohamed Lee, Chi-Guhn Univ Toronto Dept Mech & Ind Engn Toronto ON M5S 3G8 Canada

Recurrent reinforcement learning (RRL) techniques have been used to optimize asset trading systems and have achieved outstanding results. However, the majority of the previous work has been dedicated to systems with discrete action spaces. To address the challenge of continuous action and multi-dimensional state spaces, we propose the so called Stacked Deep Dynamic Recurrent Reinforcement Learning (SDDRRL) architecture to construct a real-time optimal portfolio. The algorithm captures the up-to-date market conditions and rebalances the portfolio accordingly. Under this general vision, Sharpe ratio, which is one of the most widely accepted measures of risk-adjusted returns, has been used as a performance metric. Additionally, the performance of most machine learning algorithms highly depends on their hyperparameter settings. Therefore, we equipped SDDRRL with the ability to find the best possible architecture topology using an automated Gaussian Process (GP) with Expected Improvement (El) as an acquisition function. This allows us to select the best architectures that maximizes the total return while respecting the cardinality constraints. Finally, our system was trained and tested in an online manner for 20 successive rounds with data for ten selected stocks from different sectors of the S&P 500 from January 1st, 2013 to July 31st, 2017. The experiments reveal that the proposed SDDRRL achieves superior performance compared to three benchmarks: the rolling horizon Mean-Variance optimization (MVO) model, the rolling horizon risk parity model, and the uniform buy-and-hold (UBAH) index. (C) 2019 Elsevier Ltd. All rights reserved.

关键词： Reinforcement learning Policy gradient Deep learning sequential model-based optimization Financial time series Portfolio management Trading systems

来源：评论

学校读者我要写书评

暂无评论

Scalable Gaussian process-based transfer surrogates for hyperparameter optimization

引用

MACHINE LEARNING 2018年第1期107卷 43-78页

作者： Wistuba, Martin Schilling, Nicolas Schmidt-Thieme, Lars Informat Syst & Machine Learning Lab Univ Pl 1 Hildesheim Germany

Algorithm selection as well as hyperparameter optimization are tedious task that have to be dealt with when applying machine learning to real-world problems. sequential model-based optimization (SMBO), based on so-called "surrogate models", has been employed to allow for faster and more direct hyperparameter optimization. A surrogate model is a machine learning regression model which is trained on the meta-level instances in order to predict the performance of an algorithm on a specific data set given the hyperparameter settings and data set descriptors. Gaussian processes, for example, make good surrogate models as they provide probability distributions over labels. Recent work on SMBO also includes meta-data, i.e. observed hyperparameter performances on other data sets, into the process of hyperparameter optimization. This can, for example, be accomplished by learning transfer surrogate models on all available instances of meta-knowledge;however, the increasing amount of meta-information can make Gaussian processes infeasible, as they require the inversion of a large covariance matrix which grows with the number of instances. Consequently, instead of learning a joint surrogate model on all of the meta-data, we propose to learn individual surrogate models on the observations of each data set and then combine all surrogates to a joint one using ensembling techniques. The final surrogate is a weighted sum of all data set specific surrogates plus an additional surrogate that is solely learned on the target observations. Within our framework, any surrogate model can be used and explore Gaussian processes in this scenario. We present two different strategies for finding the weights used in the ensemble: the first is based on a probabilistic product of experts approach, and the second is based on kernel regression. Additionally, we extend the framework to directly estimate the acquisition function in the same setting, using a novel technique which we name the "transfer a

关键词： Hyperparameter optimization Gaussian processes sequential model-based optimization Meta-learning

来源：评论

学校读者我要写书评

暂无评论

Hyperparameters and tuning strategies for random forest

引用

WILEY INTERDISCIPLINARY REVIEWS-DATA MINING AND KNOWLEDGE DISCOVERY 2019年第3期9卷 e1301-e1301页

作者： Probst, Philipp Wright, Marvin N. Boulesteix, Anne-Laure Ludwig Maximilians Univ Munchen Inst Med Informat Proc Biometry & Epidemiol Marchioninistr 15 D-81377 Munich Germany Leibniz Inst Prevent Res & Epidemiol BIPS Bremen Germany

The random forest (RF) algorithm has several hyperparameters that have to be set by the user, for example, the number of observations drawn randomly for each tree and whether they are drawn with or without replacement, the number of variables drawn randomly for each split, the splitting rule, the minimum number of samples that a node must contain, and the number of trees. In this paper, we first provide a literature review on the parameters' influence on the prediction performance and on variable importance measures. It is well known that in most cases RF works reasonably well with the default values of the hyperparameters specified in software packages. Nevertheless, tuning the hyperparameters can improve the performance of RF. In the second part of this paper, after a presenting brief overview of tuning strategies, we demonstrate the application of one of the most established tuning strategies, model-based optimization (MBO). To make it easier to use, we provide the tuneRanger R package that tunes RF with MBO automatically. In a benchmark study on several datasets, we compare the prediction performance and runtime of tuneRanger with other tuning implementations in R and RF with default hyperparameters. This article is categorized under: Algorithmic Development > Biological Data Mining Algorithmic Development > Statistics Algorithmic Development > Hierarchies and Trees Technologies > Machine Learning

关键词： ensemble literature review out-of-bag performance evaluation ranger sequential model-based optimization tuning parameter

来源：评论

学校读者我要写书评

暂无评论

Pre-training the deep generative models with adaptive hyperparameter optimization

引用

NEUROCOMPUTING 2017年 247卷 144-155页

作者： Yao, Chengwei Cai, Deng Bu, Jiajun Chen, Gencai Zhejiang Univ Coll Comp Sci & Technol Hangzhou Zhejiang Peoples R China Zhejiang Univ Coll Comp Sci & Technol State Key Lab CAD&CG Hangzhou Zhejiang Peoples R China

The performance of many machine learning algorithms depends crucially on the hyperparameter settings, especially in Deep Learning. Manually tuning the hyperparameters is laborious and time consuming. To address this issue, Bayesian optimization (BO) methods and their extensions have been proposed to optimize the hyperparameters automatically. However, they still suffer from highly computational expense when applying to deep generative models (DGMs) due to their strategy of the black-box function optimization. This paper provides a new hyperparameter optimization procedure at the pre-training phase of the DGMs, where we avoid combining all layers as one black-box function by taking advantage of the layer-by-layer learning strategy. Following this procedure, we are able to optimize multiple hyperparameters in an adaptive way by using Gaussian process. In contrast to the traditional BO methods, which mainly focus on the supervised models, the pre-training procedure is unsupervised where there is no validation error can be used. To alleviate this problem, this paper proposes a new holdout loss, the free energy gap, which takes into account both factors of the model fitting and over-fitting. The empirical evaluations demonstrate that our method not only speeds up the process of hyperparameter optimization, but also improves the performances of DGMs significantly in both the supervised and unsupervised learning tasks. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Deep generative model Hyperparameter optimization sequential model-based optimization Contrastive divergence

来源：评论

学校读者我要写书评

暂无评论

Hyperparameter optimization with Factorized Multilayer Perceptrons

Hyperparameter Optimization with Factorized Multilayer Perce...

引用

European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECMLPKDD)

作者： Schilling, Nicolas Wistuba, Martin Drumond, Lucas Schmidt-Thieme, Lars Univ Hildesheim Informat Syst & Machine Learning Lab D-31141 Hildesheim Germany

ISBN: (纸本)9783319235257;9783319235240

In machine learning, hyperparameter optimization is a challenging task that is usually approached by experienced practitioners or in a computationally expensive brute-force manner such as grid-search. Therefore, recent research proposes to use observed hyperparameter performance on already solved problems (i.e. data sets) in order to speed up the search for promising hyperparameter configurations in the sequential model based optimization framework. In this paper, we propose multilayer perceptrons as surrogate models as they are able to model highly nonlinear hyperparameter response surfaces. However, since interactions of hyperparameters, data sets and metafeatures are only implicitly learned in the subsequent layers, we improve the performance of multilayer perceptrons by means of an explicit factorization of the interaction weights and call the resulting model a factorized multilayer perceptron. Additionally, we evaluate different ways of obtaining predictive uncertainty, which is a key ingredient for a decent tradeoff between exploration and exploitation. Our experimental results on two public meta data sets demonstrate the efficiency of our approach compared to a variety of published baselines. For reproduction purposes, we make our data sets and all the program code publicly available on our supplementary webpage.

关键词： Hyperparameter optimization sequential model-based optimization

来源：评论

学校读者我要写书评

暂无评论

Joint model Choice and Hyperparameter optimization with Factorized Multilayer Perceptrons 27

Joint Model Choice and Hyperparameter Optimization with Fact...

引用

27th IEEE International Conference on Tools with Artificial Intelligence (ICTAI)

作者： Schilling, Nicolas Wistuba, Martin Drumond, Lucas Schmidt-Thieme, Lars Univ Hildesheim Informat Syst & Machine Learning Lab D-31141 Hildesheim Germany

ISBN: (纸本)9781509001637

Recent work has demonstrated that hyperparameter optimization within the sequential model-based optimization (SMBO) framework is generally possible. This approach replaces the expensive-to-evaluate function that maps hyperparameters to the performance of a learned model on validation data by a surrogate model which is much cheaper to evaluate. The current state of the art in hyperparameter optimization learns these surrogate models across a variety of solved data sets where a grid search has already been employed. In this way, surrogate models are learned across data sets, and thus able to generalize better. However, meta features that describe characteristics of a data set are usually needed in order for the surrogate model to differentiate between same hyperparameter configurations on different data sets. Another research area that is closely related focuses on model choice, i.e. picking the right model for a given task, which is also a problem that many practitioners face in machine learning. In this paper, we aim to solve both of these problems with a unified surrogate model that learns across different data sets, different classifiers and their respective hyperparameters. We employ factorized multilayer perceptrons, a surrogate model that consists of a multilayer perceptron architecture, but offers the prediction of a factorization machine in the first layer. In this way, data sets, models and hyperparameters are being represented in a joint lower dimensional latent feature space. Experiments on a publicly available meta data set containing 59 individual data sets and 19 prediction models demonstrate the efficiency of our approach.

关键词： Hyperparameter optimization model Choice sequential model-based optimization

来源：评论

学校读者我要写书评

暂无评论

BayesOpt: A Bayesian optimization Library for Nonlinear optimization, Experimental Design and Bandits

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2014年 15卷 3735-3739页

作者： Martinez-Cantin, Ruben Ctr Univ Defensa Zaragoza 50090 Spain

BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization characterized for being sample efficient as it builds a posterior distribution to capture the evidence and prior knowledge of the target function. Built in standard C++, the library is extremely efficient while being portable and flexible. It includes a common interface for C, C++, Python, Matlab and Octave.

关键词： Bayesian optimization efficient global optimization sequential model-based optimization sequential experimental design Gaussian processes

来源：评论

学校读者我要写书评

暂无评论

Joint model Choice and Hyperparameter optimization with Factorized Multilayer Perceptrons

Joint Model Choice and Hyperparameter Optimization with Fact...

引用

International Conference on Tools with Artificial Intelligence

作者： Nicolas Schilling Martin Wistuba Lucas Drumond Lars Schmidt-Thieme University of Hildesheim Information Systems and Machine Learning Lab Hildesheim Germany

ISBN: (纸本)9781509001644

Recent work has demonstrated that hyperparameter optimization within the sequential model-based optimization (SMBO) framework is generally possible. This approach replaces the expensive-to-evaluate function that maps hyperpa-rameters to the performance of a learned model on validation data by a surrogate model which is much cheaper to evaluate. The current state of the art in hyperparameter optimization learns these surrogate models across a variety of solved data sets where a grid search has already been employed. In this way, surrogate models are learned across data sets, and thus able to generalize better. However, meta features that describe characteristics of a data set are usually needed in order for the surrogate model to differentiate between same hyperparameter configurations on different data sets. Another research area that is closely related focuses on model choice, i.e. picking the right model for a given task, which is also a problem that many practitioners face in machine learning. In this paper, we aim to solve both of these problems with a unified surrogate model that learns across different data sets, different classifiers and their respective hyperparameters. We employ factorized multilayer perceptrons, a surrogate model that consists of a multilayer perceptron architecture, but offers the prediction of a factorization machine in the first layer. In this way, data sets, models and hyperparameters are being represented in a joint lower dimensional latent feature space. Experiments on a publicly available meta data set containing 59 individual data sets and 19 prediction models demonstrate the efficiency of our approach.

关键词： Hyperparameter optimization model Choice sequential model-based optimization

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：