检索结果-内蒙古大学图书馆

Information-Theoretic Characterizations of Generalization Error for the gibbs algorithm

IEEE TRANSACTIONS ON INFORMATION THEORY 2024年第1期70卷 632-655页

作者： Aminian, Gholamali Bu, Yuheng Toni, Laura Rodrigues, Miguel R. D. Wornell, Gregory W. Alan Turing Inst London NW1 2DB England MIT Dept Elect Engn & Comp Sci Cambridge MA 02139 USA Univ Florida Dept Elect & Comp Engn Gainesville FL 32611 USA UCL Elect & Elect Engn Dept London WC1E 6BT England

Various approaches have been developed to upper bound the generalization error of a supervised learning algorithm. However, existing bounds are often loose and even vacuous when evaluated in practice. As a result, they may fail to characterize the exact generalization ability of a learning algorithm. Our main contributions are exact characterizations of the expected generalization error of the well-known gibbs algorithm (a.k.a. gibbs posterior) using different information measures, in particular, the symmetrized KL information between the input training samples and the output hypothesis. Our result can be applied to tighten existing expected generalization errors and PAC-Bayesian bounds. Our information-theoretic approach is versatile, as it also characterizes the generalization error of the gibbs algorithm with a data-dependent regularizer and that of the gibbs algorithm in the asymptotic regime, where it converges to the standard empirical risk minimization algorithm. Of particular relevance, our results highlight the role the symmetrized KL information plays in controlling the generalization error of the gibbs algorithm.

关键词： Empirical risk minimization generalization error gibbs algorithm PAC-Bayesian learning symmetrized KL information

来源：评论

学校读者我要写书评

暂无评论

A Bayesian Tweedie exponential dispersion process with a change-point for two-phase degradation data

引用

IISE TRANSACTIONS 2025年

作者： Wang, Pingping Bae, Suk Joo Nanjing Univ Finance & Econ Dept Stat Nanjing Jiangsu Peoples R China Hanyang Univ Dept Ind Engn Seoul South Korea

Degradation analysis is an efficient way to evaluate a product's reliability in various applications by capturing its degradation characteristics quickly and accurately. In the degradation analysis, the accuracy of the reliability inference depends highly on the selected model for fitting observed degradation measurements. Covering popular stochastic process models as special cases, the Tweedie exponential dispersion process (TEDP) model has potential in describing more extensive degradation phenomena in engineering applications. In this work, we propose a TEDP with a change-point (CPTEDP) to model two-phase degradation pattern of products, where unit-specific two-piece drifts are considered to explain heterogeneous degradation patterns of testing units. We introduce a three-stage hierarchical Bayesian inference on the parameters of the CPTEDP model without the need for approximating its likelihood function that has no closed-form expression using Markov chain Monte Carlo (MCMC). To overcome difficulties posed by the continuity constraint of degradation processes on modeling and computation under the hierarchical Bayesian framework, we adopt the pseudo likelihood function instead of the real likelihood for computational efficiency. After fitting the three-stage hierarchical Bayesian CPTEDP model by adopting the gibbs algorithm, we derive the approximations of reliability measures (e.g., mean time to failure (MTTF), remaining useful life (RUL), mean residual life (MRL)) of explicit analytical forms by substituting posterior estimates obtained from the gibbs algorithm. Analytical results from a case study and a variety of simulations demonstrate that the hierarchical Bayesian CPTEDP model can provide more suitable fits to degradation data with two-phase patterns, improving the accuracy of reliability estimation for products.

关键词： Gamma process gibbs algorithm inverse Gaussian process remaining useful life Wiener process

来源：评论

学校读者我要写书评

暂无评论

Bayesian diagnostics in a partially linear model with first-order autoregressive skew-normal errors

引用

COMPUTATIONAL STATISTICS 2025年第2期40卷 1021-1051页

作者： Liu, Yonghui Lu, Jiawei Paula, Gilberto A. Liu, Shuangzhe Shanghai Univ Int Business & Econ Sch Stat & Informat Shanghai Peoples R China Univ Sao Paulo Inst Math & Stat Sao Paulo Brazil Univ Canberra Fac Sci & Technol Canberra Australia

This paper studies a Bayesian local influence method to detect influential observations in a partially linear model with first-order autoregressive skew-normal errors. This method appears suitable for small or moderate-sized data sets (n=200 similar to 400\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n=200{\sim }400$$\end{document}) and overcomes some theoretical limitations, bridging the diagnostic gap for small or moderate-sized data in classical methods. The MCMC algorithm is employed for parameter estimation, and Bayesian local influence analysis is made using three perturbation schemes (priors, variances, and data) and three measurement scales (Bayes factor, phi\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\phi $$\end{document}-divergence, and posterior mean). Simulation studies are conducted to validate the reliability of the diagnostics. Finally, a practical application uses data on the 1976 Los Angeles ozone concentration to further demonstrate the effectiveness of the diagnostics.

关键词： Bayesian local influence method gibbs algorithm Matrix differential calculus Time series model

来源：评论

学校读者我要写书评

暂无评论

A Markov Chain Monte Carlo (MCMC) gibbs Sampler Augmented with Zero Forcing Detection for OTFS Reception 2

A Markov Chain Monte Carlo (MCMC) Gibbs Sampler Augmented wi...

引用

2nd IEEE Wireless, Antenna and Microwave Symposium (WAMS)

作者： Subramaniam, M. Avinash Vasavada, Yash Dhirubhai Ambani Inst Informat & Commun Technol Gandhinagar India

ISBN: (纸本)9798350312430

The Orthogonal Time Frequency Space (OTFS) is a two-dimensional modulation scheme that operates in the Delay-Doppler domain and is robust in the presence of wireless channels experiencing a high Doppler spread. The OTFS converts a fast-varying doubly-dispersive wireless channel in the time-frequecy domain to a near-static channel in the Delay-Doppler domain. The informative symbols are transmitted in this Delay-Doppler domain with the effect that the transmitted symbols experience a channel fade that changes comparatively slowly with time. In this paper, we propose an enhanced Markov Chain Monte Carlo Markov Chain (MCMC) based gibbs sampling scheme for the detection of transmitted symbols at the OTFS receiver. A novelty of our proposal is to combine the Zero Forcing Detection (ZFD) with the MCMC-gibbs algorithm. We show that the proposed joint gibbs-ZFD method avoids a performance loss that the gibbs method exhibits at a high signal to noise ratios (SNRs). We conduct a simulation-based evaluation of our proposed approach over a range of the UE (User Equipment) speeds and for Binary Phase Shift Keying (BPSK) and Quadrature Phase Shift Keying (QPSK) modulation schemes. The numerical results show that our proposal offers an improved probability of symbol detection error compared to the two alternative OTFS receivers, namely the gibbs method and the ZFD.

关键词： OTFS MCMC gibbs algorithm Zero-Forcing

来源：评论

学校读者我要写书评

暂无评论

Efficient Gaussian Sampling for Solving Large-Scale Inverse Problems Using MCMC

引用

IEEE TRANSACTIONS ON SIGNAL PROCESSING 2015年第1期63卷 70-80页

作者： Gilavert, Clement Moussaoui, Said Idier, Jerome Ecole Cent Nantes CNRS UMR 6597 IRCCyN F-44321 Nantes 3 France

The resolution of many large-scale inverse problems using MCMC methods requires a step of drawing samples from a high dimensional Gaussian distribution. While direct Gaussian sampling techniques, such as those based on Cholesky factorization, induce an excessive numerical complexity and memory requirement, sequential coordinate sampling methods present a low rate of convergence. Based on the reversible jump Markov chain framework, this paper proposes an efficient Gaussian sampling algorithm having a reduced computation cost and memory usage, while maintaining the theoretical convergence of the sampler. The main feature of the algorithm is to perform an approximate resolution of a linear system with a truncation level adjusted using a self-tuning adaptive scheme allowing to achieve the minimal computation cost per effective sample. The connection between this algorithm and some existing strategies is given and its performance is illustrated on a linear inverse problem of image resolution enhancement.

关键词： Adaptive MCMC conjugate gradient gibbs algorithm multivariate Gaussian sampling reversible jump Monte Carlo

来源：评论

学校读者我要写书评

暂无评论

Follow a guide to solve urban problems: the creation and application of urban knowledge graph

引用

IET SOFTWARE 2017年第3期11卷 126-134页

作者： Shan, Shubing Cao, Buyang Tongji Univ Sch Software Engn 4800 Caoan Rd Shanghai Peoples R China China Intelligent Urbanizat Cocreat Ctr 1239 Siping Rd Shanghai Peoples R China

It is a hot research topic today to find out the potential knowledge from the scattered urban data and take advantage of the relationship between the knowledge to solve the challenges of urban governance and smart city construction. The urban knowledge graph is an effective way to establish the relationship between the knowledge and address the urban issues. This study proposes the methodology to create an urban knowledge graph and its framework. It elaborates the approaches of urban knowledge acquisition, reasoning and expression. Furthermore, a hybrid reasoning algorithm known as EG is given based on expectation-maximisation algorithm and gibbs algorithm, which has the complementary advantages of the both methods. Through a study case, this study illustrates constructing and working process of an urban knowledge graph. The case shows that the urban knowledge graph has a good application prospect.

关键词： knowledge acquisition inference mechanisms graph theory expectation-maximisation algorithm urban problems urban knowledge graph urban knowledge acquisition hybrid reasoning algorithm EG expectation-maximisation algorithm gibbs algorithm

来源：评论

学校读者我要写书评

暂无评论

Information-theoretic upper and lower bounds for statistical estimation

引用

IEEE TRANSACTIONS ON INFORMATION THEORY 2006年第4期52卷 1307-1321页

作者： Zhang, T Yahoo Inc New York NY 10011 USA

In this paper, we establish upper and lower bounds for some statistical estimation problems through concise information-theoretic arguments. Our upper bound analysis is based on a simple yet general inequality which we call the information exponential inequality. We show that this inequality naturally leads to a general randomized estimation method, for which performance upper bounds can be obtained. The lower bounds, applicable for all statistical estimators, are obtained by original applications of some well known information-theoretic inequalities, and approximately match the obtained upper bounds for various important problems. Moreover, our framework can be regarded as a natural generalization of the standard minimax framework, in that we allow the performance of the estimator to vary for different possible underlying distributions according to a predefined prior.

关键词： gibbs algorithm lower bound minimax PAC-Bayes randomized estimatin statistical estimation

来源：评论

学校读者我要写书评

暂无评论

Bayesian estimation of the half-normal regression model with deterministic frontier

引用

COMPUTATIONAL STATISTICS 2016年第3期31卷 1059-1078页

作者： Ortega, Francisco J. Gavilan, Jose M. Univ Seville Ave Ramon & Cajal 1 Seville Spain

A regression model with deterministic frontier is considered. This type of model has hardly been studied, partly owing to the difficulty in the application of maximum likelihood estimation since this is a non-regular model. As an alternative, the Bayesian methodology is proposed and analysed. Through the gibbs algorithm, the inference of the parameters of the model and of the individual efficiencies are relatively straightforward. The results of the simulations indicate that the utilized method performs reasonably well.

关键词： Deterministic frontier Bayesian estimation gibbs algorithm

来源：评论

学校读者我要写书评

暂无评论

Parametric inference for mixed models defined by stochastic differential equations

引用

ESAIM - Probability and Statistics 2008年第1期12卷 196-218页

作者： Donnet, Sophie Samson, Adeline Paris-Sud University Laboratoire de Mathématiques Orsay France UMR CNRS 8145 University Paris 5 Paris France INSERM U738 Paris France University Paris 5 Paris France

Non-linear mixed models defined by stochastic differential equations (SDEs) are considered: the parameters of the diffusion process are random variables and vary among the individuals. A maximum likelihood estimation method based on the Stochastic Approximation EM algorithm, is proposed. This estimation method uses the Euler-Maruyama approximation of the diffusion, achieved using latent auxiliary data introduced to complete the diffusion process between each pair of measurement instants. A tuned hybrid gibbs algorithm based on conditional Brownian bridges simulations of the unobserved process paths is included in this algorithm. The convergence is proved and the error induced on the likelihood by the Euler-Maruyama approximation is bounded as a function of the step size of the approximation. Results of a pharmacokinetic simulation study illustrate the accuracy of this estimation method. The analysis of the Theophyllin real dataset illustrates the relevance of the SDE approach relative to the deterministic approach. © 2008 EDP Sciences SMAI.

关键词： Brownian bridge diffusion process Euler-Maruyama approximation gibbs algorithm incomplete data model maximum likelihood estimation non-linear mixed effects model SAEM algorithm

来源：评论

学校读者我要写书评

暂无评论

Bayesian analysis of skew-t multivariate null intercept measurement error model

引用

STATISTICAL PAPERS 2010年第3期51卷 531-545页

作者： Lachos, Victor H. Cancho, Vicente G. Aoki, Reiko Univ Sao Paulo ICMC Dept Matemat Aplicada & Estat BR-13560970 Sao Paulo Brazil Univ Estadual Campinas IMECC Dept Estat BR-13083859 Sao Paulo Brazil

The multivariate skew-t distribution (J Multivar Anal 79:93-113, 2001;J R Stat Soc, Ser B 65:367-389, 2003;Statistics 37:359-363, 2003) includes the Student t, skew-Cauchy and Cauchy distributions as special cases and the normal and skew-normal ones as limiting cases. In this paper, we explore the use of Markov Chain Monte Carlo (MCMC) methods to develop a Bayesian analysis of repeated measures, pretest/post-test data, under multivariate null intercept measurement error model (J Biopharm Stat 13(4):763-771, 2003) where the random errors and the unobserved value of the covariate (latent variable) follows a Student t and skew-t distribution, respectively. The results and methods are numerically illustrated with an example in the field of dentistry.

关键词： Skew-t distribution gibbs algorithm Metropolis-Hasting Skewness Multivariate null intercepts model Measurement error

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：