检索结果-内蒙古大学图书馆

Learning vector quantization: The dynamics of winner-takes-all algorithms

NEUROCOMPUTING 2006年第7-9期69卷 660-670页

作者： Biehl, M Ghosh, A Hammer, B Univ Groningen NL-9700 AV Groningen Netherlands Tech Univ Clausthal Inst Comp Sci D-98678 Clausthal Zellerfeld Germany

winner-takes-all (WTA) prescriptions for learning vector quantization (LVQ) are studied in the framework of a model situation: two competing prototype vectors are updated according to a sequence of example data drawn from a mixture of Gaussialls. The theory of on-line learning allows for an exact mathernatical description of the training dynamics, even if an underlying cost function cannot be identified. We compare the typical behavior of several WTA schemes including basic LVQ and unsupervised vector quantization. The focus is on the learning curves, i.e. the achievable generalization ability as a function of the number of training examples. (c) 2005 Elsevier B.V. all rights reserved.

关键词： learning vector quantization prototype-based classification winner-takes-all algorithms on-line learning competitive learning

来源：评论

学校读者我要写书评

暂无评论

Dynamics and generalization ability of LVQ algorithms

引用

JOURNAL OF MACHINE LEARNING RESEARCH 2007年第2期8卷 323-360页

作者： Biehl, Michael Ghosh, Anarta Hammer, Barbara Univ Groningen Inst Math & Comp Sci NL-9700 AV Groningen Netherlands Tech Univ Clausthal Inst Comp Sci D-38678 Clausthal Zellerfeld Germany

Learning vector quantization (LVQ) schemes constitute intuitive, powerful classification heuristics with numerous successful applications but, so far, limited theoretical background. We study LVQ rigorously within a simplifying model situation: two competing prototypes are trained from a sequence of examples drawn from a mixture of Gaussians. Concepts from statistical physics and the theory of on-line learning allow for an exact description of the training dynamics in high-dimensional feature space. The analysis yields typical learning curves, convergence properties, and achievable generalization abilities. This is also possible for heuristic training schemes which do not relate to a cost function. We compare the performance of several algorithms, including Kohonen's LVQ1 and LVQ+/-, a limiting case of LVQ2.1. The former shows close to optimal performance, while LVQ+/- displays divergent behavior. We investigate how early stopping can overcome this difficulty. Furthermore, we study a crisp version of robust soft LVQ, which was recently derived from a statistical formulation. Surprisingly, it exhibits relatively poor generalization. Performance improves if a window for the selection of data is introduced;the resulting algorithm corresponds to cost function based LVQ2. The dependence of these results on the model parameters, for example, prior class probabilities, is investigated systematically, simulations confirm our analytical findings.

关键词： prototype based classification learning vector quantization winner-takes-all algorithms on line learning competitive learning

来源：评论

学校读者我要写书评

暂无评论

Learning dynamics and robustness of vector quantization and neural gas

引用

NEUROCOMPUTING 2008年第7-9期71卷 1210-1219页

作者： Witoelar, Aree Biehl, Michael Ghosh, Anarta Hammer, Barbara Univ Groningen NL-9700 AV Groningen Netherlands Univ Washington WaNPRC Seattle WA 98195 USA Clausthal Univ Technol Inst Comp Sci D-98678 Clausthal Zellerfeld Germany

Various alternatives have been developed to improve the winner-takes-all (WTA) mechanism in vector quantization, including the neural gas (NG). However, the behavior of these algorithms including their learning dynamics, robustness with respect to initialization, asymptotic results. etc. has only partially been studied in a rigorous mathematical analysis. The theory of on-line learning allows for an exact mathematical description of the training dynamics in model situations. We demonstrate using a system of three competing prototypes trained from a mixture of Gaussian clusters that the NG can improve convergence speed and achieves robustness to initial conditions. However, depending on the structure of the data, the NG does not always obtain the best asymptotic quantization error. (c) 2008 Elsevier B.V. all rights reserved.

关键词： vector quantization clustering online learning winner-takes-all algorithms neural gas

来源：评论

学校读者我要写书评

暂无评论

Phase transitions in vector quantization and neural gas

引用

NEUROCOMPUTING 2009年第7-9期72卷 1390-1397页

作者： Witoelar, Aree Biehl, Michael Univ Groningen NL-9700 AK Groningen Netherlands

The statistical physics of off-learning is applied to winner-takes-all (WTA) and rank-based vector quantization (VQ), including the neural gas (NG). The analysis is based on the limit of high training temperatures and the annealed approximation. The typical learning behavior is evaluated for systems of two and three prototypes with data drawn from a mixture of high dimensional Gaussian clusters. The learning curves exhibit phase transitions, i.e. a critical or discontinuous dependence of performances on the training set size and training temperature. We show how the nature and properties of the transition depend on the number of prototypes and the control parameter of rank-based cost functions. The NG-based systems are demonstrated to give an advantage over WTA in terms of robustness to initial conditions. (C) 2009 Elsevier B.V. all rights reserved.

关键词： Vector quantization Clustering Offline learning winner-takes-all algorithms Neural gas

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：