Online Learning of Single- and Multivalued Functions with an Infinite Mixture of Linear Experts

作     者：Damas, Bruno Santos-Victor, Jose 

作者机构：Inst Sistemas & Robot IST P-1049001 Lisbon Portugal Escola Super Tecnol IPS P-2910761 Setubal Portugal 

出 版 物：《NEURAL COMPUTATION》 (神经计算)

年 卷 期：2013年第25卷第11期

页      面：3044-3091页

核心收录：

学科分类：1001[医学-基础医学(可授医学、理学学位)] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

主　　题：INFINITE mixture models (Statistics) FUNCTIONS (Mathematics) DISTANCE education LINEAR models (Statistics) MACHINE learning ESTIMATION theory EXPECTATION-maximization algorithms 

摘      要：We present a supervised learning algorithm for estimation of generic input-output relations in a real-time, online fashion. The proposed method is based on a generalized expectation-maximization approach to fit an infinite mixture of linear experts (IMLE) to an online stream of data samples. This probabilistic model, while not fully Bayesian, can efficiently choose the number of experts that are allocated to the mixture, this way effectively controlling the complexity of the resulting model. The result is an incremental, online, and localized learning algorithm that performs nonlinear, multivariate regression on multivariate outputs by approximating the target function by a linear relation within each expert input domain and that can allocate new experts as needed. A distinctive feature of the proposed method is the ability to learn multivalued functions: one-to-many mappings that naturally arise in some robotic and computer vision learning domains, using an approach based on a Bayesian generative model for the predictions provided by each of the mixture experts. As a consequence, it is able to directly provide forward and inverse relations from the same learned mixture model. We conduct an extensive set of experiments to evaluate the proposed algorithm performance, and the results show that it can outperform state-of-the-art online function approximation algorithms in single-valued regression, while demonstrating good estimation capabilities in a multivalued function approximation context.