DNB: A Joint Learning Framework for Deep Bayesian Nonparametric Clustering

作     者：Wang, Zeya Ni, Yang Jing, Baoyu Wang, Deqing Zhang, Hao Xing, Eric 

作者机构：Petuum Inc Pittsburgh PA 15222 USA Texas A&M Univ Dept Stat College Stn TX 77843 USA Univ Illinois Dept Comp Sci Champaign IL 61820 USA Beihang Univ Sch Comp Sci Beijing 100083 Peoples R China 

出 版 物：《IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS》 (IEEE Trans. Neural Networks Learn. Sys.)

年 卷 期：2022年第33卷第12期

页      面：7610-7620页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：National Science Foundation (NSF) [DMS-1918851] 

主　　题：Clustering algorithms Task analysis Unsupervised learning Training Neural networks Generative adversarial networks Feature extraction Bayesian nonparametrics (BNPs) convolutional neural network (CNN) image clustering joint learning regularization 

摘      要：Clustering algorithms based on deep neural networks have been widely studied for image analysis. Most existing methods require partial knowledge of the true labels, namely, the number of clusters, which is usually not available in practice. In this article, we propose a Bayesian nonparametric framework, deep nonparametric Bayes (DNB), for jointly learning image clusters and deep representations in a doubly unsupervised manner. In doubly unsupervised learning, we are dealing with the problem of ``unknown unknowns, where we estimate not only the unknown image labels but also the unknown number of labels as well. The proposed algorithm alternates between generating a potentially unbounded number of clusters in the forward pass and learning the deep networks in the backward pass. With the help of the Dirichlet process mixtures, the proposed method is able to partition the latent representations space without specifying the number of clusters a priori. An important feature of this work is that all the estimation is realized with an end-to-end solution, which is very different from the methods that rely on post hoc analysis to select the number of clusters. Another key idea in this article is to provide a principled solution to the problem of ``trivial solution for deep clustering, which has not been much studied in the current literature. With extensive experiments on benchmark datasets, we show that our doubly unsupervised method achieves good clustering performance and outperforms many other unsupervised image clustering methods.