检索结果-内蒙古大学图书馆

data Imputation by Pursuing Better Classification: A Supervised Kernel-Based Method

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yang, Ruikai He, Fan He, Mingzhen Wang, Kaijie Huang, Xiaolin Institute of Image Processing and Pattern Recognition Department of Automation Shanghai Jiao Tong University 800 Dongchuan RD Shanghai200240 China STADIUS Center for Dynamical Systems Signal Processing and Data Analytics KU Leuven Oude Markt 13 Leuven3000 Belgium

data imputation, the process of filling in missing feature elements for incomplete data sets, plays a crucial role in data-driven learning. A fundamental belief is that data imputation is helpful for learning performance, and it follows that the pursuit of better classification can guide the data imputation process. While some works consider using label information to assist in this task, their simplistic utilization of labels lacks flexibility and may rely on strict assumptions. In this paper, we propose a new framework that effectively leverages supervision information to complete missing data in a manner conducive to classification. Specifically, this framework operates in two stages. Firstly, it leverages labels to supervise the optimization of similarity relationships among data, represented by the kernel matrix, with the goal of enhancing classification accuracy. To mitigate overfitting that may occur during this process, a perturbation variable is introduced to improve the robustness of the framework. Secondly, the learned kernel matrix serves as additional supervision information to guide data imputation through regression, utilizing the block coordinate descent method. The superiority of the proposed method is evaluated on four real-world data sets by comparing it with state-of-the-art imputation methods. Remarkably, our algorithm significantly outperforms other methods when the data is missing more than 60% of the features. Copyright © 2024, The Authors. All rights reserved.

关键词： Self-supervised learning

ENHANCING KERNEL FLEXIBILITY VIA LEARNING ASYMMETRIC LOCALLY-ADAPTIVE KERNELS

学校读者我要写书评

暂无评论

arXiv 2023年

作者： He, Fan He, Mingzhen Shi, Lei Huang, Xiaolin Suykens, Johan A.K. STADIUS Center for Dynamical Systems Signal Processing and Data Analytics KU Leuven Belgium Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China Shanghai Key Laboratory for Contemporary Applied Mathematics School of Mathematical Sciences Fudan University China

The lack of sufficient flexibility is the key bottleneck of kernel-based learning that relies on manually designed, pre-given, and non-trainable kernels. To enhance kernel flexibility, this paper introduces the concept of Locally-Adaptive-Bandwidths (LAB) as trainable parameters to enhance the Radial Basis Function (RBF) kernel, giving rise to the LAB RBF kernel. The parameters in LAB RBF kernels are data-dependent, and its number can increase with the dataset, allowing for better adaptation to diverse data patterns and enhancing the flexibility of the learned function. This newfound flexibility also brings challenges, particularly with regards to asymmetry and the need for an efficient learning algorithm. To address these challenges, this paper for the first time establishes an asymmetric kernel ridge regression framework and introduces an iterative kernel learning algorithm. This novel approach not only reduces the demand for extensive support data but also significantly improves generalization by training bandwidths on the available training data. Experimental results on real datasets underscore the remarkable performance of the proposed algorithm, showcasing its superior capability in handling large-scale datasets compared to Nyström approximation-based algorithms. Moreover, it demonstrates a significant improvement in regression accuracy over existing kernel-based learning methods and even surpasses residual neural networks. Copyright © 2023, The Authors. All rights reserved.

关键词： Bandwidth

Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： He, Fan He, Mingzhen Shi, Lei Huang, Xiaolin Suykens, Johan A.K. STADIUS Center for Dynamical Systems Signal Processing and Data Analytics KU Leuven Leuven Belgium MOE Key Laboratory of System Control and Information Processing Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Shanghai China Shanghai Key Laboratory for Contemporary Applied Mathematics School of Mathematical Sciences Fudan University Shanghai200433 China Shanghai Artificial Intelligence Laboratory Shanghai200232 China MOE Key Laboratory of System Control and Information Processing Institute of Image Processing and Pattern Recognition Institute of Medical Robotics Shanghai Jiao Tong University Shanghai200240 China

Ridgeless regression has garnered attention among researchers, particularly in light of the "Benign Overfitting" phenomenon, where models interpolating noisy samples demonstrate robust generalization. However, kernel ridgeless regression does not always perform well due to the lack of flexibility. This paper enhances kernel ridgeless regression with Locally-Adaptive-Bandwidths (LAB) RBF kernels, incorporating kernel learning techniques to improve performance in both experiments and theory. For the first time, we demonstrate that functions learned from LAB RBF kernels belong to an integral space of Reproducible Kernel Hilbert Spaces (RKHSs). Despite the absence of explicit regularization in the proposed model, its optimization is equivalent to solving an 0-regularized problem in the integral space of RKHSs, elucidating the origin of its generalization ability. Taking an approximation analysis viewpoint, we introduce an lq-norm analysis technique (with 0 © 2024, CC BY.

关键词： Vector spaces

Robust anomaly detection in videos using multilevel representations 33

学校读者我要写书评

暂无评论

Robust anomaly detection in videos using multilevel represen...

33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Annual Conference on Innovative Applications of Artificial Intelligence, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019

作者： Vu, Hung Nguyen, Tu Dinh Le, Trung Luo, Wei Phung, Dinh Center for Pattern Recognition and Data Analytics Deakin University Geelong Australia Monash University ClaytonVIC3800 Australia

ISBN: (纸本)9781577358091

Detecting anomalies in surveillance videos has long been an important but unsolved problem. In particular, many existing solutions are overly sensitive to (often ephemeral) visual artifacts in the raw video data, resulting in false positives and fragmented detection regions. To overcome such sensitivity and to capture true anomalies with semantic significance, one natural idea is to seek validation from abstract representations of the videos. This paper introduces a framework of robust anomaly detection using multilevel representations of both intensity and motion data. The framework consists of three main components: 1) representation learning using Denoising Autoencoders, 2) level-wise representation generation using Conditional Generative Adversarial Networks, and 3) consolidating anomalous regions detected at each representation level. Our proposed multilevel detector shows a significant improvement in pixel-level Equal Error Rate, namely 11.35%, 12.32% and 4.31% improvement in UCSD Ped 1, UCSD Ped 2 and Avenue datasets respectively. In addition, the model allowed us to detect mislabeled anomalies in the UCDS Ped 1. Copyright © 2019, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Anomaly detection

A privacy preserving bayesian optimization with high efficiency 22nd

学校读者我要写书评

暂无评论

A privacy preserving bayesian optimization with high efficie...

22nd Pacific-Asia Conference on Advances in Knowledge Discovery and data Mining, PAKDD 2018

作者： Nguyen, Thanh Dai Gupta, Sunil Rana, Santu Venkatesh, Svetha Center for Pattern Recognition and Data Analytics Deakin University Waurn Ponds3216 Australia

ISBN: (纸本)9783319930398

Bayesian optimization is a powerful machine learning technique for solving experimental design problems. With its use in industrial design optimization, time and cost of industrial processes can be reduced significantly. However, often the experimenters in industries may not have the expertise of optimization techniques and may require help from third-party optimization services. This can cause privacy concerns as the optimized design of an industrial process typically needs to be kept secret to retain its competitive advantages. To this end, we propose a novel Bayesian optimization algorithm that can allow the experimenters from an industry to utilize the expertise of a third-party optimization service in privacy preserving manner. Privacy of our proposed algorithm is guaranteed under a modern privacy preserving framework called Error Preserving Privacy, especially designed to maintain high utility even under the privacy restrictions. Using several benchmark optimization problems as well as optimization problems from real-world industrial processes, we demonstrate that the optimization efficiency of our algorithm is comparable to the non-private Bayesian optimization algorithm and significantly better than its differential privacy counterpart. © Springer International Publishing AG, part of Springer Nature 2018.

关键词： Product design

Trans2Vec: Learning transaction embedding via items and frequent itemsets 22nd

学校读者我要写书评

暂无评论

Trans2Vec: Learning transaction embedding via items and freq...

22nd Pacific-Asia Conference on Advances in Knowledge Discovery and data Mining, PAKDD 2018

作者： Nguyen, Dang Nguyen, Tu Dinh Luo, Wei Venkatesh, Svetha Center for Pattern Recognition and Data Analytics School of Information Technology Deakin University Geelong Australia

ISBN: (纸本)9783319930398

Learning meaningful and effective representations for transaction data is a crucial prerequisite for transaction classification and clustering tasks. Traditional methods which use frequent itemsets (FIs) as features often suffer from the data sparsity and high-dimensionality problems. Several supervised methods based on discriminative FIs have been proposed to address these disadvantages, but they require transaction labels, thus rendering them inapplicable to real-world applications where labels are not given. In this paper, we propose an unsupervised method which learns low-dimensional continuous vectors for transactions based on information of both singleton items and FIs. We demonstrate the superior performance of our proposed method in classifying transactions on four datasets compared with several state-of-the-art baselines. © Springer International Publishing AG, part of Springer Nature 2018.

关键词： Classification (of information)

Geometric enclosing networks 27

学校读者我要写书评

暂无评论

Geometric enclosing networks

27th International Joint Conference on Artificial Intelligence, IJCAI 2018

作者： Le, Trung Vu, Hung Nguyen, Tu Dinh Phung, Dinh Faculty of Information Technology Monash University Australia Center for Pattern Recognition and Data Analytics Deakin University Australia

ISBN: (纸本)9780999241127

Training model to generate data has increasingly attracted research attention and become important in modern world applications. We propose in this paper a new geometry-based optimization approach to address this problem. Orthogonal to current state-of-the-art density-based approaches, most notably VAE and GAN, we present a fresh new idea that borrows the principle of minimal enclosing ball to train a generator G (z) in such a way that both training and generated data, after being mapped to the feature space, are enclosed in the same sphere. We develop theory to guarantee that the mapping is bijective so that its inverse from feature space to data space results in expressive nonlinear contours to describe the data manifold, hence ensuring data generated are also lying on the data manifold learned from training data. Our model enjoys a nice geometric interpretation, hence termed Geometric Enclosing Networks (GEN), and possesses some key advantages over its rivals, namely simple and easy-to-control optimization formulation, avoidance of mode collapsing and efficiently learn data manifold representation in a completely unsupervised manner. We conducted extensive experiments on synthesis and real-world datasets to illustrate the behaviors, strength and weakness of our proposed GEN, in particular its ability to handle multi-modal data and quality of generated data. © 2018 International Joint Conferences on Artificial Intelligence. All right reserved.

关键词： Geometry

Batch Normalized Deep Boltzmann Machines 10

学校读者我要写书评

暂无评论

Batch Normalized Deep Boltzmann Machines

10th Asian Conference on Machine Learning, ACML 2018

作者： Vu, Hung Nguyen, Tu Dinh Le, Trung Luo, Wei Phung, Dinh Center for Pattern Recognition and Data Analytics Deakin University Geelong Australia Monash University ClaytonVIC3800 Australia

Training Deep Boltzmann Machines (DBMs) is a challenging task in deep generative model studies. The careless training usually leads to a divergence or a useless model. We discover that this phenomenon is due to the change of DBM layers’ input signals during model parameter updates, similar to other deterministic deep networks such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs). The change of layers’ input distributions not only complicates the learning process but also causes redundant neurons that simply imitate the others’ behaviors. Although this phenomenon can be coped using batch normalization in deep learning, integrating this technique into the probabilistic network of DBMs is a challenging problem since it has to satisfy two conditions of energy function and conditional probabilities. In this paper, we introduce Batch Normalized Deep Boltzmann Machines (BNDBMs) that meet both aforementioned conditions and successfully combine batch normalization and DBMs into the same framework. However, unlike CNNs, due to the probabilistic nature of DBMs, training DBMs with batch normalization has some differences: i) fixing shift parameters β but learning scale parameters γ;ii) avoiding normalizing the first hidden layer and iii) maintaining multiple pairs of population means and variances per neuron rather than one pair in CNNs. We observe that our proposed BNDBMs can stabilize the input signals of network layers and facilitate the training process as well as improve the model quality. More interestingly, BNDBMs can be trained successfully without pretraining, which is usually a mandatory step in most existing DBMs. The experimental results in MNIST, Fashion-MNIST and Caltech 101 Silhouette datasets show that our BNDBMs outperform DBMs and centered DBMs in terms of feature representation and classification accuracy (3.98% and 5.84% average improvement for pretraining and no pretraining respectively). © 2018 H. Vu, T.D. Nguyen, T. Le, W. Luo &

关键词： Network layers

Detection of unknown anomalies in streaming videos with generative energy-based Boltzmann models

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Vu, Hung Nguyen, Tu Dinh Phung, Dinh Center for Pattern Recognition and Data Analytics School of Information Technology Deakin University Geelong Australia

Abnormal event detection is one of the important objectives in research and practical applications of video surveillance. However, there are still three challenging problems for most anomaly detection systems in practical setting: limited labeled data, ambiguous definition of "abnormal" and expensive feature engineering steps. This paper introduces a unified detection framework to handle these challenges using energy-based models, which are powerful tools for unsupervised representation learning. Our proposed models are firstly trained on unlabeled raw pixels of image frames from an input video rather than hand-crafted visual features;and then identify the locations of abnormal objects based on the errors between the input video and its reconstruction produced by the models. To handle video stream, we develop an online version of our framework, wherein the model parameters are updated incrementally with the image frames arriving on the fly. Our experiments show that our detectors, using Restricted Boltzmann Machines (RBMs) and Deep Boltzmann Machines (DBMs) as core modules, achieve superior anomaly detection performance to unsupervised baselines and obtain accuracy comparable with the state-of-the-art approaches when evaluating at the pixel-level. More importantly, we discover that our system trained with DBMs is able to simultaneously perform scene clustering and scene reconstruction. This capacity not only distinguishes our method from other existing detectors but also offers a unique tool to investigate and understand how the model works. Copyright © 2018, The Authors. All rights reserved.

关键词： Video streaming