检索结果-内蒙古大学图书馆

Probabilistic learning vector quantization on manifold of symmetric positive definite matrices

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Sonna, Fengzhen Tang Feng, Haifeng Tino, Peter Si, Bailu Ji, Daxiong State Key Laboratory of Robotics Shenyang Institute of Automation Chinese Academy of Sciences Shenyang110016 China Institutes for Robotics and Intelligent Manufacturing Chinese Academy of Sciences Shenyang110169 China University of Chinese Academy of Sciences Beijing100049 China School of computer Science University of Birmingham BirminghamB15 2TT United Kingdom School of Systems Science Beijing Normal University Beijing100875 China Institute of Marine Electronics and Intelligent Systems Ocean College Zhejiang University Key Laboratory of Ocean Observation-Imaging Testbed of Zhejiang Province Engineering Research Center of Oceanic Sensing Technology and Equipment Ministry of Education Zhoushan316021 China

In this paper, we develop a new classification method for manifold-valued data in the framework of probabilistic learning vector quantization. In many classification scenarios, the data can be naturally represented by symmetric positive definite matrices, which are inherently points that live on a curved Riemannian manifold. Due to the non-Euclidean geometry of Riemannian manifolds, traditional Euclidean machine learning algorithms yield poor results on such data. In this paper, we generalize the probabilistic learning vector quantization algorithm for data points living on the manifold of symmetric positive definite matrices equipped with Riemannian natural metric (affine-invariant metric). By exploiting the induced Riemannian distance, we derive the probabilistic learning Riemannian space quantization algorithm, obtaining the learning rule through Riemannian gradient descent. Empirical investigations on synthetic data, image data, and motor imagery EEG data demonstrate the superior performance of the proposed method. © 2021, CC BY-NC-SA.

关键词： learning algorithms

Federating for learning group fair models

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Papadaki, Afroditi Martinez, Natalia Bertran, Martin Sapiro, Guillermo Rodrigues, Miguel University College London London United Kingdom Duke University DurhamNC United States

Federated learning is an increasingly popular paradigm that enables a large number of entities to collaboratively learn better models. In this work, we study minmax group fairness in paradigms where different participating entities may only have access to a subset of the population groups during the training phase. We formally analyze how this fairness objective differs from existing federated learning fairness criteria that impose similar performance across participants instead of demographic groups. We provide an optimization algorithm – FedMinMax – for solving the proposed problem that provably enjoys the performance guarantees of centralized learning algorithms. We experimentally compare the proposed approach against other methods in terms of group fairness in various federated learning setups. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2020

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Amjad, Maaz Sidorov, Grigori Zhila, Alisa Gelbukh, Alexander Rosso, Paolo Mexico United States Universitat Politècnica de València Spain

This overview paper describes the first shared task on fake news detection in Urdu language. The task was posed as a binary classification task, in which the goal is to differentiate between real and fake news. We provided a dataset divided into 900 annotated news articles for training and 400 news articles for testing. The dataset contained news in five domains: (i) Health, (ii) Sports, (iii) Showbiz, (iv) Technology, and (v) Business. 42 teams from 6 different countries (India, China, Egypt, Germany, Pakistan, and the UK) registered for the task. 9 teams submitted their experimental results. The participants used various machine learning methods ranging from feature-based traditional machine learning to neural networks techniques. The best performing system achieved an F-score value of 0.90, showing that the BERT-based approach outperforms other machine learning techniques. © 2022, CC BY.

关键词： learning algorithms

Evolving learning Rate Optimizers for deep neural networks

学校读者我要写书评

暂无评论

arXiv 2021年

作者： de Carvalho, Pedro Filipe Gomes Ramos Lourenço, Nuno Machado, Penousal University of Coimbra Department of Informatic Engineering Coimbra3030-290 Portugal

Artificial Neural Networks (ANNs) became popular due to their successful application difficult problems such image and speech recognition. However, when practitioners want to design an ANN they need to undergo laborious process of selecting a set of parameters and topology. Currently, there are several state-of-the art methods that allow for the automatic selection of some of these aspects. learning Rate optimizers are a set of such techniques that search for good values of learning rates. Whilst these techniques are effective and have yielded good results over the years, they are general solution i.e. they do not consider the characteristics of a specific network. We propose a framework called AutoLR to automatically design learning Rate Optimizers. Two versions of the system are detailed. The first one, Dynamic AutoLR, evolves static and dynamic learning rate optimizers based on the current epoch and the previous learning rate. The second version, Adaptive AutoLR, evolves adaptive optimizers that can fine tune the learning rate for each network eeight which makes them generally more effective. The results are competitive with the best state of the art methods, even outperforming them in some scenarios. Furthermore, the system evolved a classifier, ADES, that appears to be novel and innovative since, to the best of our knowledge, it has a structure that differs from state of the art *** Codes 68W50 (Primary), 68T05 (Secondary) Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

ON THE CONVERGENCE OF POLICY GRADIENT METHODS TO NASH EQUILIBRIA IN GENERAL STOCHASTIC GAMES

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Giannou, Angeliki Lotidis, Kyriakos Mertikopoulos, Panayotis Vlatakis-Gkaragkounis, Emmanouil-Vasileios University of Wisconsin-Madison United States Department of Management Science & Engineering Stanford University United States Univ. Grenoble Alpes CNRS Inria Grenoble INP LIG Grenoble38000 France Criteo AI Lab France University of California Berkeley United States

learning in stochastic games is a notoriously difficult problem because, in addition to each other’s strategic decisions, the players must also contend with the fact that the game itself evolves over time, possibly in a very complicated manner. Because of this, the convergence properties of popular learning algorithms – like policy gradient and its variants – are poorly understood, except in specific classes of games (such as potential or two-player, zero-sum games). In view of this, we examine the long-run behavior of policy gradient methods with respect to Nash equilibrium policies that are second-order stationary (SOS) in a sense similar to the type of sufficiency conditions used in optimization. Our first result is that SOS policies are locally attracting with high probability, and we show that policy gradient trajectories with gradient estimates provided by the Reinforce algorithm achieve an O(1/√n) distance-squared convergence rate if the method’s step-size is chosen appropriately. Subsequently, specializing to the class of deterministic Nash policies, we show that this rate can be improved dramatically and, in fact, policy gradient methods converge within a finite number of iterations in that *** Codes Primary 91A15, 91A26, secondary 68Q32, 68T05, 90C40 Copyright © 2022, The Authors. All rights reserved.

关键词： learning algorithms

Sequential algorithmic modification with test data reuse

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Feng, Jean Pennello, Gene Petrick, Nicholas Sahiner, Berkman Pirracchio, Romain Gossmann, Alexej Department Of Epidemiology And Biostatistics University Of California San Francisco United States U.S. Food And Drug Administration United States Department Of Anesthesiology University Of California San Francisco United States

After initial release of a machine learning algorithm, the model can be fine-tuned by retraining on subsequently gathered data, adding newly discovered features, or more. Each modification introduces a risk of deteriorating performance and must be validated on a test dataset. It may not always be practical to assemble a new dataset for testing each modification, especially when most modifications are minor or are implemented in rapid succession. Recent works have shown how one can repeatedly test modifications on the same dataset and protect against overfitting by (i) discretizing test results along a grid and (ii) applying a Bonferroni correction to adjust for the total number of modifications considered by an adaptive developer. However, the standard Bonferroni correction is overly conservative when most modifications are beneficial and/or highly correlated. This work investigates more powerful approaches using alpha-recycling and sequentially-rejective graphical procedures (SRGPs).We introduce novel extensions that account for correlation between adaptively chosen algorithmic modifications. In empirical analyses, the SRGPs control the error rate of approving unacceptable modifications and approve a substantially higher number of beneficial modifications than previous approaches. © 2022, CC BY.

关键词： learning algorithms

Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Yang, Yaoqing Theisen, Ryan Hodgkinson, Liam Gonzalez, Joseph E. Ramchandran, Kannan Martin, Charles H. Mahoney, Michael W. Dartmouth College United States University of California Berkeley United States University of Melbourne Australia Calculation Consulting United States International Computer Science Institute United States Lawrence Berkeley National Laboratory United States

Selecting suitable architecture parameters and training hyperparameters is essential for enhancing machine learning (ML) model performance. Several recent empirical studies conduct large-scale correlational analysis on neural networks (NNs) to search for effective generalization metrics that can guide this type of model selection. Effective metrics are typically expected to correlate strongly with test performance. In this paper, we expand on prior analyses by examining generalization-metric-based model selection with the following objectives: (i) focusing on natural language processing (NLP) tasks, as prior work primarily concentrates on computer vision (CV) tasks;(ii) considering metrics that directly predict test error instead of the generalization gap;(iii) exploring metrics that do not need access to data to compute. From these objectives, we are able to provide the first model selection results on large pretrained Transformers from Huggingface using generalization metrics. Our analyses consider (I) hundreds of Transformers trained in different settings, in which we systematically vary the amount of data, the model size and the optimization hyperparameters, (II) a total of 51 pretrained Transformers from eight families of Huggingface NLP models, including GPT2, BERT, etc., and (III) a total of 28 existing and novel generalization metrics. Despite their niche status, we find that metrics derived from the heavy-tail (HT) perspective are particularly useful in NLP tasks, exhibiting stronger correlations than other, more popular metrics. To further examine these metrics, we extend prior formulations relying on power law (PL) spectral distributions to exponential (EXP) and exponentially-truncated power law (E-TPL) families.1 Copyright © 2022, The Authors. All rights reserved.

关键词： learning algorithms

Discrimination and Class Imbalance Aware Online Naive Bayes

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Badar, Maryam Fisichella, Marco Iosifidis, Vasileios Nejdl, Wolfgang L3S Research Center Leibniz University of Hannover Welfengarten 1 Hannover Niedersachsen 30167 Germany

Fairness-aware mining of massive data streams is a growing and challenging concern in the contemporary domain of machine learning. Many stream learning algorithms are used to replace humans at critical decision-making points e.g., hiring staff, assessing credit risk, etc. This calls for handling massive incoming information with minimum response delay while ensuring fair and high quality decisions. Recent discrimination-aware learning methods are optimized based on overall accuracy. However, the overall accuracy is biased in favor of the majority class;therefore, state-of-the-art methods mainly diminish discrimination by partially or completely ignoring the minority class. In this context, we propose a novel adaptation of Naïve Bayes to mitigate discrimination embedded in the streams while maintaining high predictive performance for both the majority and minority classes. Our proposed algorithm is simple, fast, and attains multi-objective optimization goals. To handle class imbalance and concept drifts, a dynamic instance weighting module is proposed, which gives more importance to recent instances and less importance to obsolete instances based on their membership in minority or majority class. We conducted experiments on a range of streaming and static datasets and deduced that our proposed methodology outperforms existing state-of-the-art fairness-aware methods in terms of both discrimination score and balanced accuracy. © 2022, CC BY.

关键词： learning algorithms

An Unsupervised, Autonomous, and Objective Method for Validation or Selection of Training Samples

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Alameddine, Jihan Chehdi, Kacem Cariou, Claude University of Rennes 1 Lannion22305 France

In this paper, we present a practical new approach that can provide a solution to the problem of non-availability of training samples whether the number of classes is known or not. It also allows the validation and correction of the existing biased or simplified ground truths when they are available. This method is a necessary preliminary step of machine and deep learning algorithms. Indeed, the performance of these algorithms is strongly conditioned by the choice and the number of training samples used. Thus, their selection must be handled with objectivity and realism, regardless of their availability;that is, the samples exist but are insufficient or biased, or are difficult to access for security reasons or because of the large size of the data to be partitioned. The originality of the proposed method is to address this challenge, in a practical and efficient way, by providing users of supervised methods with reliable learning samples of all observed data classes. The proposed method is therefore executed by optimizing an objective criterion without user intervention. This process allows the end user to select classes according to their interests. To prove the relevance of the proposed method, we associate it as preliminary process to three supervised methods requiring training samples for the classification task. Three databases are used in this paper to illustrate the relevance of the proposed method: Iris data, a synthetic hyperspectral image, and a real hyperspectral image. The results show that the selection of relevant learning samples meets the real needs expressed by users and improves the performance of the three supervised algorithms. © 2022, The Authors. All rights reserved.

关键词： learning algorithms