检索结果-内蒙古大学图书馆

learning with Subset Stacking

学校读者我要写书评

暂无评论

arXiv 2021年

作者： İlker Birbil, Ş. Yıldırım, Sinan Gökalp, Kaya Hakan Akyüz, M. Amsterdam Business School University of Amsterdam Amsterdam1018 TV Netherlands Faculty of Engineering and Natural Sciences Sabancı University Istanbul34956 Turkey Econometric Institute Erasmus University Rotterdam Rotterdam3000 DR Netherlands

We propose a new regression algorithm that learns from a set of input-output pairs. Our algorithm is designed for populations where the relation between the input variables and the output variable exhibits a heterogeneous behavior across the predictor space. The algorithm starts with generating subsets that are concentrated around random points in the input space. This is followed by training a local predictor for each subset. Those predictors are then combined in a novel way to yield an overall predictor. We call this algorithm "learning with Subset Stacking" or LESS, due to its resemblance to the method of stacking regressors. We compare the testing performance of LESS with the state-of-the-art methods on several datasets. Our comparison shows that LESS is a competitive supervised learning method. Moreover, we observe that LESS is also efficient in terms of computation time and it allows a straightforward parallel implementation. © 2021, CC BY.

关键词： learning algorithms

Financial news analysis for moroccan stock trend predictions

学校读者我要写书评

暂无评论

Test Engineering and Management 2020年第1-2期82卷 1712-1717页

作者： Hicham, El Bousty Salah-Ddine, Krit Polydisciplinary Faculty of Ouarzazate Ibn Zohr University Laboratory of Engineering Sciences and Energy Agadir Morocco

This article aims to predict Moroccan stock trends based on financial news articles. Data are collected from ***. All news collected for a single stock are lower to fit any machine learning algorithm, thus they are all combined for the training and the test issues. In these experiments we used 1061 articles published from 2015 to 2019. We compared performance of Support Vector Machine (SVM), Naïve Bayes, k Nearest Neighbors (KNN) and Decision Tree algorithms. Comparisons are performed first on news headlines and then on news corpus. Later we tried to enhance accuracy of the results using a specific dictionary-based approach. We attend 60% accuracy which is an acceptable rate in this context of the Moroccan market. We also inspected the reactivity of the Moroccan market to the publication of financial news by varying the forecasting *** shows that the Moroccan Market react better to news publication four days later. © 2020 Mattingley Publishing. All rights reserved.

关键词： learning algorithms

Characterizing and understanding the generalization error of transfer learning with gibbs algorithm

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Bu, Yuheng Aminian, Gholamali Toni, Laura Rodrigues, Miguel Wornell, Gregory Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology CambridgeMA02139 United States Electronic and Electrical Engineering Department University College London United Kingdom

We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, α-weighted-ERM and two-stage-ERM. Our key result is an exact characterization of the generalization behaviour using the conditional symmetrized KL information between the output hypothesis and the target training samples given the source samples. Our results can also be applied to provide novel distribution-free generalization error upper bounds on these two aforementioned Gibbs algorithms. Our approach is versatile, as it also characterizes the generalization errors and excess risks of these two Gibbs algorithms in the asymptotic regime, where they converge to the α-weighted-ERM and two-stage-ERM, respectively. Based on our theoretical results, we show that the benefits of transfer learning can be viewed as a bias-variance trade-off, with the bias induced by the source distribution and the variance induced by the lack of target samples. We believe this viewpoint can guide the choice of transfer learning algorithms in practice. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Tuned Compositional Feature Replays for Efficient Stream learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Talbot, Morgan B. Zawar, Rushikesh Badkundri, Rohil Zhang, Mengmi Kreiman, Gabriel CambridgeMA United States Boston Children’s Hospital BostonMA United States Harvard Medical School BostonMA United States CambridgeMA United States Harvard University CambridgeMA United States Birla Institute of Technology and Science at Pilani Pilani India Singapore Singapore School of Computer Science and Engineering Nanyang Technological University Singapore

Our brains extract durable, generalizable knowledge from transient experiences of the world. Artificial neural networks come nowhere close to this ability. When tasked with learning to classify objects by training on non-repeating video frames in temporal order (online stream learning), models that learn well from shuffled datasets catastrophically forget old knowledge upon learning new stimuli. We propose a new continual learning algorithm, Compositional Replay Using Memory Blocks (CRUMB), which mitigates forgetting by replaying feature maps reconstructed by combining generic parts. CRUMB concatenates trainable and re-usable "memory block" vectors to compositionally reconstruct feature map tensors in convolutional neural networks. Storing the indices of memory blocks used to reconstruct new stimuli enables memories of the stimuli to be replayed during later tasks. This reconstruction mechanism also primes the neural network to minimize catastrophic forgetting by biasing it towards attending to information about object shapes more than information about image textures, and stabilizes the network during stream learning by providing a shared feature-level basis for all training examples. These properties allow CRUMB to outperform an otherwise identical algorithm that stores and replays raw images, while occupying only 3.6% as much memory. We stress-tested CRUMB alongside 13 competing methods on 7 challenging datasets. To address the limited number of existing online stream learning datasets, we introduce 2 new benchmarks by adapting existing datasets for stream learning. With only 3.7-4.1% as much memory and 15-43% as much runtime, CRUMB mitigates catastrophic forgetting more effectively than the state-of-the-art. Our code is available at https://***/MorganBDT/***. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

A bit better? Quantifying information for bandit learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Devraj, Adithya M. van Roy, Benjamin Xu, Kuang Stanford University StanfordCA94305 United States

The information ratio offers an approach to assessing the efficacy with which an agent balances between exploration and exploitation. Originally, this was defined to be the ratio between squared expected regret and the mutual information between the environment and action-observation pair, which represents a measure of information gain. Recent work has inspired consideration of alternative information measures, particularly for use in analysis of bandit learning algorithms to arrive at tighter regret bounds. We investigate whether quantification of information via such alternatives can improve the realized performance of information-directed sampling, which aims to minimize the information ratio. © 2021, CC BY.

关键词： learning algorithms

Ensemble and mixed learning techniques for credit card fraud detection

学校读者我要写书评

暂无评论

arXiv 2021年

作者： de Souza, Daniel H.M. Bordin, Claudio J. Universidade Federal do ABC SP Santo André09210-580 Brazil

Spurious credit card transactions are a significant source of financial losses and urge the development of accurate fraud detection algorithms. In this paper, we use machine learning strategies for such an aim. First, we apply a mixed learning technique that uses K-means preprocessing before trained classification to the problem at hand. Next, we introduce an adapted detector ensemble technique that uses OR-logic algorithm aggregation to enhance the detection rate. Then, both strategies are deployed in tandem in numerical simulations using real-world transactions data. We observed from simulation results that the proposed methods diminished computational cost and enhanced performance concerning state-of-the-art techniques. © 2021, CC BY-NC-ND.

关键词： learning algorithms

Realizable learning is All You Need

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hopkins, Max Kane, Daniel M. Lovett, Shachar Mahajan, Gaurav Department of Computer Science and Engineering UCSD California CA92092 United States Department of Mathematics UCSD California CA92092 United States Department of Computer Science Yale University Connecticut CT06511 United States

The equivalence of realizable and agnostic learnability is a fundamental phenomenon in learning theory. With variants ranging from classical settings like PAC learning and regression to recent trends such as adversarially robust learning, it's surprising that we still lack a unified theory;traditional proofs of the equivalence tend to be disparate, and rely on strong model-specific assumptions like uniform convergence and sample compression. In this work, we give the first model-independent framework explaining the equivalence of realizable and agnostic learnability: a three-line blackbox reduction that simplifies, unifies, and extends our understanding across a wide variety of settings. This includes models with no known characterization of learnability such as learning with arbitrary distributional assumptions and more general loss functions, as well as a host of other popular settings such as robust learning, partial learning, fair learning, and the statistical query model. More generally, we argue that the equivalence of realizable and agnostic learning is actually a special case of a broader phenomenon we call property generalization: any desirable property of a learning algorithm (e.g. noise tolerance, privacy, stability) that can be satisfied over finite hypothesis classes extends (possibly in some variation) to any learnable hypothesis *** Codes 68Q32 © 2021, CC BY.

关键词： learning algorithms

Incremental Class learning using Variational Autoencoders with Similarity learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Huo, Jiahao van Zyl, Terence L. Institute for Intelligent Systems University of Johannesburg South Africa

Catastrophic forgetting in neural networks during incremental learning remains a challenging problem. Previous research investigated catastrophic forgetting in fully connected networks, with some earlier work exploring activation functions and learning algorithms. Applications of neural networks have been extended to include similarity learning. Understanding how similarity learning loss functions would be affected by catastrophic forgetting is of significant interest. Our research investigates catastrophic forgetting for four well-known similarity-based loss functions during incremental class learning. The loss functions are Angular, Contrastive, Center, and Triplet loss. Our results show that the catastrophic forgetting rate differs across loss functions on multiple datasets. The Angular loss was least affected, followed by Contrastive, Triplet loss, and Center loss with good mining techniques. We implemented three existing incremental learning techniques, iCaRL, EWC, and EBLL. We further proposed a novel technique using Variational Autoencoders (VAEs) to generate representation as exemplars passed through the network’s intermediate layers. Our method outperformed three existing state-of-the-art techniques. We show that one does not require stored images (exemplars) for incremental learning with similarity learning. The generated representations from VAEs help preserve regions of the embedding space used by prior knowledge so that new knowledge does not "overwrite" it. Copyright © 2021, The Authors. All rights reserved.

关键词： learning algorithms

Open problem: Is there an online learning algorithm that learns whenever online learning is possible?

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Hanneke, Steve Toyota Technological Institute at Chicago United States