Radial basis function (RBF) networks of Gaussian activation functions have been widely used in many applications due to its simplicity, robustness, good approximation and generalization ability, etc.. However, the tra...
详细信息
ISBN:
(纸本)9783540723820
Radial basis function (RBF) networks of Gaussian activation functions have been widely used in many applications due to its simplicity, robustness, good approximation and generalization ability, etc.. However, the training of such a RBF network is still a rather difficult task in the general case and the main crucial problem is how to select the number and locations of the hidden units appropriately. In this paper, we utilize a new kind of Bayesian Ying-Yang (BYY) automated model selection (AMS) learning algorithm to select the appropriate number and initial locations of the hidden units or Gaussians automatically for an input data set. It is demonstrated well by the experiments that this BYY-AMS training method is quite efficient and considerably outperforms the typical existing training methods on the training of RBF networks for both clustering analysis and nonlinear time series prediction.
Cerebellar Model Articulation Controller Neural Networks (CMAC NN) is one of the intelligent systems used for modeling, identification, classification, and controlling of nonlinear systems. In this paper, the mathemat...
详细信息
ISBN:
(纸本)9789077381595
Cerebellar Model Articulation Controller Neural Networks (CMAC NN) is one of the intelligent systems used for modeling, identification, classification, and controlling of nonlinear systems. In this paper, the mathematical model of CMAC is presented. CMAC is implemented using Simulink environment and its parameters are tuned to get the best CMAC control action. Three different learning algorithms are tested, using a constant learning rate, a variable learning rate, and learning by the control action of the conjugate conventional controller. The effect of varying CMAC parameters is studied and discussed. The simulation results showed that the learning algorithm based on constant learning rate gives the best performance.
In a finite game the Stochastically Stable States (SSSs) of adaptive play are contained in the set of minimizers of resistance trees. Also, in potential games, the SSSs of the log-linear learning algorithm are the min...
详细信息
ISBN:
(纸本)9783319675404;9783319675398
In a finite game the Stochastically Stable States (SSSs) of adaptive play are contained in the set of minimizers of resistance trees. Also, in potential games, the SSSs of the log-linear learning algorithm are the minimizers of the potential function. The SSSs can be characterized using the resistance trees of a Perturbed Markov Chain (PMC), they are the roots of minimum resistance tree. Therefore, computing the resistance of trees in PMC is important to analyze the SSSs of learning algorithms. A learning algorithm defines the Transition Probability Function (TPF) of the induced PMC on the action space of the game. Depending on the characteristics of the algorithm the TPF may become composite and intricate. Resistance computation of intricate functions is difficult and may even be infeasible. Moreover, there are no rules or tools available to simplify the resistance computations. In this paper, we propose novel rules that simplify the computation of resistance. We first, give a generalized definition of resistance that allows us to overcome the limitations of the existing definition. Then, using this new definition we develop the rules that reduce the resistance computation of composite TPF into resistance computation of simple functions. We illustrate their strength by efficiently computing the resistance in log-linear and payoff-based learning algorithms. They provide an efficient tool for characterizing SSSs of learning algorithms in finite games.
The need to evaluate treatment effectiveness is ubiquitous in most of empirical science, and interest in flexibly investigating effect heterogeneity is growing rapidly. To do so, a multitude of model-agnostic, nonpara...
详细信息
The need to evaluate treatment effectiveness is ubiquitous in most of empirical science, and interest in flexibly investigating effect heterogeneity is growing rapidly. To do so, a multitude of model-agnostic, nonparametric meta-learners have been proposed in recent years. Such learners decompose the treatment effect estimation problem into separate sub-problems, each solvable using standard supervised learning methods. Choosing between different meta-learners in a data-driven manner is difficult, as it requires access to counterfactual information. Therefore, with the ultimate goal of building better understanding of the conditions under which some learners can be expected to perform better than others a priori, we theoretically analyze four broad meta-learning strategies which rely on plug-in estimation and pseudo-outcome regression. We highlight how this theoretical reasoning can be used to guide principled algorithm design and translate our analyses into practice by considering a variety of neural network architectures as base-learners for the discussed metalearning strategies. In a simulation study, we showcase the relative strengths of the learners under different data-generating processes.
Some currently popular and successful deep learning architectures display certain pathological behaviors (e.g. confidently classifying random data as belonging to a familiar category of nonrandom images;and misclassif...
详细信息
ISBN:
(纸本)9783319213651;9783319213644
Some currently popular and successful deep learning architectures display certain pathological behaviors (e.g. confidently classifying random data as belonging to a familiar category of nonrandom images;and misclassifying miniscule perturbations of correctly classified images). It is hypothesized that these behaviors are tied with limitations in the internal representations learned by these architectures, and that these same limitations would inhibit integration of these architectures into heterogeneous multi-component AGI architectures. It is suggested that these issues can be worked around by developing deep learning architectures that internally form states homologous to image-grammar decompositions of observed entities and events.
Humans learn from visual inputs at multiple timescales, both rapidly and flexibly acquiring visual knowledge over short periods, and robustly accumulating online learning progress over longer periods. Modeling these p...
详细信息
ISBN:
(纸本)9781713871088
Humans learn from visual inputs at multiple timescales, both rapidly and flexibly acquiring visual knowledge over short periods, and robustly accumulating online learning progress over longer periods. Modeling these powerful learning capabilities is an important problem for computational visual cognitive science, and models that could replicate them would be of substantial utility in real-world computer vision settings. In this work, we establish benchmarks for both real-time and life-long continual visual learning. Our real-time learning benchmark measures a model's ability to match the rapid visual behavior changes of real humans over the course of minutes and hours, given a stream of visual inputs. Our life-long learning benchmark evaluates the performance of models in a purely online learning curriculum obtained directly from child visual experience over the course of years of development. We evaluate a spectrum of recent deep self-supervised visual learning algorithms on both benchmarks, finding that none of them perfectly match human performance, though some algorithms perform substantially better than others. Interestingly, algorithms embodying recent trends in self-supervised learning - including BYOL, SwAV and MAE - are substantially worse on our benchmarks than an earlier generation of self-supervised algorithms such as SimCLR and MoCo-v2. We present analysis indicating that the failure of these newer algorithms is primarily due to their inability to handle the kind of sparse low-diversity datastreams that naturally arise in the real world, and that actively leveraging memory through negative sampling - a mechanism eschewed by these newer algorithms - appears useful for facilitating learning in such low-diversity environments. We also illustrate a complementarity between the short and long timescales in the two benchmarks, showing how requiring a single learning algorithm to be locally context-sensitive enough to match real-time learning changes while stab
Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful o...
详细信息
Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address many issues that SSL algorithms would face in real-world applications. After creating a unified reimplementation of various widely-used SSL techniques, we test them in a suite of experiments designed to address these issues. We find that the performance of simple baselines which do not use unlabeled data is often under reported, SSL methods differ in sensitivity to the amount of labeled and unlabeled data, and performance can degrade substantially when the unlabeled dataset contains out-of-distribution examples. To help guide SSL research towards real-world applicability, we make our unified reimplemention and evaluation platform publicly available.
In this paper, we consider a two-dimensional (2-D) formation problem for multi-agent systems subject to switching topologies that dynamically change along both a finite time axis and an infinite iteration axis. We pre...
详细信息
ISBN:
(纸本)9781479901784
In this paper, we consider a two-dimensional (2-D) formation problem for multi-agent systems subject to switching topologies that dynamically change along both a finite time axis and an infinite iteration axis. We present a distributed iterative learning control (ILC) algorithm via the nearest neighbor rules. By employing the 2-D approach, we develop both the asymptotic and exponentially fast convergence of our formation ILC, which can be guaranteed by conditions in terms of the spectral radius and the matrix norms, respectively.
This paper extends and improves work of Fortnow and Klivans [5], who showed that if a circuit class C has an efficient learning algorithm in Angluin's model of exact learning via equivalence and membership queries...
详细信息
ISBN:
(纸本)9783642220067
This paper extends and improves work of Fortnow and Klivans [5], who showed that if a circuit class C has an efficient learning algorithm in Angluin's model of exact learning via equivalence and membership queries [2], then we have the lower bound EXPNP not subset of C. We use entirely different techniques involving betting games [4] to remove the NP oracle and improve the lower bound to EXP not subset of C. This shows that it is even more difficult to design a learning algorithm for C than the results of Fortnow and Klivans indicated.
This paper analyses the performance and efficiency of reinforcement learning algorithms for matching the availability of uncertain renewable energy sources (RES) with flexible loads. More specifically, this paper prop...
详细信息
ISBN:
(纸本)9781665467612
This paper analyses the performance and efficiency of reinforcement learning algorithms for matching the availability of uncertain renewable energy sources (RES) with flexible loads. More specifically, this paper proposes a novel scalable (with the number of customers) and efficient learning-based energy matching solution for maximizing social welfare in dynamic matching markets. The key features of the proposed solution is combining a simple rule-based function and a learnable component to achieve the aforementioned properties. The output of the learnable component is a probability distribution over the matching decisions for the individual customers. The proposed hybrid model enables the learning algorithm to find an effective matching policy that simultaneously satisfies the customers' servicing preferences. Extensive simulations are presented to show that the learning algorithm learns an effective matching policy for different generation-consumption profiles despite of the complexity reduction. The proposed solution exhibits significantly better performance compared to standard online matching heuristics such as Match on Arrival, Match to the Highest, and Match to the Earliest Deadline policies.
暂无评论