We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined her...
详细信息
We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in ''blind'' signal processing.
Many proposals have been presented for the acquisition of inverse models in multilayered neural networks. However, most are concerned with the backpropagation rule or its improvement. In learning in a multilayered neu...
详细信息
Many proposals have been presented for the acquisition of inverse models in multilayered neural networks. However, most are concerned with the backpropagation rule or its improvement. In learning in a multilayered neural network based on the backpropagation rule, there must be a supervisor signal for the output layer, and there must be a particular path to propagate the learning signal in the reverse direction. In addition, convergence is slow due to the use of the method of steepest descent in updating the weights. Consequently, this paper proposes a forward propagation rule in which the neural network model is trained by propagating the motion error exhibited by the control object in the forward direction in the neural network. In the proposed algorithm, the extended Newton's method is used to derive the goal signal (which corresponds to the supervisor signal) in the hidden layer and the output layer. Since linear multiple regression can be used in weight updating for realizing the goal signals, the iteration of weight updating can be reduced compared to the method of steepest descent. A computer simulation was performed for acquisition of a two-link arm model, and the effectiveness of the proposed learning scheme was verified. (C) 2005 Wiley Periodicals, Inc.
On the web, we can find dictionaries for viewing a sign of French Sign Language (FSL), from a word. However, finding a word from a sign is much more complicated. For this purpose, we propose to design a web applicatio...
详细信息
ISBN:
(纸本)9783319085999;9783319085982
On the web, we can find dictionaries for viewing a sign of French Sign Language (FSL), from a word. However, finding a word from a sign is much more complicated. For this purpose, we propose to design a web application to find the meaning of a FSL sign in the French language from the sign's features. In order to do this, we have developed an intelligent system capable of learning and self-improving by feeding off the information presented to it during its use. We have managed to find a middle ground between the reliability of the results and the ergonomics of Human-Machine Interfaces (HMI).
Image compression is an important area of multimedia investigation and neural network methods have attracted more and more attentions for using in image coding. Recently a random neural network model, which has the so...
详细信息
ISBN:
(纸本)0819438758
Image compression is an important area of multimedia investigation and neural network methods have attracted more and more attentions for using in image coding. Recently a random neural network model, which has the solutions with product form in steady state (i.e. the steady state probability distribution of network can always be expressed as the product of the probabilities of the states of each neuron) on some conditions, was brought forward. Among the diverse random neural network models, the feed-forward one is very practicable because its solutions exist and are unique. In this paper, a new learning method for feed-forward random neural network, which can be implemented easier than the learning algorithm of the RNN presented by Gelenbe, was presented. Using the new learning formulas we developed, we designed a new image coding method, which applies the random neural network method in classical DCT-based coding framework. The experimental results show that our new method could gain a lot in PSNR (1 similar to 2dB) compared with standard neural network coding methods. In conclusion, we stated that the DCT-based image compression method using random neural network is an efficient algorithm for image coding.
Clustering algorithms depend strongly on the dissimilarity considered to evaluate the sample proximities. In real applications, several dissimilarities are available that may come from different object representations...
详细信息
ISBN:
(纸本)9781424496365
Clustering algorithms depend strongly on the dissimilarity considered to evaluate the sample proximities. In real applications, several dissimilarities are available that may come from different object representations or data sources. Each dissimilarity provides usually complementary information about the problem. Therefore, they should be integrated in order to reflect accurately the object proximities. In many applications, the user feedback or the a priory knowledge about the problem provide pairs of similar and dissimilar examples. In this paper, we address the problem of learning a linear combination of dissimilarities using side information in the form of equivalence constraints. The minimization of the error function is based on a quadratic optimization algorithm. A smoothing term is included that penalizes the complexity of the family of distances and avoids overfitting. The experimental results suggest that the method proposed outperforms a standard metric learning algorithm and improves classification and clustering results based on a single dissimilarity and data source.
The brain stores various kinds of temporal sequences as long-term memories, such as motor sequences, episodes, and melodies. The present study aims at clarifying the general principle underlying such memories. For thi...
详细信息
The brain stores various kinds of temporal sequences as long-term memories, such as motor sequences, episodes, and melodies. The present study aims at clarifying the general principle underlying such memories. For this purpose, the memory mechanism of sequential patterns is examined from the viewpoint of computational theory and neural network modeling, and a neural network model of sequential pattern memory based on a simple and reasonable principle is presented. Specifically, spatio-temporal patterns varying gradually with time are stably stored in a network consisting of pairs of excitatory and inhibitory cells with recurrent connections;such a pair can achieve non-monotonic input-output characteristics which are essential for smooth sequential recall. Storage is performed using a simple learning algorithm which is based on the covariance rule and requires only that the sequence be input several times and retrieval is highly tolerant to noise. It is thought that a similar principle is used in cerebral memory systems, and the relevance of this model to the brain is discussed. Also, possible roles of hippocampus and basal ganglia in memorizing sequences are suggested.
Fuzzy Cognitive Maps (FCM) may be defined as Recurrent Neural Networks that allow causal reasoning. According to the transformation function used for updating the activation value of concepts they can be characterized...
详细信息
Fuzzy Cognitive Maps (FCM) may be defined as Recurrent Neural Networks that allow causal reasoning. According to the transformation function used for updating the activation value of concepts they can be characterized as discrete or continuous. It is remarkable that FCM having discrete neurons never exhibit chaotic states, but this premise cannot be guaranteed for FCM having continuous concepts. On the other hand, complex Sigmoid FCM resulting from experts or learning algorithms often show chaotic or cyclic patterns, therefore leading to confusing interpretation of the investigated system. The first contribution of this paper is focused on explaining why most studies on FCM stability are not applicable to FCM used on classification or decision-making tasks. Next we describe a non-direct learning methodology based on Swarm Intelligence for improving the system stability once the causal weight estimation is done. The objective here is to find a specific threshold function for each map neuron simulating an external stimulus, instead of using the same transformation function for all concepts. At the end, we can compute more stable maps, so better consistency in hidden patterns is achieved.
Since learning models from data allows direct and accurate model approximation, it has lately become a tool of increasing interest in robotics. However, most learning methods require significant amounts of samples in ...
详细信息
Since learning models from data allows direct and accurate model approximation, it has lately become a tool of increasing interest in robotics. However, most learning methods require significant amounts of samples in the anticipated region to achieve adequate performance. In this paper, a novel modeling framework via sparsity and feature learning (SFL) is proposed for approximating multi-DOF manipulator inverse dynamics. A large number of features and their weights are recognized and selected from the analytical form of the dynamical equation. A matrix-free procedure based on the Alternating Direction Multiplier Method (ADMM) and reweighted L1-minimization technique are presented for time and space efficiency. Finally, we conducted the torque prediction and control experiments on a 7-DOF industrial robot to show the effectiveness of the proposed learning framework and compared the performance with different model classes and procedures. Superior to existing modeling methods, the merits of our method are that it needs no prior knowledge of the mechanical parameters of the robot thus avoiding modeling and assembly errors introduced by the geometric parameter identification procedure. Moreover, as the restriction of the features with specific forms is viewed as a strong priority of learning, the SFL method outperforms the universal approximation method on real data while generalizing for finite training samples.
This study concerns with the diagnosis of aerospace structure defects by applying a HPC parallel implementation of a novel learning algorithm, named U-BRAIN. The Soft Computing approach allows advanced multi-parameter...
详细信息
ISBN:
(纸本)9781479920693
This study concerns with the diagnosis of aerospace structure defects by applying a HPC parallel implementation of a novel learning algorithm, named U-BRAIN. The Soft Computing approach allows advanced multi-parameter data processing in composite materials testing. The HPC parallel implementation overcomes the limits due to the great amount of data and the complexity of data processing. Our experimental results illustrate the effectiveness of the U-BRAIN parallel implementation as defect classifier in aerospace structures. The resulting system is implemented on a Linux-based cluster with multi-core architecture.
Complex "lifelike" behaviors are composed of local interactions of individuals under fundamental rules of artificial life. In this paper, fundamental rules for cooperative group behaviors, "flocking&quo...
详细信息
ISBN:
(纸本)0819460745
Complex "lifelike" behaviors are composed of local interactions of individuals under fundamental rules of artificial life. In this paper, fundamental rules for cooperative group behaviors, "flocking" and "arrangement" of multiple autonomouse mobile robots were represented by a small number of fuzzy rules by Subtractive clustering algorithm and DNA coding method. Fuzzy rules in Sugeno type and their related parameters were automatically generated from clustering input-output data obtained from the algorithms for the group behaviors. Simulations demonstrate the fuzzy rules successfully realize group intelligence of mobile robots.
暂无评论