predictive coding Network (PCN) is an important neural network inspired by visual processing models in neuroscience. It combines the feedforward and feedback processing and has the architecture of recurrent neural net...
详细信息
predictive coding Network (PCN) is an important neural network inspired by visual processing models in neuroscience. It combines the feedforward and feedback processing and has the architecture of recurrent neural networks (RNNs). This type of network is usually trained with backpropagation through time (BPTT). With infinite recurrent steps, PCN is a dynamic system. However, as one of the most important properties, stability is rarely studied in this type of network. Inspired by reservoir computing, we investigate the stability of hierarchical RNNs from the perspective of dynamic systems, and propose a sufficient condition for their echo state property (ESP). Our study shows the global stability is determined by stability of the local layers and the feedback between neighboring layers. Based on it, we further propose Weight Norm Supervision, a new algorithm that controls the stability of PCN dynamics by imposing different weight norm constraints on different parts of the network. We compare our approach with other training methods in terms of stability and prediction capability. The experiments show that our algorithm learns stable PCNs with a reliable prediction precision in the most effective and controllable way.
Deep-predictive-coding networks (DPCNs) are hierarchical, generative models. They rely on feed-forward and feedback connections to modulate latent feature representations of stimuli in a dynamic and context-sensitive ...
详细信息
Deep-predictive-coding networks (DPCNs) are hierarchical, generative models. They rely on feed-forward and feedback connections to modulate latent feature representations of stimuli in a dynamic and context-sensitive manner. A crucial element of DPCNs is a forward-backward inference procedure to uncover sparse, invariant features. However, this inference is a major computational bottleneck. It severely limits the network depth due to learning stagnation. Here, we prove why this bottleneck occurs. We then propose a new forward-inference strategy based on accelerated proximal gradients. This strategy has faster theoretical convergence guarantees than the one used for DPCNs. It overcomes learning stagnation. We also demonstrate that it permits constructing deep and wide predictive-coding networks. Such convolutional networks implement receptive fields that capture well the entire classes of objects on which the networks are trained. This improves the feature representations compared with our lab's previous nonconvolutional and convolutional DPCNs. It yields unsupervised object recognition that surpass convolutional autoencoders and is on par with convolutional networks trained in a supervised manner.
This paper investigates distributed predictivecoding of correlated sources with memory, which are communicated to a central receiver. This is the setting typically encountered in sensor networks. While source memor...
详细信息
ISBN:
(纸本)9781424413973;1424413974
This paper investigates distributed predictivecoding of correlated sources with memory, which are communicated to a central receiver. This is the setting typically encountered in sensor networks. While source memory may be exploited by distributed coding of large source blocks (vectors), the growth in complexity (and delay) is often unacceptable in practice, hence the interest in a low complexity predictive approach. We first consider the inherent "conflict" between distributed and predictive coding due to the impact of distributed quantization on the prediction loop. This is coupled with the effects of closed loop prediction, which destabilize standard Lloyd-like code design methods. An iterative algorithm is derived, which optimizes the overall system while imposing zero decoder drift due to distributed quantization. The approach circumvents convergence and stability issues of traditional predictive quantizer design by employing an "asymptotic closed loop" framework which is adapted for distributed predictive system design. The scheme efficiently utilizes both the temporal and inter-source correlations and subsumes as extreme special cases both separate source predictive coding, and distributed coding of memoryless correlated sources.
An adaptive predictive coding with dynamic quantization adjustment(APC-DQA) is proposed for speech coding at 16 kbits/s, which aims to reduce processing delay and hardware complexity, while attaining "toll qualit...
详细信息
An adaptive predictive coding with dynamic quantization adjustment(APC-DQA) is proposed for speech coding at 16 kbits/s, which aims to reduce processing delay and hardware complexity, while attaining "toll quality". The proposed scheme utilizes time-domain processing to lessen the processing delay. Moreover, it employs backward processing in both prediction and quantization, which requires less side information than forward processing. It also incorporates adaptive bit allocation in sub-intervals of each frame so as to remove redundancies due to periodic concentration of the prediction residual energy. The performance evaluation results show that the processing delay of the APC-DQA is 2/3 that of an adaptive predictive coding with adaptive bit allocation scheme (APC-AB) [1]. Moreover, its hardware complexity is approximately 70% to 80% of the APC-AB. It was also shown that this scheme can provide speech quality subjectively equivalent to 6.6 bit Log-PCM.
Adaptive predictive coding of speech signals at bit rates lower than 10 kbits/sec often requires the use of 2-level (1 bit) quantization of the samples of the prediction residual. Such a coarse quantization of the pre...
详细信息
Adaptive predictive coding of speech signals at bit rates lower than 10 kbits/sec often requires the use of 2-level (1 bit) quantization of the samples of the prediction residual. Such a coarse quantization of the prediction residual can produce audible quantizing noise in the reproduced speech signal at the receiver. This paper describes a new method of quantization for improving the speech quality. The improvement is obtained by center clipping the prediction residual and by fine quantization of the high-amplitude portions of the prediction residual. The threshold of center clipping is adjusted to provide encoding of the prediction residual at a specified bit rate. This method of quantization not only improves the speech quality by accurate quantization of the prediction residual when its amplitude is large but also allows encoding of the prediction residual at bit rates below 1 bit/sample.
A new algorithm of speech coding "recursive and adaptive prediction" is proposed and tested. An adaptive linear prediction of the input is carried out sample by sample, and only predictive residuals are quan...
详细信息
A new algorithm of speech coding "recursive and adaptive prediction" is proposed and tested. An adaptive linear prediction of the input is carried out sample by sample, and only predictive residuals are quantized and transmitted in binary codes. predictive coefficients are adaptively controlled by quantized prediction error. Segmental SNR of almost 22 dB is obtained at 16 kb/s by the cascade connection of 2 stages of prediction. The algorithm can handle mixed voices and be implemented by single DSP.
This paper reports experiments in image compression using adaptive sampling and predictive coding. A picture usually comprises of areas of varying detail. The authors first detect these areas, and then sample key poin...
详细信息
This paper reports experiments in image compression using adaptive sampling and predictive coding. A picture usually comprises of areas of varying detail. The authors first detect these areas, and then sample key points from them. An area with greater detail requires many sample points to suitably represent it, whereas regions with lesser detail can be represented with fewer points. These key sample points are then used to predict the remainder of the image. Combining the use of adaptive sampling with predictive encoding, the authors' algorithm manages to achieve compression while maintaining the desired image quality. Its performance is compared to JPEG in terms of compression ratio and image quality.
Cybercrime is growing rampantly around the world, which has caused huge monetary damages in recent years. One of the major difficulties in cybercrime forensic analysis is to identify relevant digital evidence from a l...
详细信息
ISBN:
(数字)9781728198743
ISBN:
(纸本)9781728198750
Cybercrime is growing rampantly around the world, which has caused huge monetary damages in recent years. One of the major difficulties in cybercrime forensic analysis is to identify relevant digital evidence from a large amount of electronic documents. Traditional methods, such as manual inspection and keyword searching, are no longer effective both in terms of time and accuracy. In order to reduce the cost, save time and improve the accuracy of forensic investigation, the paper proposed a predictive coding scheme to study and identify relevant digital evidence. The experimental results show that the predictive coding based on semantic searching is feasible, and more efficient and accurate than the keyword searching.
Relationship between consecutive frame in the generalized harmonics analysis and possibility of predictive coding is described. In the GHA, each frame of a signal is shown as sum of sinusoids. The parameters which sho...
详细信息
ISBN:
(纸本)9781424438273
Relationship between consecutive frame in the generalized harmonics analysis and possibility of predictive coding is described. In the GHA, each frame of a signal is shown as sum of sinusoids. The parameters which show the same sinusoid are often extracted from two consecutive frames. We propose a method to extract the parameters which show the same sinusoid using cross correlation function and greedily selection algorithm. This algorithm is applied to several audio signals and it is confirmed that sinusoids are extracted across multiple consecutive frames. Possibility to apply the proposed method to predictive coding is also shown.
暂无评论