Can we reduce the search cost of neural Architecture Search (NAS) from days down to only a few hours? NAS methods automate the design of Convolutional Networks (ConvNets) under hardware constraints and they have emerg...
详细信息
Can we reduce the search cost of neural Architecture Search (NAS) from days down to only a few hours? NAS methods automate the design of Convolutional Networks (ConvNets) under hardware constraints and they have emerged as key components of AutoML frameworks. However, the NAS problem remains challenging due to the combinatorially large design space and the significant search time (at least 200 GPU-hours). In this article, we alleviate the NAS search cost down to less than 3 hours, while achieving state-of-the-art image classification results under mobile latency constraints. We propose a novel differentiable NAS formulation, namely Single-Path NAS, that uses one single-path over-parameterized ConvNet to encode all architectural decisions based on shared convolutional kernel parameters, hence drastically decreasing the search overhead. Single-Path NAS achieves state-of-the-art top-1 imageNet accuracy (75.62%), hence outperforming existing mobile NAS methods in similar latency settings (similar to 80 ms). In particular, we enhance the accuracy-runtime tradeoff in differentiable NAS by treating the Squeeze-and-Excitation path as a fully searchable operation with our novel single-path encoding. Our method has an overall cost of only 8 epochs (24 TPU-hours), which is up to 5,000x faster compared to prior work. Moreover, we study how different NAS formulation choices affect the performance of the designed ConvNets. Furthermore, we exploit the efficiency of our method to answer an interesting question: instead of empirically tuning the hyperparameters of the NAS solver (as in prior work), can we automatically find the hyperparameter values that yield the desired accuracy-runtime trade-off (e.g., target runtime for different platforms)? We view our extensive experimental results as a valuable exploration for NAS-based cloud AutoML services, and we open-source our entire codebase at: https://***/dstamoulis/single-path-nas.
The rapid advancement of artificial intelligence and widespread use of smartphones have resulted in an exponential growth of image data, both real (camera-captured) and virtual (AI-generated). This surge underscores t...
详细信息
There have been several successful deep learning models that perform audio super-resolution. Many of these approaches involve using preprocessed feature extraction which requires a lot of domain-specific signal proces...
详细信息
Despite the tremendous success of deep neural networks in various learning problems, it has been observed that adding intentionally designed adversarial perturbations to inputs of these architectures leads to erroneou...
详细信息
Despite the tremendous success of deep neural networks in various learning problems, it has been observed that adding intentionally designed adversarial perturbations to inputs of these architectures leads to erroneous classification with high confidence in the prediction. In this work, we show that adversarial examples can be generated using a generic approach that relies on the perturbation analysis of learning algorithms. Formulated as a convex program, the proposed approach retrieves many current adversarial attacks as special cases. It is used to propose novel attacks against learning algorithms for classification and regression tasks under various new constraints with closed-form solutions in many instances. In particular, we derive new attacks against classification algorithms which are shown to be top-performing on various architectures. Although classification tasks have been the main focus of adversarial attacks, we use the proposed approach to generate adversarial perturbations for various regression tasks. Designed for single pixel and single subset attacks, these attacks are applied to autoencoding, image colorization and real-time object detection tasks, showing that adversarial perturbations can degrade equally gravely the output of regression tasks. (1) (1) In the spirit of encouraging reproducible research, the implementations used in this paper have been made available at: ***/ebalda/adversarialconvex.
The Convolutional neural Networks (CNNs) are able to learn basic and high level features hierarchically with the highlight that it implements an end-to-end learning method. However, lacking in the ability to utilize p...
详细信息
ISBN:
(纸本)9781450362047
The Convolutional neural Networks (CNNs) are able to learn basic and high level features hierarchically with the highlight that it implements an end-to-end learning method. However, lacking in the ability to utilize prior information and domain knowledge has led to the neural networks hard to train. In this paper, a method using prior information is proposed, which is by appending prior feature-maps through a bypass input structure. As an implementation, we evaluate a convolutional neural network integrating with the Self-Quotient image (SQI) algorithm. Through the bypass, we import the feature-maps from the SQI algorithm and concat them with the output of the first convolution layer. With the help of traditional imageprocessingmethods, CNNs can directly improve the accuracy and training stability, while the bypass is exactly a consistent point. Finally, the necessity of this bypass pattern is that it avoids the direct modification of original images. As CNNs are able to focus on far richer features than basic imageprocessingmethods, it is advisable for us to expose CNNs to the original data. It is exactly the main design idea that we make the output from synergistic processing algorithm bypass from the side.
This tutorial covers biomedical image reconstruction, from the foundational concepts of system modeling and direct reconstruction to modern sparsity and learning-based approaches. Imaging is a critical tool in biologi...
详细信息
This tutorial covers biomedical image reconstruction, from the foundational concepts of system modeling and direct reconstruction to modern sparsity and learning-based approaches. Imaging is a critical tool in biological research and medicine, and most imaging systems necessarily use an image reconstruction algorithm to create an image;the design of these algorithms has been a topic of research since at least the 1960's. In the last few years, machine learning-based approaches have shown impressive performance on image reconstruction problems, triggering a wave of enthusiasm and creativity around the paradigm of learning. Our goal is to unify this body of research, identifying common principles and reusable building blocks across decades and among diverse imaging modalities. We first describe system modeling, emphasizing how a few building blocks can be used to describe a broad range of imaging modalities. We then discuss reconstruction algorithms, grouping them into three broad generations. The first are the classical direct methods, including Tikhonov regularization;the second are the variational methods based on sparsity and the theory of compressive sensing;and the third are the learning-based (also called data-driven) methods, especially those using deep convolutional neural networks. There are strong links between these generations: classical (first-generation) methods appear as modules inside the latter two, and the former two are used to inspire new designs for learning-based (third-generation) methods. As a result, a solid understanding of all three generations is necessary for the design of state-of-the-art algorithms.
We analyze the performance of feedforward vs. recurrent neural network (RNN) architectures and associated training methods for learned frame prediction. To this effect, we trained a residual fully convolutional neural...
详细信息
ISBN:
(纸本)9781538662496
We analyze the performance of feedforward vs. recurrent neural network (RNN) architectures and associated training methods for learned frame prediction. To this effect, we trained a residual fully convolutional neural network (FCNN), a convolutional RNN (CRNN), and a convolutional long short-term memory (CLSTM) network for next frame prediction using the mean square loss. We performed both stateless and stateful training for recurrent networks. Experimental results show that the residual FCNN architecture performs the best in terms of peak signal to noise ratio (PSNR) at the expense of higher training and test (inference) computational complexity. The CRNN can be trained stably and very efficiently using the stateful truncated backpropagation through time procedure, and it requires an order of magnitude less inference runtime to achieve near real-time frame prediction with an acceptable performance.
暂无评论