Attributed to the ever-increasing large image datasets, Convolutional neuralnetworks (CNNs) have become popular for vision-based tasks. It is generally admirable to have larger-sized datasets for higher network train...
详细信息
Attributed to the ever-increasing large image datasets, Convolutional neuralnetworks (CNNs) have become popular for vision-based tasks. It is generally admirable to have larger-sized datasets for higher network training accuracies. However, the impact of dataset quality has not to be involved. It is reasonable to assume the near-duplicate images exist in the datasets. For instance, the Street View House Numbers (SVHN) dataset having cropped house plate digits from 0 to 9 are likely to have repetitive digits from the same/similar house plates. Redundant images may take up a certain portion of the dataset without consciousness. While contributing little to no accuracy improvement for the CNNs training, these duplicated images unnecessarily pose extra resource and computation consumption. To this end, this paper proposes a framework to assess the impact of the near-duplicate images on CNN training performance, called CE-Dedup. Specifically, CE-Dedup associates a hashing-based image deduplication approach with downstream CNNs-based image classification tasks. CE-Dedup balances the tradeoff between a large deduplication ratio and a stable accuracy by adjusting the deduplication threshold. The effectiveness of CE-Dedup is validated through extensive experiments on well-known CNN benchmarks. On one hand, while maintaining the same validation accuracy, CE-Dedup can reduce the dataset size by 23%. On the other hand, when allowing a small validation accuracy drop (by 5%), CE-Dedup can trim the dataset size by 75%.
The proceedings contain 9 papers. The special focus in this conference is on Multiple-Aspect Analysis of Semantic Trajectories. The topics include: Uncovering Hidden Concepts from AIS Data: A network Abstraction of Ma...
ISBN:
(纸本)9783030380809
The proceedings contain 9 papers. The special focus in this conference is on Multiple-Aspect Analysis of Semantic Trajectories. The topics include: Uncovering Hidden Concepts from AIS Data: A network Abstraction of Maritime Traffic for Anomaly Detection;learning from Our Movements – The Mobility Data Analytics Era;preface;multi-channel Convolutional neuralnetworks for Handling Multi-dimensional Semantic Trajectories and Predicting Future Semantic Locations;A Neighborhood-Augmented LSTM Model for Taxi-Passenger Demand Prediction;predicting Fishing Effort and Catch Using Semantic Trajectories and Machine Learning;prospective Data Model and distributed Query processing for Mobile Sensing Data Streams.
The proceedings contain 261 papers. The topics discussed include: MBSE-based modeling technology for aircraft assembly tooling design demand;modeling and implementation of distributed rain water storage and utilizatio...
The proceedings contain 261 papers. The topics discussed include: MBSE-based modeling technology for aircraft assembly tooling design demand;modeling and implementation of distributed rain water storage and utilization;modeling free surface elevation around tandem piers of the longitudinal bridge by computational fluid dynamics;L1-finite difference method for inverse source problem of fractional diffusion equation;EMD and singular value difference spectrum based bearing fault characteristics extraction;research on a distribution-outlier detection algorithm based on logistics distribution data;activation detection algorithms for pattern division multiple access uplink grant-free transmission;application of enhanced whale adaptive threshold noise reduction method in transformer ultrasonic 3D imaging detection;human body shape reconstruction from binary image using convolutional neuralnetwork;a pricing mechanism which implements allocation in Shannon formula of home cellular wireless communication;and research on power consumption information acquisition system based on broadband power line carrier technology.
In this paper, we are able to focus on the fiber-optic distributed acoustic systems required to isolate the oil and gas pipeline, remote areas, facilities and lines in the area. Threatening performances of network str...
详细信息
ISBN:
(纸本)9781728119045
In this paper, we are able to focus on the fiber-optic distributed acoustic systems required to isolate the oil and gas pipeline, remote areas, facilities and lines in the area. Threatening performances of network structures of different complexity and depth. All measurements, the criteria for working independently from the field, the results obtained and the results obtained and on-site troubleshooting methods in different geographical areas. It analyzes the performance results and shows a high performance network in the field in real time.
Currently, artificial intelligence technology is attracting much attention, and image processing field is also making remarkable progress in recognition rate through CNN models. Furthermore, Capsule network which is f...
详细信息
ISBN:
(纸本)9781728129464
Currently, artificial intelligence technology is attracting much attention, and image processing field is also making remarkable progress in recognition rate through CNN models. Furthermore, Capsule network which is flexible in changing pose of image is being studied in various fields by improving disadvantage of Pooling Layer of CNN model. We propose a method to accelerate the learning of CapsNet model, which is much slower than the existing neuralnetwork model. TensorFlow, Google's deep-running library, and the Apache Foundation's Hadoop framework are run through Python. We can confirm that the learning time is decreased in inverse proportion to the increase of the number of nodes in learning CapsNet according to the proposed method of distributedprocessing.
Despite the soaring use of convolutional neuralnetworks (CNNs) in mobile applications, uniformly sustaining high-performance inference on mobile has been elusive due to the excessive computational demands of modern C...
详细信息
Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication e...
详细信息
Communication overhead is a major bottleneck hampering the scalability of distributed machine learning systems. Recently, there has been a surge of interest in using gradient compression to improve the communication efficiency of distributedneuralnetwork training. Using 1-bit quantization, signSGD with majority vote achieves a 32x reduction on communication cost. However, its convergence is based on unrealistic assumptions and can diverge in practice. In this paper, we propose a general distributed compressed SGD with Nesterov's momentum. We consider two-way compression, which compresses the gradients both to and from workers. Convergence analysis on nonconvex problems for general gradient compressors is provided. By partitioning the gradient into blocks, a blockwise compressor is introduced such that each gradient block is compressed and transmitted in 1-bit format with a scaling factor, leading to a nearly 32x reduction on communication. Experimental results show that the proposed method converges as fast as full-precision distributed momentum SGD and achieves the same testing accuracy. In particular, on distributed ResNet training with 7 workers on the ImageNet, the proposed algorithm achieves the same testing accuracy as momentum SGD using full-precision gradients, but with 46% less wall clock time.
distributed learning paradigms such as federated learning often involve transmission of model updates, or gradients, over a network, thereby avoiding transmission of private data. However, it is possible for sensitive...
ISBN:
(纸本)9781713845393
distributed learning paradigms such as federated learning often involve transmission of model updates, or gradients, over a network, thereby avoiding transmission of private data. However, it is possible for sensitive information about the training data to be revealed from such gradients. Prior works have demonstrated that labels can be revealed analytically from the last layer of certain models (e.g., ResNet), or they can be reconstructed jointly with model inputs by using Gradients Matching [1] with additional knowledge about the current state of the model. In this work, we propose a method to discover the set of labels of training samples from only the gradient of the last layer and the id to label mapping. Our method is applicable to a wide variety of model architectures across multiple domains. We demonstrate the effectiveness of our method for model training in two domains - image classification, and automatic speech recognition. Furthermore, we show that existing reconstruction techniques improve their efficacy when used in conjunction with our method. Conversely, we demonstrate that gradient quantization and sparsification can significantly reduce the success of the attack.
In real-time systems, energy consumption is one of the most critical challenges. Dynamic voltage and frequency scaling (DVFS) algorithms have been widely applied to balance the trade-off between performance and power ...
详细信息
In real-time systems, energy consumption is one of the most critical challenges. Dynamic voltage and frequency scaling (DVFS) algorithms have been widely applied to balance the trade-off between performance and power consumption. We first review some classic DVFS methods, e.g., LA-EDF and CCEDF, and then a hybrid DVFS algorithm (Soft-LA2) that we proposed recently. Soft-LA2 needs domain knowledge to set the trade-off parameter manually to achieve energy saving compared with classic methods. Motivated by recent deep learning technologies applied in DVFS, in this paper, we propose a new framework for Soft-LA2. Our method automatically optimizes parameters for Soft-LA2, which leads to more power saving compared with random parameter setting in Soft-LA2. Simulation results show that under certain neuralnetwork architecture setting, our method can find the best parameters automatically to achieve power saving performance. Furthermore, with test data, our proposed method saves 4.9% and 9.6% power consumption compared with relevant learning method DQL-EES and random setting Soft-LA2, respectively.
Depthwise Separable Convolution can effectively reduce parameters and operations with little loss in precision, which becomes more and more popular in many innovative neuralnetworks such as MobileNet and Xception. Du...
详细信息
Depthwise Separable Convolution can effectively reduce parameters and operations with little loss in precision, which becomes more and more popular in many innovative neuralnetworks such as MobileNet and Xception. Due to limited computing resources and storage space, how to deploy Depthwise Separable CNN(DSCNN) in a more efficient manner for resource constraint scenarios is still an open and important problem to be explored. In this paper, a Fine-Grained Pipelined Acceleration scheme (FGPA) is proposed. Different from existing works, we partition the DSCNN accelerator into multiple computational modules that support fine-grained operations according to resource constraint and data dependency analysis, which is able to support starting next layer computation before previous layer completion. Along with the proposed scheduling strategy Start as Soon as Possible (SSP), we can accelerate DSCNN in a pipelined parallel manner, and all the intermediate data between layers can be kept on-chip memory to avoid the stream-out cost. In this way, we can not only improve the utilization of on-chip memory and computing resources but also reduce the network computational delay. As a case study, MobileNetV1 is implemented on Zynq XC7Z045. Experimental results indicate that the proposed DSCNN accelerator can achieve 99.68 GOPS and 0.17 GOPS/DSP under 250MHz clock frequency.
暂无评论