Fluorescence microscopy imaging technology is a crucial imaging technology widely used in biomedical fields such as brain science. However, its images often have random noise without a fixed pattern, causing a series ...
详细信息
This paper describes a software implementation of a fast distributed scatterer search algorithm for the problem of displacement velocity calculation based on the Apache Spark platform. A complete scheme for calculatin...
详细信息
This paper describes a software implementation of a fast distributed scatterer search algorithm for the problem of displacement velocity calculation based on the Apache Spark platform. A complete scheme for calculating displacement velocities by the persistent scatterer method is considered. The proposed algorithm is integrated into the scheme after the stage of subpixel-accuracy alignment of a stack of time-series images. The search for distributed scatterers is carried out independently in shift windows over the entire area of the image. The presence of distributed scatterers is determined based on the assumption that pairs of samples in the window, which are composed of vectors of complex pixel values in each of the N images, are homogeneous. This assumption stems from the fulfillment of the Kolmogorov-Smirnov criterion for each pair. Toestimate phases of homeogenic pixels, the maximization problem is solved. It is shown that the proposed algorithm is not iterative and can be implemented in the framework of the parallel computing paradigm. Toenable distributed in-memory processing of radar data arrays (from 60 images) across many physical nodes in a network environment, we use the Apache Spark parallelprocessing platform. In this case, the time it takes to find distributed scatterers is reduced by a factor of 10 on average as compared to a single-processor implementation of the algorithm. The comparative results of testing the computing system on a demo cluster are presented. The algorithm is implemented in Python with a detailed description of the objects and methods of the algorithm.
K-SVD (K-Singular Value Decomposition)is a com-monly used dictionary learning method that progressively opti-mizes dictionaries to better represent data by iterating sparse coding and dictionary update steps. However,...
详细信息
ISBN:
(数字)9798331511074
ISBN:
(纸本)9798331511081
K-SVD (K-Singular Value Decomposition)is a com-monly used dictionary learning method that progressively opti-mizes dictionaries to better represent data by iterating sparse coding and dictionary update steps. However, with the increasing data size, the traditional K -SVD method faces significant challenges in terms of computational efficiency. Therefore, this paper proposes an asynchronous distributedparallel K-SVD method based on Ray framework to cope with the computational efficiency problem in large-scale data processing. This method achieves efficient parallel computation by decomposing the sparse coding and dictionary updating steps of the K-SVD method into multiple independent tasks and utilizing the distributed scheduling capability of the Ray framework. Since the zero elements of the coefficient matrix are unevenly distributed, we introduce a stride partitioning approach when performing the decomposition. In addition, in order to avoid the method falling into local optima and failing to find global optima due to parallelism, this paper synchronizes with the global at the same time as the atomic update of the dictionary. Meanwhile, for the purpose of avoiding the overhead caused by data synchronization, a parameter server is introduced to realize asynchronous updating of dictionary and coefficient matrix to reduce the overhead caused by communication. The experimental results show that, compared with the traditional K-SVD method, the method proposed in this paper has a significant advantage in computational efficiency and is able to efficiently process hyperspectral image datasets under the premise of guaranteeing the sparse representation accuracy. Compared with the synchronous parallel K-SVD method, the proposed method has faster error convergence and does not fall into local optima.
Intelligent transportation is an important guarantee for the safety and efficiency of urban transportation in smart cities, and regular road pavement inspection is the focus of road and bridge maintenance in intellige...
详细信息
With image generation and manipulation as one of impressive progress of convolutional neural networks (CNNs), facial image synthesis methods, e.g., DeepFakes, pose serious challenges to social and personal security. S...
详细信息
ISBN:
(纸本)9781510642782;9781510642775
With image generation and manipulation as one of impressive progress of convolutional neural networks (CNNs), facial image synthesis methods, e.g., DeepFakes, pose serious challenges to social and personal security. Specifically, we find that (1) CNN-based synthesized facial image detection methods generally fail to identify synthesized images generated by other synthesis methods;(2) classical detection methods exploiting one-class support vector machines (SVMs) and traditional features of video clips fail when only one image is available. In view of the above challenges, we propose and experimentally verify a method combining CNNs features and one-class SVMs, which not only effectively detects synthesized facial images generated by different methods, but also has good robustness to the variances of the scene content.
The Segment Anything Model (SAM) has demonstrated remarkable capabilities in its performance on natural images. However, it faces considerable challenges when applied to medical datasets. Specifically, the performance...
详细信息
In order to realize the multi-level grid management and efficient processing of massive land resource data, a land resource multi-level grid management platform LR-MGSP based on the cloud server Amazon AWS architectur...
详细信息
Subgraph search problems such as maximal clique enumeration and subgraph matching generate a search-space tree which is traversed in depth-first manner by serial backtracking algorithms that are recursive. Since Jenki...
详细信息
As neural network models become gigantic, they increasingly demand more time and memory for training. To meet these demands, advanced parallel computing techniques have become essential. Our research focuses on hybrid...
详细信息
ISBN:
(数字)9798331524937
ISBN:
(纸本)9798331524944
As neural network models become gigantic, they increasingly demand more time and memory for training. To meet these demands, advanced parallel computing techniques have become essential. Our research focuses on hybrid parallelism, an extension of pipeline parallelism. Pipeline parallelism splits the neural network into sub-networks distributed across a sequence of processing units, enabling simultaneous processing of different data segments on each device. Hybrid parallelism extends this concept by allocating multiple devices to each sub-network. Our research focuses on optimizing hybrid parallelism by improving how the model is partitioned and how computational devices are assigned. We address these issues by modeling the neural network as a directed acyclic graph of tensor operators, and then demonstrating that optimally partitioning this graph is NP-complete. Then, we propose a two-step approach. The first step is to determine a sequence of nodes. The second step is dynamic programming, which partitions the sequence to maintain balance across the assigned devices. In transforming the graph into a sequence, we explore two methods: one employs topological sorting, while the other clusters non-sequential subgraphs. We apply both methods and select the more effective one based on performance outcomes. We implement our algorithm and conduct experiments. The results show substantial enhancements in both the speed of partitioning and training throughput, with speedups reaching up to 23 in partitioning time and a 1.3 -fold increase in training throughput.
Many dynamic programming approaches are existing for 1-0 Knapsack problem (KP) for fast GPU-based solution. These dynamic programming methods can be used for solving the problem of Bounded Knapsack Problem (BKP) after...
详细信息
暂无评论