检索结果-内蒙古大学图书馆

Accelerating Massively distributed Deep Learning Through Efficient Pseudo-Synchronous Update Method

INTERNATIONAL JOURNAL OF parallel PROGRAMMING 2024年第3期52卷 125-146页

作者： Wen, Yingpeng Qiu, Zhilin Zhang, Dongyu Huang, Dan Xiao, Nong Lin, Liang Sun Yat Sen Univ Sch Comp Sci & Engn Guangzhou Guangdong Peoples R China

In recent years, deep learning models have been successfully applied to large-scale data analysis, including image classification, video caption, natural language processing, etc. Large-scale data analyses take advantage of parallel computing to accelerate the speed of model training, in which data parallelism has become the dominant method for deep learning model training due to its high throughput rate. Synchronous stochastic gradient descent optimization becomes a well-recognized optimization method to ensure model convergence, but the overhead of gradients synchronization increases linearly as the number of workers increases, causing a huge waste of time. Although some efficiency-first asynchronous methods have been proposed, these methods cannot guarantee their convergence in large-scale distributed training. To solve this problem, we propose an efficient pseudo-synchronous approach that updates the network with the previous gradient, performing the synchronization of a new gradient to overlap computation and synchronization. This idea will obviously affect the normal convergence of the model, so we propose a novel adaptive exponential smoothing predicted gradient algorithm for model optimization, which can adaptively adjust the confidence coefficient of the history gradient to ensure the normal convergence of the training process. Experiments prove that our method can speed up the training process and achieve a comparable accuracy rate with standard synchronous SGD. Besides, our method has more efficient weak scalability compared to the traditional synchronous SGD and those in previous related work. We apply our methods to image recognition and video caption applications at most 12288 cores with strong scalability on Tianhe ii. Evaluations show that, when configured appropriately, our method attains near-linear scalability using 128 nodes. We get 93.4% weak scaling efficiency on 64 nodes, 90.5% on 128 nodes.

关键词： Deep learning (DL) Stochastic gradient descent (SGD) Optimization distributed computing Tianhe-2 supercomputer Data parallel

来源：评论

学校读者我要写书评

暂无评论

image denoising based on Swin Transformer Residual Conv U-Net 27

Image denoising based on Swin Transformer Residual Conv U-Ne...

引用

27th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and parallel/distributed Computing (SNPD)

作者： Gan, Yong Zhou, Shaohui Chen, Haonan Wang, Yuefeng Zhengzhou Univ Technol Zhengzhou Peoples R China Zhengzhou Univ Light Ind Coll Comp Sci & Technol Zhengzhou Peoples R China

ISBN: (纸本)9798350391961;9798350391954

In the field of computer vision, image denoising remains a fundamental and challenging problem, playing a crucial role in the preprocessing of various image processing tasks. The introduction of Convolutional Neural Networks (CNNs) into the image denoising domain has yielded significant improvements across different levels of visual tasks. In recent years, models based on the Swin Transformer have also been applied to the image denoising field, demonstrating superior denoising performance that surpasses CNN-based methods, thus becoming advanced techniques in current image denoising research. This paper proposes a Swin-Conv module that combines the local modeling capabilities of residual convolutional layers with the non-local modeling capabilities of the Swin Transformer and integrates this module into the UNet architecture for image denoising. For the dataset used in the model training process, data augmentation techniques were employed to randomly enhance the dataset, thereby improving overall robustness. The results indicate that the proposed Swin Transformer Residual Conv U-Net model shows improvement over current advanced networks, achieving PSNR and SSIM values of 36.09 and 0.963 at sigma = 15, 33.87 and 0.915 at sigma = 25, and 28.96 and 0.810 at sigma = 50.

关键词： image Denoising Deep Learning Transformer Convolutional Neural Networks U-Net

来源：评论

学校读者我要写书评

暂无评论

Benchmarking Federated Learning on High-Performance Computing: Aggregation methods and Their Impact 32

Benchmarking Federated Learning on High-Performance Computin...

引用

32nd Euromicro International Conference on parallel, distributed and Network-Based processing (PDP)

作者： Annunziata, Daniela Canzaniello, Marzia Savoia, Martina Cuomo, Salvatore Piccialli, Francesco Univ Naples Federico II Dept Math & Applicat R Caccioppoli Naples Italy

ISBN: (纸本)9798350363074;9798350363081

Federated Learning (FL) diverges from traditional Machine Learning (ML) models decentralizing data utilization, addressing privacy concerns. This approach involves iterative model updates, where individual devices compute gradients based on local data, share updates with a central server, and receive an improved global model. High-Performance Computing (HPC) systems enhance FL efficiency by leveraging parallel processing. In this study, we aim to explore FL efficiency using four aggregation methods on three datasets across six clients, assess metrics like global model accuracy and communication efficiency, and evaluate FL on HPC. We employ Flower, a versatile FL framework, in our experiments. Our chosen datasets include MNIST, Digits, and Semeion Handwritten Digit, distributed among two clients each. We utilize NVIDIA GPUs for computation, with aggregation methods such as FedAvg, FedProx, FedOpt, and FedYogi. Metrics include Convergence Time, Global Model Accuracy, Communication Efficiency, and HPC Throughput. The results will provide insights into FL performance, especially in HPC environments, impacting convergence, communication, and resource utilization.

关键词： Federated learning High-Performance Computing (HPC) Aggregation methods Flower Framework

来源：评论

学校读者我要写书评

暂无评论

Self-supervised Adversarial Hashing for Large-scale image Retrieval 30

Self-supervised Adversarial Hashing for Large-scale Image Re...

引用

30th IEEE International Conference on parallel and distributed Systems, ICPADS 2024

作者： Cao, Yuan Xu, Xue Liu, Junwei Chen, Xiangru Ocean University of China School of Computer Science and Technology Qingdao China

ISBN: (纸本)9798331515966

Hashing has been widely used in large-scale image retrieval due to its high storage and search efficiency. Unsupervised learning saves a significant amount of labor costs compared to supervised learning. Existing unsupervised methods mostly convert unsupervised problems into supervised problems by reconstructing semantic information. In this paper, we propose a novel Self-supervised Adversarial Hashing (SAH) method which utilizes unsupervised semantic reconstruction methods along with self-supervised generative adversarial and contrastive learning methods to improve the model's accuracy and robustness. In addition, we propose a novel method for semantic relationship reconstruction, taking into account the similarity between different categories. Based on this, we have designed a multi-joint loss to further achieve reasonable intra-class aggregation and inter-class differentiation. The experimental results show that the proposed method outperforms state-of-the-art hashing methods for large-scale image retrieval. © 2024 IEEE.

关键词： Self-supervised learning

来源：评论

学校读者我要写书评

暂无评论

Inverse Quantum Fourier Transform Inspired Algorithm for Unsupervised image Segmentation

Inverse Quantum Fourier Transform Inspired Algorithm for Uns...

引用

37th IEEE International parallel and distributed processing Symposium (IPDPS)

作者： Akinola, Taoreed Li, Xiangfang Wilkins, Richard Obiomon, Pamela Qian, Lijun Prairie View A&M Univ Dept Elect & Comp Engn Prairie View TX 77446 USA

ISBN: (纸本)9798350311990

image segmentation is a very popular and important task in computer vision. In this paper, inverse quantum Fourier transform (IQFT) for image segmentation has been explored and a novel IQFT-inspired algorithm is proposed and implemented by leveraging the underlying mathematical structure of the IQFT. Specifically, the proposed method takes advantage of the phase information of the pixels in the image by encoding the pixels' intensity into qubit relative phases and applying IQFT to classify the pixels into different segments automatically and efficiently. To the best of our knowledge, this is the first attempt of using IQFT for unsupervised image segmentation. The proposed method has low computational cost comparing to the deep learning based methods and more importantly it does not require training, thus make it suitable for real-time applications. The performance of the proposed method is compared with K-means and Otsuthresholding. The proposed method outperforms both of them on the PASCAL VOC 2012 segmentation benchmark and the xVIEW2 challenge dataset by as much as 50% in terms of mean Intersection-Over-Union (mIOU).

关键词： Inverse Quantum Fourier Transform Computer Vision image Segmentation

来源：评论

学校读者我要写书评

暂无评论

ER-SFM: Efficient and Robust Cluster-Based Structure from Motion 7th

ER-SFM: Efficient and Robust Cluster-Based Structure from Mo...

引用

7th Chinese Conference on Pattern Recognition and Computer Vision

作者： Ye, Zongxin Li, Wenyu Liu, Sidun Qiao, Peng Dou, Yong Natl Univ Def Technol Coll Comp Natl Key Lab Parallel & Distributed Comp Changsha Peoples R China

ISBN: (纸本)9789819785070;9789819785087

Structure from Motion (SfM) is a fundamental computer vision technique that recovers scene structure and camera motion from multi-view images. When facing large-scale scenarios, cluster-based methods are commonly employed to improve reconstruction efficiency. However, these methods currently face challenges regarding their limited robustness, redundant computation, and drift. To address these issues, we propose a unified pipeline called ER-SfM, which enhances the three key aspects of cluster-based SfM: image clustering, local reconstruction, and merging. In terms of image clustering, we propose a three-stage image clustering method to ensure adequate and reliable connections between clusters. In the local reconstruction stage, we expedite the reconstruction process by eliminating duplicate point cloud computation. In the final merging stage, we introduce a global merging algorithm without scale ambiguity to address the drift problem. Extensive experimental results demonstrate the superior performance of our method in terms of both robustness and efficiency compared to state-of-the-art methods.

关键词： parallel structure from motion 3D Reconstruction image clustering Global averaging

来源：评论

学校读者我要写书评

暂无评论

A large-scale lychee image parallel classification algorithm based on spark and deep learning

引用

COMPUTERS AND ELECTRONICS IN AGRICULTURE 2025年 230卷

作者： Xiao, Yiming Wang, Jianhua Xiong, Hongyi Xiao, Fangjun Huang, Renhuan Hong, Licong Wu, Bofei Zhou, Jinfeng Long, Yongbin Lan, Yubin South China Agr Univ Coll Elect Engn Coll Artificial Intelligence Guangzhou Peoples R China South China Agr Univ Guangdong Lab Lingnan Modern Agr Guangzhou Peoples R China South China Agr Univ Natl Ctr Int collaborat Res precis Agr Aviat Pesti Guangzhou Peoples R China

Accurate and rapid classification of large-scale lychee images is crucial for collecting germplasm resources and studying the characteristics of different lychee varieties, and it requires the construction of accurate classification models and the design of rapid classification algorithms. However, the current deep learning-based classification methods for lychee images are unable to simultaneously meet the processing requirements of accuracy and timeliness in large-scale lychee image classification. To address the problem above, this paper proposes a largescale parallel classification algorithm for lychee images based on Spark and deep learning. Specifically, first, the T_ECBAM_ResNetS-34 model architecture was designed and trained using a self-built dataset covering ten types of lychee images and the PyTorch deep learning framework, which improved the accuracy of model classification;Second, the model inference algorithm trained by PyTorch was restructured, utilizing Apache Spark RDD and broadcast variables and data structures to implement data partitioning and model parallel computation across nodes. The experimental results show that the method proposed in this paper surpasses existing technologies in both classification accuracy and the speed of large-scale lychee image classification.

关键词： Litchi Classification Apache Spark ResNet-34 distributed Model Inference

来源：评论

学校读者我要写书评

暂无评论

QGIP: A Framework Bridging Quantum Grayscale image processing and Applications 22

QGIP: A Framework Bridging Quantum Grayscale Image Processin...

引用

22nd IEEE International Symposium on parallel and distributed processing with Applications, ISPA 2024

作者： Che, Xilong Zhang, Jiale Chen, Shuo Peng, Shun Hu, Juncheng Jilin University College of Computer Science and Technology Changchun China

ISBN: (纸本)9798331509712

Quantum computing offers parallel processing capabilities and resource-saving advantages, particularly useful for managing expansive datasets and complex image processing tasks. Grayscale images, being the simplest single-channel image mode, are frequently employed in artificial intelligence training. Before actual image applications, various image processing operations are typically required. However, the restoration of a grayscale image of dimensions 2n × 2n after a series of linear transformations poses a challenge. Existing methods typically involve finding the inverse of the most recent linear transformation or re-encoding the image followed by repeated operations until the final transformation, resulting in excessive computational overhead and disconnection from subsequent quantum grayscale image applications. To address this issue, we propose a universal quantum linear restoration algorithm for grayscale image, denoted as QLR, which effectively bridges the stages of linear transformation and subsequent image applications. QLR reduces the time complexity from O(2n) to O(n) compared to classical counterpart. Building upon the QLR algorithm, we further propose two quantum resource-optimized compression methods for optional lossless image storage. Combining with other quantum algorithms and techniques, we design a framework (QGIP) aimed at bridging the processes of quantum grayscale image processing and applications. Experiments simulated on the IBM Quantum platform validate the correctness and efficiency of our proposal. © 2024 IEEE.

关键词： image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Low-frequency Radio Interferometric Array Imaging Pipeline Optimization Based on distributed Execution Framework

引用

Chinese Astronomy and Astrophysics 2024年第2期48卷 389-412页

作者： Yao-jie, W.E.I. Jie-lin, F.U. Bao-qiang, L.A.O. Key Laboratory of Cognitive Radio and Information Processing of the Ministry of Education School of Information and Communication Guilin University of Electronic Science and Technology Guilin 541004 China Shanghai Astronomical Observatory Chinese Academy of Sciences Shanghai 200030 China School of Physics and Astronomy Yunnan University Kunming 650500 China

The Square Kilometre Array (SKA) project is an international collaboration to build the world's largest radio telescope, whose sensitivity and measurement speed will be an order of magnitude higher than those of all current radio telescopes. Radio continuum survey is one of the main observation mode of the SKA, and the establishment of a standard map of the survey area based on continuum imaging will provide an important foundation for subsequent astronomical science. The GaLactic and Extragalactic All-sky Murchison Widefield Array survey eXtended (GLEAM-X) is a project of the SKA pilot telescope Murchison Widefield Array (MWA) in 2018—2020. GLEAM-X is a new radio continuum survey project to be carried out with the MWA Phase ii expansion array in 2018—2020. The experience of optimizing the imaging pipeline based on the distributed execution framework will help to solve the problem of massive data processing. In this paper, we describe the process steps of GLEAM-X imaging pipeline, integrate and improve it, and realize parallel processing of multiple pipelines on the China SKA Regional Centre Prototype (CSRC-P), and verify the deployment and test the correctness of the imaging pipeline system using GLEAM-X observation data. The GLEAM-X observations were used to validate the deployment of the imaging pipeline system and test its correctness. Then, to optimize the pipelines and improve the processing efficiency, the Data Activated Liu Graph Engine (DALiuGE) was used to integrate the imaging pipelines into the DALiuGE execution framework to automate the distributed parallel processing of the pipelines. Performance tests and results’ analysis show that the optimized imaging pipeline based on the DALiuGE execution framework has better performance, more flexible adaptability, and scalability than the traditional parallel approach, and can support future large-scale continuum imaging experiments during the first phase of SKA commissioning. © 2019

关键词： general image processing—radio continuum instrumentation interferometers—methods interferometric—techniques observational—techniques

来源：评论

学校读者我要写书评

暂无评论

parallel COREGISTRATION ALGORITHM FOR SAR imageS BASED ON HADOOP

PARALLEL COREGISTRATION ALGORITHM FOR SAR IMAGES BASED ON HA...

引用

IEEE International Geoscience and Remote Sensing Symposium (IGARSS)

作者： Li, Jiawei Zeng, Guobing Xu, Huaping Beihang Univ Sch Elect & Informat Engn Beijing 100191 Peoples R China

ISBN: (纸本)9798350320107

As the availability of SAR images continues to grow, efficient coregistration of massive SAR images presents a greater challenge. Traditional serial coregistration methods impose an unbearable time overhead. To reduce this overhead and make full use of computing resources, a parallel coregistration strategy based on Hadoop is proposed for SAR images. The Hadoop distributed File System (HDFS) is used to store SAR image data in chunks, and Hadoop's distributed computing strategy MapReduce is used to realize distributed parallel processing of SAR images. Two distributed parallel coregistration methods are presented with the proposed parallel strategy: one based on the maximum correlation method and the other on the DEM-assisted coregistration method. These methods are evaluated through coregistration experiments on the same dataset, and they are verified by comparing the coregistration results and processing time.

关键词： SAR Hadoop parallel Coregistration

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：