检索结果-内蒙古大学图书馆

Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Deng, Xiumei Li, Jun Wei, Kang Shi, Long Xiong, Zeihui Ding, Ming Chen, Wen Jin, Shi Poor, H. Vincent School of Electronic and Optical Engineering Nanjing University of Science and Technology Nanjing210094 China The Department of Computing Hong Kong Polytechnic University 999077 Hong Kong The Pillar of Information Systems Technology and Design Singapore University of Technology and Design Singapore Data61 CSIRO SydneyNSW2015 Australia Department of Electronics Engineering Shanghai Jiao Tong University Shanghai200240 China The National Mobile Communications Research Laboratory Southeast University Nanjing210096 China Department of Electrical and Computer Engineering Princeton University NJ08544 United States

Adaptive moment estimation (Adam), as a Stochastic Gradient Descent (SGD) variant, has gained widespread popularity in federated learning (FL) due to its fast convergence. However, federated Adam (FedAdam) algorithms suffer from a threefold increase in uplink communication overhead compared to federated SGD (FedSGD) algorithms, which arises from the necessity to transmit both local model updates and first and second moment estimates from distributed devices to the centralized server for aggregation. Driven by this issue, we propose a novel sparse FedAdam algorithm called FedAdam-SSM, wherein distributed devices sparsify the updates of local model parameters and moment estimates and subsequently upload the sparse representations to the centralized server. To further reduce the communication overhead, the updates of local model parameters and moment estimates incorporate a shared sparse mask (SSM) into the sparsification process, eliminating the need for three separate sparse masks. Theoretically, we develop an upper bound on the divergence between the local model trained by FedAdam-SSM and the desired model trained by centralized Adam, which is related to sparsification error and imbalanced data distribution. By minimizing the divergence bound between the model trained by FedAdam-SSM and centralized Adam, we optimize the SSM to mitigate the learning performance degradation caused by sparsification error. Additionally, we provide convergence bounds for FedAdam-SSM in both convex and non-convex objective function settings, and investigate the impact of local epoch, learning rate and sparsification ratio on the convergence rate of FedAdam-SSM. Experimental results show that FedAdam-SSM outperforms baselines in terms of convergence rate (over 1.1× faster than the sparse FedAdam baselines) and test accuracy (over 14.5% ahead of the quantized FedAdam baselines). Copyright © 2024, The Authors. All rights reserved.

关键词： Stochastic systems

Sewer Image Super-Resolution with Depth Priors and Its Lightweight Network

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Pan, Gang Wang, Chen Sui, Zhijie Guo, Shuai Lv, Yaozhi Li, Honglie Sun, Di Xia, Zixia College of Intelligence and Computing Tianjin University Tianjin China Department of Municipal Engineering Hefei University of Technology Anhui China Key laboratory of Infrastructure Durability Tianjin Municipal Engineering Design and Research Institute Tianjin China West-East Gas Pipeline Company PipeChina Shanghai China College of Artificial Intelligence Tianjin University of Science and Technology Tianjin China Donald Bren School of Information and Computer Sciences University of California IrvineCA United States

The Quick-view (QV) technique serves as a primary method for detecting defects within sewerage systems. However, the effectiveness of QV is impeded by the limited visual range of its hardware, resulting in suboptimal image quality for distant portions of the sewer network. Image super-resolution is an effective way to improve image quality and has been applied in a variety of scenes. However, research on super-resolution for sewer images remains considerably unexplored. In response, this study leverages the inherent depth relationships present within QV images and introduces a novel Depth-guided, Reference-based Super-Resolution framework denoted as DSRNet. It comprises two core components: a depth extraction module and a depth information matching module (DMM). DSRNet utilizes the adjacent frames of the low-resolution image as reference images and helps them recover texture information based on the correlation. By combining these modules, the integration of depth priors significantly enhances both visual quality and performance benchmarks. Besides, in pursuit of computational efficiency and compactness, a super-resolution knowledge distillation model based on an attention mechanism is introduced. This mechanism facilitates the acquisition of feature similarity between a more complex teacher model and a streamlined student model, with the latter being a lightweight version of DSRNet. Experimental results demonstrate that DSRNet significantly improves PSNR and SSIM compared with other methods. This study also conducts experiments on sewer defect semantic segmentation, object detection, and classification on the Pipe dataset and Sewer-ML dataset. Experiments show that the method can improve the performance of low-resolution sewer images in these tasks. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantic Segmentation

Performance Analysis of ZCC-Optical-CDMA over SMF for Fiber-To-The-Home Access Network 2

学校读者我要写书评

暂无评论

Performance Analysis of ZCC-Optical-CDMA over SMF for Fiber-...

2nd Joint International Conference on Emerging computing Technology and Sports, JICETS 2019

作者： Ghazi, Alaan Aljunid, S.A. Fareed, Alaa Idrus, Syed Zulkarnain Syed Rashidi, C.B.M. Albayaty, Marwa Al-Dawoodi, Aras Fakhrudeen, Ahmed M. School of Human Development and Techno Communication University Malaysia Perlis Malaysia School of Computer and Communication Engineering University Malaysia Perlis Malaysia Computer Technical Engineering Alkitab University Iraq Internet Research Laboratory School of Computing University Utara Malaysia Malaysia Computer Science Department College of Computer Science and Information Technology Kirkuk University Iraq Networks Department College of Computer Science and Information Technology Kirkuk University Iraq

To provide a sustainable fiber-to-the-home (FTTH), several multiplexing techniques have been developed for this purpose. The correlation features are the main obstacle behind the network performance limitation, which imposes to a high level of multiple access interference. However, the development of multiplexing techniques helps to overcome these limitations, such as optical-code division multiple access (Optical-CDMA). Optical-CDMA is considered as one of the most powerful solutions for FTTH. This paper aims to enhance FTTH network performance by applying Zero cross-correlation code (ZCC) with Optical-CDMA with maximum link single-mode fiber. In the simulation, the system performance is demonstrated in terms of bit error rate, Q-Factor and eye diagram measurements. © 2020 IOP Publishing Ltd. All rights reserved.

关键词： Fiber to the x

Unsupervised Domain Adaptation on Person Re-Identification via Dual-level Asymmetric Mutual Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Qiong Li, Jiahan Dai, Pingyang Ye, Qixiang Cao, Liujuan Wu, Yongjian Ji, Rongrong Institute of Artificial Intelligence The Media Analytics and Computing Laboratory Department of Artificial Intelligence School of Informatics Xiamen University Xiamen361005 China School of Information and Control Engineering China University of Mining and Technology Xuzhou221000 China The Media Analytics and Computing Laboratory Department of Artificial Intelligence School of Informatics Xiamen University Xiamen361005 China The Peng Cheng Laboratory Shenzhen518066 China The School of Electronics Electrical and Communication Engineering University of Chinese Academy of Sciences Beijing100049 China The Media Analytics and Computing Lab Department of Computer Science School of Informatics Xiamen University Xiamen361005 China The Youtu Laboratory Tencent Shanghai200233 China The Fujian Engineering Research Center of Trusted Artificial Intelligence Analysis and Application Institute of Artificial Intelligence Xiamen University Xiamen361005 China

Unsupervised domain adaptation person re-identification (Re-ID) aims to identify pedestrian images within an unlabeled target domain with an auxiliary labeled source-domain dataset. Many existing works attempt to recover reliable identity information by considering multiple homogeneous networks. And take these generated labels to train the model in the target domain. However, these homogeneous networks identify people in approximate subspaces and equally exchange their knowledge with others or their mean net to improve their ability, inevitably limiting the scope of available knowledge and putting them into the same mistake. This paper proposes a Dual-level Asymmetric Mutual Learning method (DAML) to learn discriminative representations from a broader knowledge scope with diverse embedding spaces. Specifically, two heterogeneous networks mutually learn knowledge from asymmetric subspaces through the pseudo label generation in a hard distillation manner. The knowledge transfer between two networks is based on an asymmetric mutual learning manner. The teacher network learns to identify both the target and source domain while adapting to the target domain distribution based on the knowledge of the student. Meanwhile, the student network is trained on the target dataset and employs the ground-truth label through the knowledge of the teacher. Extensive experiments in Market-1501, CUHK-SYSU, and MSMT17 public datasets verified the superiority of DAML over state-of-the-arts. Copyright © 2023, The Authors. All rights reserved.

关键词： Distillation

Few-shot learning based histopathological image classification of colorectal cancer

学校读者我要写书评

暂无评论

Intelligent Medicine 2024年第4期4卷 256-267页

作者： Rui Li Xiaoyan Li Hongzan Sun Jinzhu Yang Md Rahaman Marcin Grzegozek Tao Jiang Xinyu Huang Chen Li Key Laboratory of Intelligent Computing in Medical Image Ministry of EducationNortheastern UniversityShenyangLiaoning 110167China Cancer Hospital China Medical UniversityShenyangLiaoning 110122China Shengjing Hospital China Medical UniversityShenyang.Liaoning 110000China Institute for Medical Informatics University of LuebeckGermany Department of Knowledge Engineering University of Economics in KatowicePoland School of Intelligent Medicine Chengdu University of Traditional Chinese MedicineChengduSichuan 610075China International Joint Institute of Robotics and Intelligent Systems Chengdu University of Information TechnologyChengduSichuan 610225China School of Computer Science and Engineering University of New South WalesSydneyNSW 2052Australia

Background Colorectal cancer is a prevalent and deadly disease worldwide,posing significant diagnostic *** histopathologic image classification is often inefficient and *** some histopathologists use computer-aided diagnosis to improve efficiency,these methods depend heavily on exten-sive data and specific annotations,limiting their *** address these challenges,this paper proposes a method based on few-shot *** This study introduced a few-shot learning approach that combines transfer learning and contrastive learning to classify colorectal cancer histopathology images into benign and malignant *** model comprises modules for feature extraction,dimensionality reduction,and classification,trained using a combi-nation of contrast loss and cross-entropy *** this paper,we detailed the setup of hyperparameters:n-way,κ-shot,β,and the creation of support,query,and test *** Our method achieved over 98% accuracy on a query dataset with 35 samples per category using only 10 training samples per *** documented the model’s loss,accuracy,and the confusion matrix of the ***,we employed the t-SNE algorithm to analyze and assess the model’s classification *** The proposed model may demonstrate significant advantages in accuracy and minimal data depen-dency,performing robustly across all tested n-way,κ-shot *** consistently achieved over 93% accuracy on comprehensive test datasets,including 1916 samples,confirming its high classification accuracy and strong generalization *** research could advance the use of few-shot learning in medical diagnostics and also lays the groundwork for extending it to deal with rare,difficult-to-diagnose cases.

关键词： Colorectal cancer Few-shot learning Transfer learning Contrastive learning Histopathological images Benign and malignant categories

Performance Optimization for Intelligent Reflecting Surface Assisted Multicast MIMO Networks

学校读者我要写书评

暂无评论

Performance Optimization for Intelligent Reflecting Surface ...

GLOBECOM 2022 - 2022 IEEE Global Communications Conference

作者： Songling Zhang Zhaohui Yang Mingzhe Chen Danpu Liu Kai-Kit Wong H. Vincent Poor Beijing Laboratory of Advanced Information Network Beijing University of Posts and Telecommunications Beijing China Zhejiang University Hangzhou China Zhejiang Lab Hangzhou China Department of Electrical and Computer Engineering and Institute for Data Science and Computing University of Miami Coral Gables FL USA Department of Electronic and Electrical Engineering University College London London UK Department of Electrical and Computer Engineering Princeton University Princeton NJ USA

ISBN: (纸本)9781665435413

In this paper, the problem of maximizing the sum rate of all users in an intelligent reflecting surface (IRS)-assisted millimeter wave multicast multiple-input multiple-output communication system is studied. In the considered model, one IRS is deployed to assist the communication from a multi-antenna base station (BS) to the multi-antenna users that are clustered into several groups. Our goal is to maximize the sum rate of all users by jointly optimizing the transmit beamforming matrices of the BS, the receive beamforming matrices of the users, and the phase shifts of the IRS. To solve this non-convex problem, we first use a block diagonalization method to represent the beamforming matrices of the BS and the users by the phase shifts of the IRS. Then, substituting the expressions of the beamforming matrices of the BS and the users, the original sum-rate maximization problem can be transformed into a problem that only needs to optimize the phase shifts of the IRS. To solve the transformed problem, a manifold method is used. Simulation results show that the proposed scheme can achieve up to 13.3 % gain in terms of the sum rate of all users compared to the algorithm that optimizes the hybrid beamforming matrices of the BS and the users using our proposed scheme and randomly determines the phase shifts of the IRS.

关键词： Manifolds Multicast algorithms Array signal processing Surface waves Simulation Performance gain Matrices

Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Ma, Chuan Li, Jun Wei, Kang Liu, Bo Ding, Ming Yuan, Long Han, Zhu Poor, H. Vincent Zhejiang Laboratory Hangzhou311121 China Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education China The School of Electronic and Optical Engineering Nanjing University of Science and Technology Nanjing210096 China The Department of Computing Hong Kong Polytechnic University 999077 Hong Kong The University of Technology SydneyNSW2007 Australia Data61 CSIRO NSW2015 Australia The School of Computer Science Nanjing University of Science and Technology Nanjing210096 China The Department of Electrical and Computer Engineering The University of Houston HoustonTX77004 United States Department of Computer Science and Engineering Kyung Hee University Seoul Korea Republic of The Department of Electrical and Computer Engineering Princeton University PrincetonNJ08544 United States

Motivated by the advancing computational capacity of distributed end-user equipment (UE), as well as the increasing concerns about sharing private data, there has been considerable recent interest in machine learning (ML) and artificial intelligence (AI) that can be processed on distributed UEs. Specifically, in this paradigm, parts of a ML process are outsourced to multiple distributed UEs. Then the processed information is aggregated on a certain level at a central server, which turns a centralized ML process into a distributed one, and brings about significant benefits. However, this new distributed ML paradigm raises new risks of privacy and security issues. In this paper, we provide a survey on the emerging security and privacy risks of distributed ML from a unique perspective of information exchange levels, which are defined according to the key steps of a ML process, i.e.: i) the level of pre-processed data, ii) the level of learning models, iii) the level of extracted knowledge and, iv) the level of intermediate results. We explore and analyze the potential of threats for each information exchange level based on an overview of the current state-of-the-art attack mechanisms, and then discuss the possible defense methods against such threats. Finally, we complete the survey by providing an outlook on the challenges and possible directions for future research in this critical area. Copyright © 2022, The Authors. All rights reserved.

关键词： Multi agent systems

Comprehensive evaluation of deep and graph learning on drug-drug interactions prediction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Lin, Xuan Dai, Lichang Zhou, Yafang Yu, Zu-Guo Zhang, Wen Shi, Jian-Yu Cao, Dong-Sheng Zeng, Li Chen, Haowen Song, Bosheng Yu, Philip S. Zeng, Xiangxiang Xiangtan China Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education Xiangtan University China College of Informatics Huazhong Agricultural University China The Northwestern Polytechnical University Xi’an China The Xiangya School of Pharmaceutical Sciences Central South University China AIDD Department of Yuyao Biotech Shanghai China Hunan University China The College of Information Science and Engineering Hunan University Changsha China Computer Science The University of Illinois Chicago United States

Recent advances and achievements of artificial intelligence (AI) as well as deep and graph learning models have established their usefulness in biomedical applications, especially in drug-drug interactions (DDIs). DDIs refer to a change in the effect of one drug to the presence of another drug in the human body, which plays an essential role in drug discovery and clinical research. DDIs prediction through traditional clinical trials and experiments is an expensive and time-consuming process. To correctly apply the advanced AI and deep learning, the developer and user meet various challenges such as the availability and encoding of data resources, and the design of computational methods. This review summarizes chemical structure based, network based, NLP based and hybrid methods, providing an updated and accessible guide to the broad researchers and development community with different domain knowledge. We introduce widely-used molecular representation and describe the theoretical frameworks of graph neural network models for representing molecular structures. We present the advantages and disadvantages of deep and graph learning methods by performing comparative experiments. We discuss the potential technical challenges and highlight future directions of deep and graph learning models for accelerating DDIs prediction. Copyright © 2023, The Authors. All rights reserved.

关键词： Forecasting

Resource Allocation and Workload Scheduling for Large-Scale Distributed Deep Learning: A Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liang, Feng Zhang, Zhen Lu, Haifeng Li, Chengming Leung, Victor C.M. Guo, Yanyi Hu, Xiping Artificial Intelligence Research Institute Shenzhen MSU-BIT University 518107 China Guangdong-Hong Kong-Macao Joint Laboratory for Emotional Intelligence and Pervasive Computing Shenzhen MSU-BIT University 518107 China Gansu Provincial Key Laboratory of Wearable Computing School of Information Science and Engineering Lanzhou University 730000 China Department of Electrical and Computer Engineering The University of British Columbia V6T 1Z4 Canada Frontier Cross Disciplinary Research Institute Shenzhen MSU-BIT University China School of Mechanical and Electrical Engineering Beijing Institute of Technology China School of Medical Technology Beijing Institute of Technology 10081 China

With rapidly increasing distributed deep learning workloads in large-scale data centers, efficient distributed deep learning framework strategies for resource allocation and workload scheduling have become the key to high-performance deep learning. The large-scale environment with large volumes of datasets, models, and computational and communication resources raises various unique challenges for resource allocation and workload scheduling in distributed deep learning, such as scheduling complexity, resource and workload heterogeneity, and fault tolerance. To uncover these challenges and corresponding solutions, this survey reviews the literature, mainly from 2019 to 2024, on efficient resource allocation and workload scheduling strategies for large-scale distributed DL. We explore these strategies by focusing on various resource types, scheduling granularity levels, and performance goals during distributed training and inference processes. We highlight critical challenges for each topic and discuss key insights of existing technologies. To illustrate practical large-scale resource allocation and workload scheduling in real distributed deep learning scenarios, we use a case study of training large language models. This survey aims to encourage computer science, artificial intelligence, and communications researchers to understand recent advances and explore future research directions for efficient framework strategies for large-scale distributed deep learning. © 2024, CC BY.

关键词： Fault tolerance