检索结果-内蒙古大学图书馆

IEEE Transactions on Artificial Intelligence 2024年第12期5卷 6369-6384页

作者： Wang, Xiuhua Wang, Shuai Li, Yiwei Fan, Fengrui Li, Shikang Lin, Xiaodong Huazhong University of Science and Technology Hubei Key Laboratory of Distributed System Security Hubei Engineering Research Center on Big Data Security School of Cyber Science and Engineering Wuhan430074 China University of Electronic Science and Technology of China National Key Laboratory of Wireless Communications Chengdu611731 China Xiamen University of Technology Fujian Key Laboratory of Communication Network and Information Processing Xiamen361024 China University of Guelph School of Computer Science GuelphONN1G 2W1 Canada

Federated learning (FL) is a popular distributed paradigm where enormous clients collaboratively train a machine learning (ML) model under the orchestration of a central server without knowing the clients' private raw data. The development of effective FL algorithms faces multiple practical challenges including data heterogeneity and clients' privacy protection. Despite that numerous attempts have been made to deal with data heterogeneity or rigorous privacy protection, none have effectively tackled both issues simultaneously. In this article, we propose a differentially private and heterogeneity-robust FL algorithm, named DP-FedCVR to mitigate the data heterogeneity by following the client-variance-reduction strategy. Besides, it adopts a sophisticated differential privacy (DP) mechanism where the privacy-amplified strategy is applied, to achieve a rigorous privacy protection guarantee. We show that the proposed DP-FedCVR algorithm maintains its heterogeneity-robustness though DP noises are incorporated, while achieving a sublinear convergence rate for a nonconvex FL problem. Numerical experiments based on image classification tasks are presented to demonstrate that DP-FedCVR provides superior performance over the benchmark algorithms in the presence of data heterogeneity and various DP privacy budgets. © 2020 IEEE.

关键词： Federated learning

来源：评论

学校读者我要写书评

暂无评论

A Heterogeneous PIM Hardware-Software Co-Design for Energy-Efficient Graph Processing

A Heterogeneous PIM Hardware-Software Co-Design for Energy-E...

引用

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Yu Huang Long Zheng Pengcheng Yao Jieshan Zhao Xiaofei Liao Hai Jin Jingling Xue National Engineering Research Center for Big Data Technology and System/Service Computing Technology and System Lab/Cluster and Grid Computing Lab Huazhong University of Science and Technology China UNSW Sydney Australia

ISBN: (数字)9781728168760

ISBN: (纸本)9781728168777

Processing-In-Memory (PIM) is an emerging technology that addresses the memory bottleneck of graph processing. In general, analog memristor-based PIM promises high parallelism provided that the underlying matrix-structured crossbar can be fully utilized while digital CMOS-based PIM has a faster single-edge execution but its parallelism can be low. In this paper, we observe that there is no absolute winner between these two representative PIM technologies for graph applications, which often exhibit irregular workloads. To reap the best of both worlds, we introduce a new heterogeneous PIM hardware, called Hetraph, to facilitate energy-efficient graph processing. Hetraph incorporates memristor-based analog computation units (for high-parallelism computing) and CMOS-based digital computation cores (for efficient computing) on the same logic layer of a 3D die-stacked memory device. To maximize the hardware utilization, our software design offers a hardware heterogeneity-aware execution model and a workload offloading mechanism. For performance speedups, such a hardware-software co-design outperforms the state-of-the-art by 7.54 ×(CPU), 1.56 ×(GPU), 4.13× (memristor-based PIM) and 3.05× (CMOS-based PIM), on average. For energy savings, Hetraph reduces the energy consumption by 57.58× (CPU), 19.93× (GPU), 14.02 ×(memristor-based PIM) and 10.48 ×(CMOS-based PIM), on average.

关键词： Hardware Computer architecture Parallel processing Three-dimensional displays Computational modeling Graphics processing units Memristors

来源：评论

学校读者我要写书评

暂无评论

Spara: An Energy-Efficient ReRAM-Based Accelerator for Sparse Graph Analytics Applications

Spara: An Energy-Efficient ReRAM-Based Accelerator for Spars...

引用

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Long Zheng Jieshan Zhao Yu Huang Qinggang Wang Zhen Zeng Jingling Xue Xiaofei Liao Hai Jin National Engineering Research Center for Big Data Technology and System/Service Computing Technology and System Lab/Cluster and Grid Computing Lab Huazhong University of Science and Technology China UNSW Sydney Australia

ISBN: (数字)9781728168760

ISBN: (纸本)9781728168777

Resistive random access memory (ReRAM) addresses the high memory bandwidth requirement challenge of graph analytics by integrating the computing logic in the memory. Due to the matrix-structured crossbar architecture, existing ReRAM-based accelerators, when handling real-world graphs that often have the skewed degree distribution, suffer from the severe sparsity problem arising from zero fillings and activation nondeterminism, incurring substantial ineffectual *** this paper, we observe that the sparsity sources lie in the consecutive mapping of source and destination vertex index onto the wordline and bitline of a crossbar. Although exhaustive graph reordering improves the sparsity-induced inefficiency, its totally-random (source and destination) vertex mapping leads to expensive overheads. This work exploits the insight in a mid-point vertex mapping with the random wordlines and consecutive bitlines. A cost-effective preprocessing is proposed to exploit the insight by rapidly exploring the crossbar-fit vertex reorderings but ignores the sparsity arising from activation dynamics. We present a novel ReRAM-based graph analytics accelerator, named Spara, which can maximize the workload density of crossbars dynamically by using a tightly-coupled bank parallel architecture further proposed. Results on real-world and synthesized graphs show that Spara outperforms GraphR and GraphSAR by 8.21 × and 5.01 × in terms of performance, and by 8.97 × and 5.68× in terms of energy savings (on average), while incurring a reasonable (

关键词： Indexes Electrodes Random access memory Parallel architectures Resistance Memory management

来源：评论

学校读者我要写书评

暂无评论

FlexiFed: Personalized Federated Learning for Edge Clients with Heterogeneous Model Architectures 23

FlexiFed: Personalized Federated Learning for Edge Clients w...

引用

2023 World Wide Web Conference, WWW 2023

作者： Wang, Kaibin He, Qiang Chen, Feifei Chen, Chunyang Huang, Faliang Jin, Hai Yang, Yun School of Computer Science and Technology Huazhong University of Science and Technology China Department of Computing Technologies Swinburne University of Technology Australia School of Information Technology Deakin University Australia Faculty of Information Technology Monash University Australia Guangxi Key Lab of Human-machine Interaction and Intelligent Decision Nanning Normal University China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab Huazhong University of Science and Technology Wuhan430074 China

ISBN: (纸本)9781450394161

Mobile and Web-of-Things (WoT) devices at the network edge account for more than half of the world's web traffic, making a great data source for various machine learning (ML) applications, particularly federated learning (FL) which offers a promising solution to privacy-preserving ML feeding on these data. FL allows edge mobile and WoT devices to train a shared global ML model under the orchestration of a central parameter server. In the real world, due to resource heterogeneity, these edge devices often train different versions of models (e.g., VGG-16 and VGG-19) or different ML models (e.g., VGG and ResNet) for the same ML task (e.g., computer vision and speech recognition). Existing FL schemes have assumed that participating edge devices share a common model architecture, and thus cannot facilitate FL across edge devices with heterogeneous ML model architectures. We explored this architecture heterogeneity challenge and found that FL can and should accommodate these edge devices to improve model accuracy and accelerate model training. This paper presents our findings and FlexiFed, a novel scheme for FL across edge devices with heterogeneous model architectures, and three model aggregation strategies for accommodating architecture heterogeneity under FlexiFed. Experiments with four widely-used ML models on four public datasets demonstrate 1) the usefulness of FlexiFed;and 2) that compared with the state-of-the-art FL scheme, FlexiFed improves model accuracy by 2.6%-9.7% and accelerates model convergence by 1.24 × -4.04 ×. © 2023 ACM.

关键词： Speech recognition

来源：评论

学校读者我要写书评

暂无评论

SMOG: Accelerating Subgraph Matching on GPUs

SMOG: Accelerating Subgraph Matching on GPUs

引用

IEEE Conference on High Performance Extreme computing (HPEC)

作者： Zhibin Wang Ziheng Meng Xue Li Xi Lin Long Zheng Chen Tian Sheng Zhong State Key Laboratory for Novel Software Technology Nanjing University Alibaba Group“ National Engineering Research Center for Big Data Technology and System/ Services Computing Technology and System Lab/Cluster and Grid Computing Laboratory Huazhong University of Science and Technology Zhejiang Lab

Subgraph matching is a crucial problem in graph theory with diverse applications in fields, such as bioinformatics, social networks and recommendation systems. Accelerating subgraph matching can be greatly facilitated by GPUs, which offer exceptional parallelism and high memory bandwidth. By leveraging the power of multiple GPU cards, subgraph matching can be scaled to achieve unprecedented levels of performance. In this paper, we propose SMOG, an abbreviation for. Subgraph Matching On Multi-Card GPUs. It is a general, high-performance and scalable sub graph matching system that utilizes multi-card GPUs. To address the issue of duplication resulting from subgraph automorphism, SMOG introduces a two-step approach. Firstly, it analyzes the symmetry within the subgraph. Then, it adaptively adjusts the graph preprocessing and generates subgraph-aware GPU codes tailored to the given subgraph. Furthermore, SMOG leverages multi-level parallelism by designing the specific strategy for each level, enabling it to scale from 1 to 1,024 GPU cards, resulting in an extraordinary $553\times$ speedup. We evaluate SMOG on various subgraph queries and datasets. The experimental results demonstrate that SMOG outperforms the triangle-specific system TRUST with an average speedup of $2.94\times$ . And it performs significantly better than the subgraph matching system RPS by $203.55\times$ and the graph processing system Gunrock by 35, $455.52\times$ on average.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Duo: Improving data Sharing of Stateful Serverless Applications by Efficiently Caching Multi-Read data

Duo: Improving Data Sharing of Stateful Serverless Applicati...

引用

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Zhuo Huang Hao Fan Chaoyi Cheng Song Wu Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China

A growing number of applications are moving to serverless architectures for high elasticity and fine-grained billing. For stateful applications, however, the use of serverless architectures is likely to lead to significant performance degradation, as frequent data sharing between different execution stages involves time-consuming remote storage access. Current platforms leverage memory cache to speed up remote access. However, conventional caching strategies show limited performance improvement. We experimentally find that the reason is that current strategies overlook the stage-dependent access patterns of stateful serverless applications, i.e., data that are read multiple times across stages (denoted as multi-read data) are wrongly evicted by data that are read only once (denoted as read-once data), causing a high cache miss ***, we propose a new caching strategy, Duo, whose design principle is to cache multi-read data as long as possible. Specifically, Duo contains a large cache list and a small cache list, which act as Leader list and Wingman list, respectively. Leader list ignores the data that is read for the first time to prevent itself from being polluted by massive read-once data at each stage. Wingman list inspects the data that are ignored or evicted by Leader list, and pre-fetches the data that will probably be read again based on the observation that multi-read data usually appear periodically in groups. Compared to the state-of-the-art works, Duo improves hit ratio by 1.1×-2.1× and reduces the data sharing overhead by 25%-62%.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Scalable Optimal Margin Distribution Machine

arXiv

引用

arXiv 2023年

作者： Wang, Yilin Cao, Nan Zhang, Teng Shi, Xuanhua Jin, Hai National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China

Optimal margin Distribution Machine (ODM) is a newly proposed statistical learning framework rooting in the latest margin theory, which demonstrates better generalization performance than the traditional large margin based counterparts. However, it suffers from the ubiquitous scalability problem regarding both computation time and memory storage as other kernel methods. This paper proposes a scalable ODM, which can achieve nearly ten times speedup compared to the original ODM training method. For nonlinear kernels, we put forward a novel distribution-aware partition method to make the local ODM trained on each partition be close and converge fast to the global one. When linear kernel is applied, we extend a communication efficient SVRG method to accelerate the training further. Extensive empirical studies validate that our proposed method is highly computational efficient and almost never worsen the generalization. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

Co-training partial domain adaptation networks for industrial Fault Diagnosis

arXiv

引用

arXiv 2024年

作者： Chen, Gecheng Luo, Chengwen Li, Jianqiang Chen, Xinkai The National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China The Department of Electronic Information Systems Shibaura Institute of Technology Saitama337-8570 Japan

The partial domain adaptation (PDA) challenge is a prevalent issue in industrial fault diagnosis. Current PDA approaches primarily rely on adversarial learning for domain adaptation and use reweighting strategies to exclude source samples deemed outliers. However, the transferability of features diminishes from general feature extraction layers to higher task-specific layers in adversarial learning-based adaptation modules, leading to significant negative transfer in PDA settings. We term this issue the adaptation-discrimination paradox (ADP). Furthermore, reweighting strategies often suffer from unreliable pseudo-labels, compromising their effectiveness. Drawing inspiration from traditional classification settings where such partial challenge is not a concern, we propose a novel PDA framework called Interactive Residual Domain Adaptation Networks (IRDAN), which introduces domain-wise models for each domain to provide a new perspective for the PDA challenge. Each domain-wise model is equipped with a residual domain adaptation (RDA) block to mitigate the ADP problem. Additionally, we introduce a confident information flow via an interactive learning strategy, training the modules of IRDAN sequentially to avoid cross-interference. We also establish a reliable stopping criterion for selecting the best-performing model, ensuring practical usability in real-world applications. Experiments have demonstrated the superior performance of the proposed IRDAN. Copyright © 2024, The Authors. All rights reserved.

关键词： Adversarial machine learning

来源：评论

学校读者我要写书评

暂无评论

Question-Answer Cross Language Image Matching for Weakly Supervised Semantic Segmentation

arXiv

引用

arXiv 2024年

作者： Deng, Songhe Zhuo, Wei Xie, Jinheng Shen, Linlin Computer Vision Institute School of Computer Science and Software Engineering Shenzhen University China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China

Class Activation Map (CAM) has emerged as a popular tool for weakly supervised semantic segmentation (WSSS), allowing the localization of object regions in an image using only image-level labels. However, existing CAM methods suffer from under-activation of target object regions and false-activation of background regions due to the fact that a lack of detailed supervision can hinder the model’s ability to understand the image as a whole. In this paper, we propose a novel Question-Answer Cross-Language-Image Matching framework for WSSS (QA-CLIMS), leveraging the vision-language foundation model to maximize the text-based understanding of images and guide the generation of activation maps. First, a series of carefully designed questions are posed to the VQA (Visual Question Answering) model with Question-Answer Prompt engineering (QAPE) to generate a corpus of both foreground target objects and backgrounds that are adaptive to query images. We then employ contrastive learning in a Region Image Text Contrastive (RITC) network to compare the obtained foreground and background regions with the generated corpus. Our approach exploits the rich textual information from the open vocabulary as additional supervision, enabling the model to generate high-quality CAMs with a more complete object region and reduce false-activation of background regions. We conduct extensive analysis to validate the proposed method and show that our approach performs state-of-the-art on both PASCAL VOC 2012 and MS COCO datasets. Code is available at: https://***/CVI-SZU/QA-CLIMS Copyright © 2024, The Authors. All rights reserved.

关键词： Chemical activation

来源：评论

学校读者我要写书评

暂无评论

Multi-Stage Transfer Learning Evolutionary Algorithm for Dynamic Multiobjective Optimization

Multi-Stage Transfer Learning Evolutionary Algorithm for Dyn...

引用

Congress on Evolutionary Computation

作者： Qianhui Wang Qingling Zhu Junkai Ji College of Computer Science and Software Engineering Shenzhen University Shenzhen China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen PR China

ISBN: (数字)9798350308365

ISBN: (纸本)9798350308372

Recently, the application of transfer learning within dynamic multiobjective evolutionary algorithms (DMOEAs) has shown significant potential to solve dynamic multiobjective optimization problems (DMOPs). This approach utilizes the transfer learning technique which reuses information from previous environments to accelerate the search process in the new environment. However, the risk of negative transfer can lead to an erroneous search direction and inefficient use of computational resources. To address this issue, this paper proposes a novel multistage transfer learning DMOEA, named MSTL. The algorithm is designed to enhance the convergence and diversity of the population through the combination of the pre-transfer learning stage and the transfer learning and validation stage when environmental changes occur. In the pre-transfer learning stage, a suitable target domain, source domain, and validation classifier sample sets are generated guided by historical information. In the transfer learning and validation stage, the TrAdaboost technique to transfer knowledge from the target to the source domain, with the results verified by two validation classifiers to mitigate negative transfer. Experimental results demonstrate that our proposed algorithm outperforms four competing algorithms in terms of diversity and convergence on a widely recognized benchmark suite.

关键词： Accuracy Heuristic algorithms Transfer learning Sociology Evolutionary computation Benchmark testing Classification algorithms

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：