检索结果-内蒙古大学图书馆

QoS-Aware and Cost-Efficient Dynamic Resource Allocation for Serverless ML Workflows

学校读者我要写书评

暂无评论

QoS-Aware and Cost-Efficient Dynamic Resource Allocation for...

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Hao Wu Junxiao Deng Hao Fan Shadi Ibrahim Song Wu Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China Inria Univ. Rennes CNRS IRISA France

Machine Learning (ML) workflows are increasingly deployed on serverless computing platforms to benefit from their elasticity and fine-grain pricing. Proper resource allocation is crucial to achieve fast and cost-efficient execution of serverless ML workflows (specially for hyperparameter tuning and model training). Unfortunately, existing resource allocation methods are static, treat functions equally, and rely on offline prediction, which limit their efficiency. In this paper, we introduce CE-scaling – a Cost-Efficient autoscaling framework for serverless ML work-flows. During the hyperparameter tuning, CE-scaling partitions resources across stages according to their exact usage to minimize resource waste. Moreover, it incorporates an online prediction method to dynamically adjust resources during model training. We implement and evaluate CE-scaling on AWS Lambda using various ML models. Evaluation results show that compared to state-of-the-art static resource allocation methods, CE-scaling can reduce the job completion time and the monetary cost by up to 63% and 41% for hyperparameter tuning, respectively; and by up to 58% and 38% for model training.

关键词：

SMOG: Accelerating Subgraph Matching on GPUs

学校读者我要写书评

暂无评论

SMOG: Accelerating Subgraph Matching on GPUs

IEEE Conference on High Performance Extreme computing (HPEC)

作者： Zhibin Wang Ziheng Meng Xue Li Xi Lin Long Zheng Chen Tian Sheng Zhong State Key Laboratory for Novel Software Technology Nanjing University Alibaba Group“ National Engineering Research Center for Big Data Technology and System/ Services Computing Technology and System Lab/Cluster and Grid Computing Laboratory Huazhong University of Science and Technology Zhejiang Lab

Subgraph matching is a crucial problem in graph theory with diverse applications in fields, such as bioinformatics, social networks and recommendation systems. Accelerating subgraph matching can be greatly facilitated by GPUs, which offer exceptional parallelism and high memory bandwidth. By leveraging the power of multiple GPU cards, subgraph matching can be scaled to achieve unprecedented levels of performance. In this paper, we propose SMOG, an abbreviation for. Subgraph Matching On Multi-Card GPUs. It is a general, high-performance and scalable sub graph matching system that utilizes multi-card GPUs. To address the issue of duplication resulting from subgraph automorphism, SMOG introduces a two-step approach. Firstly, it analyzes the symmetry within the subgraph. Then, it adaptively adjusts the graph preprocessing and generates subgraph-aware GPU codes tailored to the given subgraph. Furthermore, SMOG leverages multi-level parallelism by designing the specific strategy for each level, enabling it to scale from 1 to 1,024 GPU cards, resulting in an extraordinary $553\times$ speedup. We evaluate SMOG on various subgraph queries and datasets. The experimental results demonstrate that SMOG outperforms the triangle-specific system TRUST with an average speedup of $2.94\times$ . And it performs significantly better than the subgraph matching system RPS by $203.55\times$ and the graph processing system Gunrock by 35, $455.52\times$ on average.

关键词：

Personalized Federated Learning with Enhanced Implicit Generalization

学校读者我要写书评

暂无评论

Personalized Federated Learning with Enhanced Implicit Gener...

International Joint Conference on Neural Networks (IJCNN)

作者： Heping Liu Songbai Liu Junkai Ji Qiuzhen Lin Jianyong Chen Kay Chen Tan College of Computer Science and Software Engineering Shenzhen University Shenzhen China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen China Department of Computing The Hong Kong Polytechnic University

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Integrating personalization into federated learning is crucial for addressing data heterogeneity and surpassing the limitations of a single aggregated model. Personalized federated learning excels at capturing inter-client similarities and meeting diverse client needs through custom-made models. However, even with personalized approaches, it’s essential to aggregate knowledge among clients to ensure universal benefits. This paper proposes Federated Dual Objectives and Dual Models (FedDodm), a novel approach that employs two independent models to separately address explicit personalization and implicit generalization objectives in personalized federated learning. By treating these objectives as distinct loss functions and training models accordingly, we achieve a balance between the two through a fusion method. Extensive experiments across various models and learning tasks demonstrate that FedDodm outperforms state-of-the-art federated learning approaches, marking a significant advancement in effectively integrating personalized and generalized knowledge.

关键词： Training Federated learning Aggregates Neural networks data models

Node Importance Estimation Leveraging LLMs for Semantic Augmentation in Knowledge Graphs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Xinyu Zhang, Tianyu Hou, Chengbin Wang, Jinbao Xue, Jianye Lv, Hairong School of Computing and Artificial Intelligence Fuyao University of Science and Technology Fuzhou China Department of Automation Tsinghua University Beijing China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China

Node Importance Estimation (NIE) is a task that quantifies the importance of node in a graph. Recent research has investigated to exploit various information from Knowledge Graphs (KGs) to estimate node importance scores. However, the semantic information in KGs could be insufficient, missing, and inaccurate, which would limit the performance of existing NIE models. To address these issues, we leverage Large Language Models (LLMs) for semantic augmentation thanks to the LLMs’ extra knowledge and ability of integrating knowledge from both LLMs and KGs. To this end, we propose the LLMs Empowered Node Importance Estimation (LENIE) method to enhance the semantic information in KGs for better supporting NIE tasks. To our best knowledge, this is the first work incorporating LLMs into NIE. Specifically, LENIE employs a novel clustering-based triplet sampling strategy to extract diverse knowledge of a node sampled from the given KG. After that, LENIE adopts the node-specific adaptive prompts to integrate the sampled triplets and the original node descriptions, which are then fed into LLMs for generating richer and more precise augmented node descriptions. These augmented descriptions finally initialize node embeddings for boosting the downstream NIE model performance. Extensive experiments demonstrate LENIE’s effectiveness in addressing semantic deficiencies in KGs, enabling more informative semantic augmentation and enhancing existing NIE models to achieve the state-of-the-art performance. The source code of LENIE is freely available at https://***/XinyuLin-FZ/LENIE. Copyright © 2024, The Authors. All rights reserved.

关键词： Knowledge graph

ECLIPSE: Expunging Clean-label Indiscriminate Poisons via Sparse Diffusion Purification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wang, Xianlong Hu, Shengshan Zhang, Yechao Zhou, Ziqi Zhang, Leo Yu Xu, Peng Wan, Wei Jin, Hai National Engineering Research Center for Big Data Technology and System China Services Computing Technology and System Lab China Cluster and Grid Computing Lab China Hubei Engineering Research Center on Big Data Security China Hubei Key Laboratory of Distributed System Security China School of Cyber Science and Engineering Huazhong University of Science and Technology Wuhan430074 China School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China School of Information and Communication Technology Griffith University SouthportQLD4215 Australia

Clean-label indiscriminate poisoning attacks add invisible perturbations to correctly labeled training images, thus dramatically reducing the generalization capability of the victim models. Recently, defense mechanisms such as adversarial training, image transformation techniques, and image purification have been proposed. However, these schemes are either susceptible to adaptive attacks, built on unrealistic assumptions, or only effective against specific poison types, limiting their universal applicability. In this research, we propose a more universally effective, practical, and robust defense scheme called ECLIPSE. We first investigate the impact of Gaussian noise on the poisons and theoretically prove that any kind of poison will be largely assimilated when imposing sufficient random noise. In light of this, we assume the victim has access to an extremely limited number of clean images (a more practical scene) and subsequently enlarge this sparse set for training a denoising probabilistic model (a universal denoising tool). We then introduce Gaussian noise to absorb the poisons and apply the model for denoising, resulting in a roughly purified dataset. Finally, to address the trade-off of the inconsistency in the assimilation sensitivity of different poisons by Gaussian noise, we propose a lightweight corruption compensation module to effectively eliminate residual poisons, providing a more universal defense approach. Extensive experiments demonstrate that our defense approach outperforms 10 state-of-the-art defenses. We also propose an adaptive attack against ECLIPSE and verify the robustness of our defense scheme. Our code is available at https://***/CGCL-codes/ECLIPSE. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep neural networks

A General Offloading Approach for Near-DRAM Processing-In-Memory Architectures

学校读者我要写书评

暂无评论

A General Offloading Approach for Near-DRAM Processing-In-Me...

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Dan Chen Hai Jin Long Zheng Yu Huang Pengcheng Yao Chuangyi Gui Qinggang Wang Haifeng Liu Haiheng He Xiaofei Liao Ran Zheng National Engineering Research Center for Big Data Technology and System/Services Computing Technology and System Lab/Clusters and Grid Computing Lab Huazhong University of Science and Technology China

Processing-in-memory (PIM) is promising to solve the well-known data movement challenge by performing in-situ computations near the data. Leveraging PIM features is pretty profitable to boost the energy efficiency of applications. Early studies mainly focus on improving the programmability for computation offloading on PIM architectures. They lack a comprehensive analysis of computation locality and hence fail to accelerate a wide variety of applications. In this paper, we present a general-purpose instruction-level offloading technique for near-DRAM PIM architectures, namely IOTPIM, to exploit PIM features comprehensively. IOTPIM is novel with two technical advances: 1) a new instruction offloading policy that fully considers the locality of the whole on-chip cache hierarchy, and 2) an offloading performance benefit prediction model that directly predicts offloading performance benefits of an instruction based on the input dataset characterizes, preserving low analysis overheads. The evaluation demonstrates that IOTPIM can be applied to accelerate a wide variety of applications, including graph processing, machine learning, and image processing. IOT-PIM outperforms the state-of-the-art PIM offloading techniques by 1.28×-1.51× while ensuring offloading accuracy as high as 91.89% on average.

关键词： Analytical models Technological innovation Distributed processing systematics Image processing Computer architecture Machine learning

OblivTime: Oblivious and Efficient Interval Skyline Query Processing Over Encrypted Time-Series data

学校读者我要写书评

暂无评论

IEEE Transactions on Services computing 2025年

作者： Ouyang, Huajie Zheng, Yifeng Wang, Songlei Hua, Zhongyun Harbin Institute of Technology School of Computer Science and Technology Guangdong Shenzhen518055 China The Hong Kong Polytechnic University Department of Electrical and Electronic Engineering Hong Kong Shenzhen University National Engineering Laboratory for Big Data System Computing Technology Shenzhen518055 China

Time-series data is prevalent in many applications like smart homes, smart grids, and healthcare. And it is now increasingly common to store and query time-series data in the cloud. Despite the benefits, data privacy concerns in such outsourced services are pressing, making it imperative to embed privacy assurance mechanisms from the outset. Most existing related works have been focused on querying for different types of aggregate statistics. In this paper, we instead focus on the secure support for advanced interval skyline queries, which allow to identify time series that are not dominated by any other time series within a query time interval. This is valuable for time-series data analytics in applications like remote health monitoring (e.g., identifying patients with high heart rates in a certain week). We present OblivTime, a new system framework for oblivious and efficient interval skyline query processing over encrypted time-series data. OblivTime is built from a synergy of time-series data analytics, lightweight cryptography, and GPU parallel computing, achieving stronger security guarantees and lower online query latency over the state-of-the-art prior work. Extensive experiments demonstrate that OblivTime can achieve up to 666\times speedup in online query latency over the state-of-the-art prior work. © 2008-2012 IEEE.

关键词： Structured Query Language

StreamFP: Learnable Fingerprint-guided data Selection for Efficient Stream Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Shi, Tongjun Zhang, Shuhao Chen, Binbin He, Bingsheng National Engineering Research Center for Big DataTechnology and System Services Computing Technology and System Lab Cluster Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China Singapore University of Technology and Design Singapore National University of Singapore Singapore

Stream Learning (SL) requires models that can quickly adapt to continuously evolving data, posing significant challenges in both computational efficiency and learning accuracy. Effective data selection is critical in SL to ensure a balance between information retention and training efficiency. Traditional rule-based data selection methods struggle to accommodate the dynamic nature of streaming data, highlighting the necessity for innovative solutions that effectively address these challenges. Recent approaches to handling changing data distributions face challenges that limit their effectiveness in fast-paced environments. In response, we propose StreamFP, a novel approach that uniquely employs dynamic, learnable parameters called fingerprints to enhance data selection efficiency and adaptability in stream learning. StreamFP optimizes coreset selection through its unique fingerprint-guided mechanism for efficient training while ensuring robust buffer updates that adaptively respond to data dynamics, setting it apart from existing methods in stream learning. Experimental results demonstrate that StreamFP outperforms state-of-the-art methods by achieving accuracy improvements of 15.99%, 29.65%, and 51.24% compared to baseline models across varying data arrival rates, alongside a training throughput increase of 4.6x. © 2024, CC BY.

关键词： data accuracy

Securely Fine-tuning Pre-trained Encoders Against Adversarial Examples

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Ziqi Li, Minghui Liu, Wei Hu, Shengshan Zhang, Yechao Wan, Wei Xue, Lulu Zhang, Leo Yu Yao, Dezhong Jin, Hai National Engineering Research Center for Big Data Technology and System China Services Computing Technology and System Lab. China Cluster and Grid Computing Lab. China Hubei Engineering Research Center on Big Data Security China Hubei Key Laboratory of Distributed System Security China School of Computer Science and Technology Huazhong University of Science and Technology China School of Software Engineering Huazhong University of Science and Technology China School of Cyber Science and Engineering Huazhong University of Science and Technology China School of Information and Communication Technology Griffith University Australia

With the evolution of self-supervised learning, the pre-training paradigm has emerged as a predominant solution within the deep learning landscape. Model providers furnish pre-trained encoders designed to function as versatile feature extractors, enabling downstream users to harness the benefits of expansive models with minimal effort through fine-tuning. Nevertheless, recent works have exposed a vulnerability in pre-trained encoders, highlighting their susceptibility to downstream-agnostic adversarial examples (DAEs) meticulously crafted by attackers. The lingering question pertains to the feasibility of fortifying the robustness of downstream models against DAEs, particularly in scenarios where the pre-trained encoders are publicly accessible to the attackers. In this paper, we initially delve into existing defensive mechanisms against adversarial examples within the pre-training paradigm. Our findings reveal that the failure of current defenses stems from the domain shift between pre-training data and downstream tasks, as well as the sensitivity of encoder parameters. In response to these challenges, we propose Genetic Evolution-Nurtured Adversarial Fine-tuning (Gen-AF), a two-stage adversarial fine-tuning approach aimed at enhancing the robustness of downstream models. Gen-AF employs a genetic-directed dual-track adversarial fine-tuning strategy in its first stage to effectively inherit the pre-trained encoder. This involves optimizing the pre-trained encoder and classifier separately while incorporating genetic regularization to preserve the model’s topology. In the second stage, Gen-AF assesses the robust sensitivity of each layer and creates a dictionary, based on which the top-k robust redundant layers are selected with the remaining layers held fixed. Upon this foundation, we conduct evolutionary adaptability fine-tuning to further enhance the model’s generalizability. Our extensive experiments, conducted across ten self-supervised training methods and six

关键词： Topology