检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Wu, Hao Deng, Junxiao Yu, Minchen Yu, Yue Liu, Yaochen Fan, Hao Wu, Song Wang, Wei National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China Hong Kong University of Science and Technology Hong Kong The Chinese University of Hong Kong Shenzhen China

Serverless computing has gained significant traction for machine learning inference applications, which are often deployed as serverless workflows consisting of multiple CPU and GPU functions with data dependency. However, existing data-passing solutions for serverless computing primarily reply on host memory for fast data transfer, mandating substantial data movement and resulting in salient I/O overhead. In this paper, we present FaaSTube, a GPU-efficient data passing system for serverless inference. FaaSTube manages intermediate data within a GPU memory pool to facilitate direct data exchange between GPU functions. It enables fine-grained bandwidth sharing over PCIe and NVLink, minimizing data-passing latency for both host-to-GPU and GPU-to-GPU while providing performance isolation between functions. Additionally, FaaSTube implements an elastic GPU memory pool that dynamically scales to accommodate varying data-passing demands. Evaluations on real-world applications show that FaaSTube reduces end-to-end latency by up to 90% and achieves up to 12x higher throughput compared to the state-of-the-art. Copyright © 2024, The Authors. All rights reserved.

关键词： Computer graphics equipment

来源：评论

学校读者我要写书评

暂无评论

FusionANNS: An Efficient CPU/GPU Cooperative Processing Architecture for Billion-scale Approximate Nearest Neighbor Search

arXiv

引用

arXiv 2024年

作者： Tian, Bing Liu, Haikun Tang, Yuhang Xiao, Shihai Duan, Zhuohui Liao, Xiaofei Zhang, Xuecang Zhu, Junhua Zhang, Yu National Engineering Research Center for Big Data Technology and System Service Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China Huawei Technologies Co. Ltd China

Approximate nearest neighbor search (ANNS) has emerged as a crucial component of database and AI infrastructure Ever-increasing vector datasets pose significant challenges in terms of performance, cost, and accuracy for ANNS services None of modern ANNS systems can address these issues simultaneously. We present FusionANNS, a high-throughput, low-latency cost-efficient, and high-accuracy ANNS system for billion scale datasets using SSDs and only one entry-level GPU The key idea of FusionANNS lies in CPU/GPU collabo rative filtering and re-ranking mechanisms, which signifi cantly reduce I/O operations across CPUs, GPU, and SSDs to break through the I/O performance bottleneck. Specifically we propose three novel designs: (1) multi-tiered indexing to avoid data swapping between CPUs and GPU, (2) heuristic re-ranking to eliminate unnecessary I/Os and computations while guaranteeing high accuracy, and (3) redundant-aware I/O deduplication to further improve I/O efficiency. We imple ment FusionANNS and compare it with the state-of-the-art SSD-based ANNS system–SPANN and GPU-accelerated in memory ANNS system–RUMMY. Experimental results show that FusionANNS achieves 1) 9.4-13.1× higher query per second (QPS) and 5.7-8.8× higher cost efficiency compared with SPANN;2) and 2-4.9× higher QPS and 2.3-6.8× higher cost efficiency compared with RUMMY, while guaranteeing low latency and high accuracy. Copyright © 2024, The Authors. All rights reserved.

关键词： Nearest neighbor search

来源：评论

学校读者我要写书评

暂无评论

No Free Lunch Theorem for Privacy-Preserving LLM Inference

arXiv

引用

arXiv 2024年

作者： Zhang, Xiaojin Fei, Yulin Kang, Yan Chen, Wei Fan, Lixin Jin, Hai Yang, Qiang Huazhong University of Science and Technology China WeBank China Hong Kong University of Science and Technology China National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan430074 China

Individuals and businesses have been significantly benefited by Large Language Models (LLMs) including PaLM, Gemini and ChatGPT in various ways. For example, LLMs enhance productivity, reduce costs, and enable us to focus on more valuable tasks. Furthermore, LLMs possess the capacity to sift through extensive datasets, uncover underlying patterns, and furnish critical insights that propel the frontiers of technology and science. However, LLMs also pose privacy concerns. Users’ interactions with LLMs may expose their sensitive personal or company information. A lack of robust privacy safeguards and legal frameworks could permit the unwarranted intrusion or improper handling of individual data, thereby risking infringements of privacy and the theft of personal identities. To ensure privacy, it is essential to minimize the dependency between shared prompts and private information. Various randomization approaches have been proposed to protect prompts’ privacy, but they may incur utility loss compared to unprotected LLMs prompting. Therefore, it is essential to evaluate the balance between the risk of privacy leakage and loss of utility when conducting effective protection mechanisms. The current study develops a framework for inferring privacy-protected Large Language Models (LLMs) and lays down a solid theoretical basis for examining the interplay between privacy preservation and utility. The core insight is encapsulated within a theorem that is called as the NFL (abbreviation of the word No-Free-Lunch) Theorem. Copyright © 2024, The Authors. All rights reserved.

关键词： Computational linguistics

来源：评论

学校读者我要写书评

暂无评论

A Data-Centric Accelerator for High-Performance Hypergraph Processing 22

A Data-Centric Accelerator for High-Performance Hypergraph P...

引用

Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture

作者： Qinggang Wang Long Zheng Ao Hu Yu Huang Pengcheng Yao Chuangyi Gui Xiaofei Liao Hai Jin Jingling Xue National Engineering Research Center for Big Data Technology and System/Services Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China and Zhejiang Lab Hangzhou China National Engineering Research Center for Big Data Technology and System/Services Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China School of Computer Science and Engineering University of New South Wales Sydney NSW Australia

ISBN: (纸本)9781665462723

Hypergraph processing has emerged as a powerful approach for analyzing complex multilateral relationships among multiple entities. Past research on building hypergraph systems suggests that changing the scheduling order of bipartite edge tasks can improve the overlap-induced data locality in hypergraph processing. However, due to the complex intertwined connections between vertices and hyperedges, it is almost impossible to find a locality-optimal scheduling order. Thus, these task-centric hypergraph systems often suffer from substantial off-chip *** this paper, we first propose a novel data-centric Load-Trigger-Reduce (LTR) execution model to exploit fully the locality in hypergraph processing. Unlike a task-centric model that loads the required data along with a task, our LTR model invokes tasks as per the data used. Specifically, once the hypergraph data is loaded into the on-chip memory, all of its relevant computation tasks will be triggered simultaneously to output intermediate results, which are finally reduced to update the final results. Our LTR model enables all hypergraph data to be accessed once in each iteration. To fully exploit the LTR performance potential, we further architect an LTR-driven hypergraph accelerator, XuLin, which features with an adaptive data loading mechanism to minimize the loading cost via chunk merging at runtime. XuLin is also equipped with a priority-based differential data reduction scheme to reduce the impact of conflicting updates on performance. We have implemented XuLin both on a Xilinx Alveo U250 FPGA card and using a cycle-accurate simulator. The results show that XuLin outperforms the state-of-the-art hypergraph processing solutions Hygra and ChGraph by 20.47× and 8.77× on average, respectively.

关键词： hypergraph data-centric accelerator

来源：评论

学校读者我要写书评

暂无评论

GradedDAG: An Asynchronous DAG-based BFT Consensus with Lower Latency

GradedDAG: An Asynchronous DAG-based BFT Consensus with Lowe...

引用

Reliable Distributed systems

作者： Xiaohai Dai Zhaonan Zhang Jiang Xiao Jingtao Yue Xia Xie Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China School of Computer Science and Technology Hainan University China

To enable parallel processing, the Directed Acyclic Graph (DAG) structure is introduced to the design of asyn-chronous Byzantine Fault Tolerant (BFT) consensus protocols, known as DAG-based BFT. Existing DAG-based BFT protocols operate in successive waves, with each wave containing three or four Reliable Broadcast (RBC) rounds to broadcast data, resulting in high latency due to the three communication steps required in each RBC. For instance, Tusk, a state-of-the-art DAG-based BFT protocol, has a good-case latency of 7 communication steps and an expected worst latency of 21 communication steps. To reduce latency, we propose GradedDAG, a new DAG-based BFT consensus protocol based on our adapted RBC called Graded RBC (GRBC) and the Consistent Broadcast (CBC), with each wave consisting of only one GRBC round and one CBC round. Through GRBC, a replica can deliver data with a grade of 1 or 2, and a non-faulty replica delivering the data with grade 2 can ensure that more than 2/3 of replicas have delivered the same data. Meanwhile, through CBC, data delivered by different non-faulty replicas must be identical. In each wave, a block in the GRBC round will be elected as the leader. If a leader block has been delivered with grade 2, it and all its ancestor blocks can be committed. GradedDAG offers a good-case latency of 4 communication steps and an expected worst latency of 7.5 communication steps, significantly lower than the state-of-the-art. Experimental results demonstrate GradedDAG's feasibility and efficiency.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Efficient mobile code offloading in heterogeneous wireless networks

Efficient mobile code offloading in heterogeneous wireless n...

引用

作者： Lu, Feng Liu, Ruoxue Li, Wei Jin, Hai Zomaya, Albert Y. Services Computing Technology and System Lab Big Data Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China Center for Distributed and High Performance Computing School of Information Technologies The University of Sydney Sydney Australia

Mobile data offloading has already appeared to offer the means of addressing the challenges of limited computing capability and battery life of mobile devices. Most existing code offloading frameworks only consider migrating an application within a single network at a time and failed to fully utilize the energy efficiency mechanism of the latest CPU chips. To tackle these issues, we propose an offloading approach under both a multitasking environment and a heterogeneous network to increase energy and execution efficiency. The offloading problem was formulated as a biobjective optimization problem that aims to save energy and keep a good performance by combining mobile cloud computing with *** technology under a heterogeneous network equipped with cellular and WiFi connectivity. By varying the applications and network scenarios, the experimental results show that with offloading a single application using our proposed framework, the data-intensive application can obviously benefit when the access point density of WiFi reaches 0.0002 unit/m2. Under multiple application scenarios, our proposed framework can increase processing speed by an average of 0.3× over the single application and can save an average of 25% power over the single application. © 2019 John Wiley & Sons, Ltd.

关键词： Heterogeneous networks

来源：评论

学校读者我要写书评

暂无评论

QoS-Aware and Cost-Efficient Dynamic Resource Allocation for Serverless ML Workflows

QoS-Aware and Cost-Efficient Dynamic Resource Allocation for...

引用

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Hao Wu Junxiao Deng Hao Fan Shadi Ibrahim Song Wu Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China Inria Univ. Rennes CNRS IRISA France

Machine Learning (ML) workflows are increasingly deployed on serverless computing platforms to benefit from their elasticity and fine-grain pricing. Proper resource allocation is crucial to achieve fast and cost-efficient execution of serverless ML workflows (specially for hyperparameter tuning and model training). Unfortunately, existing resource allocation methods are static, treat functions equally, and rely on offline prediction, which limit their efficiency. In this paper, we introduce CE-scaling – a Cost-Efficient autoscaling framework for serverless ML work-flows. During the hyperparameter tuning, CE-scaling partitions resources across stages according to their exact usage to minimize resource waste. Moreover, it incorporates an online prediction method to dynamically adjust resources during model training. We implement and evaluate CE-scaling on AWS Lambda using various ML models. Evaluation results show that compared to state-of-the-art static resource allocation methods, CE-scaling can reduce the job completion time and the monetary cost by up to 63% and 41% for hyperparameter tuning, respectively; and by up to 58% and 38% for model training.

关键词：

来源：评论

学校读者我要写书评

暂无评论

LightDAG: A Low-latency DAG-based BFT Consensus through Lightweight Broadcast

LightDAG: A Low-latency DAG-based BFT Consensus through Ligh...

引用

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Xiaohai Dai Guanxiong Wang Jiang Xiao Zhengxuan Guo Rui Hao Xia Xie Hai Jin National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology China School of Computer Science and Artificial Intelligence Wuhan University of Technology China School of Computer Science and Technology Hainan University China

ISBN: (数字)9798350387117

ISBN: (纸本)9798350387124

To improve the throughput of Byzantine Fault Tolerance (BFT) consensus protocols, the Directed Acyclic Graph (DAG) topology has been introduced to parallel data processing, leading to the development of DAG-based BFT consensus. However, existing DAG-based works heavily rely on Reliable Broadcast (RBC) protocols for block broadcasting, which introduces significant latency due to the three communication steps involved in each RBC. For instance, DAgrider, a representative DAG-based protocol, exhibits the best latency of 12 steps, considerably higher than non-DAG protocols like PBFT, which only requires 3 steps. To tackle this issue, we propose LightDAG, which replaces RBC with lightweight broadcasting protocols such as Consistent Broadcast (CBC) and Plain Broadcast (PBC). Since CBC and PBC can be implemented in two and one communication steps, respectively, LightDAG achieves low *** our proposal, we present two variants of LightDAG, namely LightDAG1 and LightDAG2, each providing a trade-off between the best latency and the expected worst latency. In LightDAG1, every block is broadcast using CBC, which exhibits a best latency of 5 steps and an expected worst latency of 14 steps. Since CBC cannot guarantee the totality property, we design a block retrieval mechanism in LightDAG1 to assist replicas in retrieving missing blocks. LightDAG2 utilizes a combination of PBC and CBC for block broadcasting, resulting in the best latency of 4 steps and an expected worst latency of 12(t+1) steps, where t represents the number of actual Byzantine replicas. Since a Byzantine replica may equivocate through PBC, LightDAG2 prohibits blocks from directly referencing contradictory blocks. To ensure liveness, we propose a mechanism to identify and exclude Byzantine replicas if they engage in equivocation attacks. Extensive experiments have been conducted to evaluate LightDAG, and the results demonstrate its feasibility and efficiency.

关键词： Fault tolerance Distributed processing Directed acyclic graph Fault tolerant systems Broadcasting Throughput Data processing

来源：评论

学校读者我要写书评

暂无评论

A Scalable, Efficient, and Robust Dynamic Memory Management Library for HLS-based FPGAs

A Scalable, Efficient, and Robust Dynamic Memory Management ...

引用

IEEE/ACM International Symposium on Microarchitecture (MICRO)

作者： Qinggang Wang Long Zheng Zhaozeng An Shuyi Xiong Runze Wang Yu Huang Pengcheng Yao Xiaofei Liao Hai Jin Jingling Xue National Engineering Research Center for Big Data Technology and System/Services Computing Technology and System Lab/Cluster and Grid Computing Lab School of Computer Science and Technology Huazhong University of Science and Technology Wuhan China Zhejiang Lab Hangzhou China School of Computer Science and Engineering University of New South Wales Sydney NSW Australia

ISBN: (数字)9798350350579

ISBN: (纸本)9798350350586

Nowadays, high-level synthesis (HLS) has gained prominence for FPGA-based architecture prototyping, enhancing productivity significantly. Despite this advancement, HLS tools are impeded by a critical drawback: they lack support for dynamic memory management (DMM), leading to static mem-ory allocation and suboptimal use of memory resources. In response, numerous efforts have been made to develop DMM solutions compatible with HLS. However, our analysis indicates that existing solutions fail to concurrently meet the desired trifecta of scalability (efficient management of memory of any size), efficiency (minimal latency in memory (de-)allocation), and robustness (low allocation failure rates). This limitation hampers their applicability in real-world scenarios. In this paper, we introduce GraDMM, a “three-birds-one- stone” solution that comprehensively enhances the scalability, efficiency, and robustness of DMM. The key insight is to formulate memory (de-)allocation as graph analytics and lever-age sophisticated FPGA-based graph processing techniques. To achieve scalability, GraDMM specializes a simplified pipeline that significantly suppresses resource utilization expansion caused by managed memory scaling. This is crucial for managing arbitrarily sized memory on resource-limited FPGA platforms. For efficiency, GraDMM implements a data-centric concurrent traversal scheme and a shortcut-assisted fast traversal policy to accelerate (de-)allocation-guided graph traversal, reducing mem-ory (de-)allocation latency. To enhance robustness, GraDMM incorporates an adaptive memory defragmenter that defragments managed memory to minimize fragmentation-induced allocation failures. GraDMM is encapsulated as a library, providing high- level interfaces for users and ensuring synthesizability with Vi- vado HLS. Experimental results demonstrate that GraDMM out-performs three state-of-the-art HLS-compatible DMM solutions by significant margins: 56.71 %-85.59% in resource consumption sa

关键词：

来源：评论

学校读者我要写书评

暂无评论

Working Smarter Not Harder: Hybrid Cooling for Deep Learning in Edge Datacenters

IEEE Transactions on Sustainable Computing

引用

IEEE Transactions on Sustainable computing 2025年

作者： Pei, Qiangyu Yuan, Yongjie Hu, Haichuan Wang, Lin Zhang, Dong Yan, Bingheng Yu, Chen Liu, Fangming Huazhong University of Science and Technology National Engineering Research Center for Big Data Technology and System Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and Technology 1037 Luoyu Road Wuhan430074 China Paderborn University TU Darmstadt Germany Jinan Inspur Data Co. Ltd. China Huazhong University of Science and Technology Peng Cheng Laboratory China

The proliferation of deep-learning-based mobile and IoT applications has driven the increasing deployment of edge datacenters equipped with domain-specific accelerators. The unprecedented computing power offered by these accelerators puts a heavy burden on the cooling system, motivating more potent cooling techniques like cold water cooling. However, we observe that cold water cooling results in significant energy waste in edge datacenters due to the fluctuating resource utilization both spatially and temporally. To tackle this issue, we propose the concept of "working smarter"by slowing down accelerators deliberately whenever possible and enabling warm water cooling during these times to achieve cooling efficiency. Based on this concept, we develop Hyco - a hybrid water cooling system tailored for edge datacenters running deep learning workloads. First, Hyco features a zone-based cooling architecture enabling dynamic switching between cold water and warm water cooling. Then, based on a lightweight latency estimation method, Hyco incorporates a learning-based scheduling scheme to determine "which"accelerator workers and "when"to slow down through an adaptive and intelligent power-latency trade-off for deep learning models. The simulation with real-world traces shows that Hyco reduces the cooling energy consumption by up to 34.74× while satisfying latency constraints more than 99% of the time for deep-learning-based applications. © 2016 IEEE.

关键词： Deep reinforcement learning

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：