检索结果-内蒙古大学图书馆

22nd IEEE International Symposium on parallel and distributed Processing with Applications, ISPA 2024

作者： Tan, Yifan Mi, Zeyu Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University China

ISBN: (纸本)9798331509712

NVIDIA's H100 Confidential Computing (CC) counters the security hazards inherent in cloud AI workloads. It enforces data encryption to achieve data confidentiality, which leads to substantial throughput reductions as high as 93% in various AI workloads (such as TensorRT, PEFT and vLLM). Confronting this substantial overhead issue, we first delve into the underlying causes through meticulous analysis. This groundwork enables us to devise an innovative runtime system that operates seamlessly in the background, completely transparent to end-users. The cornerstone of our system lies in leveraging multiple encryption workers. Experiments demonstrate that our solution effectively reduces throughput drop to less than 28.1%. © 2024 IEEE.

关键词： Confidential Computing GPU H100 LLM

来源：评论

学校读者我要写书评

暂无评论

Serialization/Deserialization-free State Transfer in Serverless Workflows 24

Serialization/Deserialization-free State Transfer in Serverl...

引用

19th European Conference on Computer systems, EuroSys 2024

作者： Lu, Fangming Wei, Xingda Huang, Zhuobin Chen, Rong Wu, Minyu Chen, Haibo Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University China Shanghai Artificial Intelligence Laboratory China University of Electronic Science and Technology of China China Institute of Parallel and Distributed Systems Germany

ISBN: (纸本)9798400704376

Serialization and deserialization play a dominant role in the state transfer time of serverless workflows, leading to substantial performance penalties during workflow execution. We identify the key reason as a lack of ability to efficiently access the (remote) memory of another function. We propose RMMap, an OS primitive for remote memory map. It allows a serverless function to directly access the memory of another function, even if it is located remotely. RMMap is the first to completely eliminates serialization and deserialization when transferring states between any pairs of functions in (unmodified) serverless workflows. To make remote memory map efficient and feasible, we co-design it with fast networking (RDMA), OS, language runtime, and serverless platform. Evaluations using real-world serverless workloads show that integrating RMMap with Knative reduces the serverless workflow execution time on Knative by up to 2.6 × and improves resource utilizations by 86.3%. © 2024 ACM.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Enhancing Data Trustworthiness in Explorative Analysis: An Interactive Approach for Data Quality Monitoring

引用

SN Computer Science 2024年第5期5卷 465页

作者： Behringer, Michael Hirmer, Pascal Villanueva, Alejandro Rapp, Jannis Mitschang, Bernhard Institute for Parallel and Distributed Systems University of Stuttgart Universitätsstr. 38 Stuttgart 70569 Germany

The volume of data to be analyzed has increased tremendously in recent years. In order to extract knowledge from this data, domain experts gain new insights with the help of graphical analysis tools for explorative analyses. Here, the reliability and trustworthiness of an exploratory analysis is determined by the quality of the underlying data. Existing approaches require manual testing to ensure data quality which is often neglected. This research aims to introduce a novel interactive approach for seamlessly integrating data quality considerations into the process of explorative data analysis conducted by domain experts. We derive requirements, conduct an extensive literature review, and develop an approach that efficiently combines stakeholders’ strengths, allowing unobtrusive data quality integration in interactive analysis. Our approach enhances trustworthiness due to unobtrusive monitoring of data quality within the context of explorative data analysis. Domain experts gain insights more reliably, bridging the gap between technical requirements and domain expertise. In conclusion, our research presents a promising solution for improving the reliability and trustworthiness of explorative data analysis, especially for domain experts who may lack technical knowledge. By seamlessly integrating data quality into the analytical process, we empower domain experts to extract valuable insights from the ever-increasing volume of data, thereby advancing the field of data-driven decision-making. © The Author(s) 2024.

关键词： Data Quality Data Mashups Explorative Data Analysis Human-in-the-Loop

来源：评论

学校读者我要写书评

暂无评论

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU 24

PowerInfer: Fast Large Language Model Serving with a Consume...

引用

30th ACM Symposium on Operating systems Principles, SOSP 2024

作者： Song, Yixin Mi, Zeyu Xie, Haotong Chen, Haibo Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University Shanghai China

ISBN: (纸本)9798400712517

This paper introduces PowerInfer, a high-speed Large Language Model (LLM) inference engine on a personal computer (PC) equipped with a single consumer-grade GPU. The key principle underlying the design of PowerInfer is exploiting the high locality inherent in LLM inference, characterized by a power-law distribution in neuron activation. This distribution indicates that a small subset of neurons, termed hot neurons, are consistently activated across inputs, while the majority, cold neurons, vary based on specific inputs. PowerInfer exploits such an insight to design a GPU-CPU hybrid inference engine: hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons are computed on the CPU, thus significantly reducing GPU memory demands and CPU-GPU data transfers. PowerInfer further integrates adaptive predictors and neuron-aware sparse operators, optimizing the efficiency of neuron activation and computational sparsity. The evaluation shows that PowerInfer significantly outperforms *** by up to 11.69× while retaining model accuracy across various LLMs (including OPT-175B) on a single NVIDIA RTX 4090 GPU. For the OPT-30B model, PowerInfer achieves performance comparable to that of a high-end server-grade A100 GPU, reaching 82% of its token generation rate on a single consumer-grade RTX 4090 GPU. © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.

关键词： Personal computers

来源：评论

学校读者我要写书评

暂无评论

Characterization and Reclamation of Frozen Garbage in Managed FaaS Workloads 24

Characterization and Reclamation of Frozen Garbage in Manage...

引用

19th European Conference on Computer systems, EuroSys 2024

作者： Zhao, Ziming Wu, Mingyu Chen, Haibo Zang, Binyu Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University Shanghai China Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University Engineering Research Center for Domain-specific Operating Systems Ministry of Education China Shanghai China

ISBN: (纸本)9798400704376

FaaS (function-as-a-service) is becoming a popular workload in cloud environments due to its virtues such as auto-scaling and pay-as-you-go. High-level languages like JavaScript and Java are commonly used in FaaS for programmability, but their managed runtimes complicate memory management in the cloud. This paper first observes the issue of frozen garbage, which is caused by freezing cached function instances where their threads have been paused but the unused memory (e.g., garbage) is not reclaimed due to the semantic gap between FaaS and the managed runtime. This paper presents the first characterization of the negative effects induced by frozen garbage with various functions, which uncovers that it can occupy more than half of FaaS instances' memory resources on average. To this end, this paper proposes Desiccant, a freeze-aware memory manager for managed workloads in FaaS, which reclaims idle memory resources consumed by frozen garbage from managed runtime instances and thus notably improves memory efficiency. The evaluation on various FaaS workloads shows that Desiccant can reduce FaaS functions' peak memory consumption by up to 6.72×. Such saved memory consumption allows caching more FaaS instances to reduce the frequency of cold boots (creating instances before function execution) and p99 latency by up to 4.49× and 37.5%, respectively. © 2024 ACM.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption 25

PipeLLM: Fast and Confidential Large Language Model Services...

引用

30th ACM International Conference on Architectural Support for Programming Languages and Operating systems, ASPLOS 2025

作者： Tan, Yifan Tan, Cheng Mi, Zeyu Chen, Haibo Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University Shanghai China Northeastern University Boston United States

ISBN: (纸本)9798400706981

Confidential computing on GPUs, like NVIDIA H100, mitigates the security risks of outsourced Large Language Models (LLMs) by implementing strong isolation and data encryption. Nonetheless, this encryption incurs a significant performance overhead, reaching up to 52.8% and 88.2% throughput drop when serving OPT-30B and OPT-66B, respectively. To address this challenge, we introduce PipeLLM, a user-transparent runtime system. PipeLLM removes the overhead by overlapping the encryption and GPU computation through pipelining-an idea inspired by the CPU instruction pipelining-thereby effectively concealing the latency increase caused by encryption. The primary technical challenge is that, unlike CPUs, the encryption module lacks prior knowledge of the specific data needing encryption until it is requested by the GPUs. To this end, we propose speculative pipelined encryption to predict the data requiring encryption by analyzing the serving patterns of LLMs. Further, we have developed an efficient, low-cost pipeline relinquishing approach for instances of incorrect predictions. Our experiments show that compared with vanilla systems without confidential computing (e.g., vLLM, PEFT, and FlexGen), PipeLLM incurs modest overhead ( © 2025 ACM.

关键词： Modeling languages

来源：评论

学校读者我要写书评

暂无评论

Embracing connected intelligence with the YuanOS architecture:one OS kit for all

引用

Frontiers of Computer Science 2024年第1期18卷 1-3页

作者： Haibo CHEN Ning JIA Jie YIN Institute of Parallel and Distributed Systems School of Electronic Information and Electrical EngineeringShanghai Jiao Tong UniversityShanghai 200240China OS Kernel Lab Huawei Technologies Co.Ltd.Beijing 100085China

Operating system abstracts hardware and creates application execution environment,and thus is the key pillar for IT *** its first invention in 1956,operating system has been evolved over three main stages:Mainframe,PC&Internet,Mobile Internet[1].Now,the entire society is firmly stepping towards the connected intelligence era[2],how should operating system be evolved to embrace this new era?

关键词： hardware execution connected

来源：评论

学校读者我要写书评

暂无评论

MetaConfigurator: A User-Friendly Tool for Editing Structured Data Files

引用

Datenbank-Spektrum 2024年第2期24卷 161-169页

作者： Neubauer, Felix Bredl, Paul Xu, Minye Patel, Keyuriben Pleiss, Jürgen Uekermann, Benjamin Institute for Parallel and Distributed Systems University of Stuttgart Universitätsstraße 38 Stuttgart70569 Germany Institute of Biochemistry and Technical Biochemistry University of Stuttgart Allmandring 31 Stuttgart70569 Germany

Textual formats to structure data, such as JSON, XML, and YAML, are widely used for structuring data in various domains, from configuration files to research data. However, manually editing data in these formats can be complex and time-consuming. Graphical user interfaces (GUIs) can significantly reduce manual efforts and assist the user in editing the files, but developing a file-format-specific GUI requires substantial development and maintenance efforts. To address this challenge, we introduce MetaConfigurator: an open-source web application that generates its GUI depending on a given schema. Our approach differs from other schema-to-UI approaches in three key ways: 1) It offers a unified view that combines the benefits of both GUIs and text editors, 2) it enables schema editing within the same tool, and 3) it supports advanced schema features, including conditions and constraints. In this paper, we discuss the design and implementation of MetaConfigurator, backed by insights from a small-scale qualitative user study. The results indicate the effectiveness of our approach in retrieving information from data and schemas and in editing them. © The Author(s) 2024.

关键词： Graphical user interfaces

来源：评论

学校读者我要写书评

暂无评论

A superconvergence result in the RBF-FD method 9

A superconvergence result in the RBF-FD method

引用

9th European Thermal Sciences Conference, EUROTHERM 2024

作者： Kolar-Pozun, Andrej Jančič, Mitja Kosec, Gregor Parallel and Distributed Systems Laboratory Jozef Stefan Institute Ljubljana Slovenia Faculty of Mathematics and Physics University of Ljubljana Ljubljana Slovenia

Radial Basis Function-generated Finite Differences (RBF-FD) is a meshless method that can be used to numerically solve partial differential equations. The solution procedure consists of two steps. First, the differential operator is discretised on given scattered nodes and afterwards, a global sparse matrix is assembled and inverted to obtain an approximate solution. Focusing on Polyharmonic Splines as our Radial Basis Functions (RBFs) of choice, appropriately augmented with monomials, it is well known that the truncation error of the differential operator approximation is determined by the degree of monomial augmentation. Naively, one might think that the solution error will have the same order of convergence. We present a superconvergence result that shows otherwise - for some augmentation degrees, order of convergence is higher than expected. © Published under licence by IOP Publishing Ltd.

关键词： Finite difference method

来源：评论

学校读者我要写书评

暂无评论

Flexible application-aware approximation for modern distributed graph processing frameworks 5

Flexible application-aware approximation for modern distribu...

引用

5th ACM SIGMOD Joint International Workshop on Graph Data Management Experiences and systems and Network Data Analytics, GRADES-NDA 2022, co-located with ACM SIGMOD Conference

作者： Schramm, Michael Bhowmik, Sukanya Rothermel, Kurt Institute for Parallel and Distributed Systems University of Stuttgart Germany

ISBN: (纸本)9781450393843

The interest in the ability of processing data that has an underlying graph structure has grown in the recent past. This has led to the development of many distributed graph processing systems. However, due to rapidly growing amount of data, e.g., web graphs and social graphs, even such distributed graph processing frameworks end up requiring several minutes or even several hours to execute popular graph algorithms. This leads to the question: do we always need to know the exact answer for a large graph? The aforementioned modern distributed graph processing frameworks execute graph algorithms by exchanging messages between vertices. This paper introduces a novel message-dropping approach for approximation in these frameworks. As dropping messages would result in degradation of quality of result, our objective is to drop messages that have least adverse impact on quality. More specifically, we propose an application-aware approach that dynamically drops messages at runtime. We evaluate the effects of our approach for the PageRank algorithm on several representative real-world web graphs and compare its performance to that of existing approximation techniques for modern frameworks.. © 2022 ACM.

关键词： Drops

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：