检索结果-内蒙古大学图书馆

PipeLLM: Fast and Confidential Large Language Model Services with Speculative Pipelined Encryption 25

学校读者我要写书评

暂无评论

PipeLLM: Fast and Confidential Large Language Model Services...

30th ACM International Conference on Architectural Support for Programming Languages and Operating systems, ASPLOS 2025

作者： Tan, Yifan Tan, Cheng Mi, Zeyu Chen, Haibo Institute of Parallel and Distributed Systems Seiee Shanghai Jiao Tong University Shanghai China Northeastern University Boston United States

ISBN: (纸本)9798400706981

Confidential computing on GPUs, like NVIDIA H100, mitigates the security risks of outsourced Large Language Models (LLMs) by implementing strong isolation and data encryption. Nonetheless, this encryption incurs a significant performance overhead, reaching up to 52.8% and 88.2% throughput drop when serving OPT-30B and OPT-66B, respectively. To address this challenge, we introduce PipeLLM, a user-transparent runtime system. PipeLLM removes the overhead by overlapping the encryption and GPU computation through pipelining-an idea inspired by the CPU instruction pipelining-thereby effectively concealing the latency increase caused by encryption. The primary technical challenge is that, unlike CPUs, the encryption module lacks prior knowledge of the specific data needing encryption until it is requested by the GPUs. To this end, we propose speculative pipelined encryption to predict the data requiring encryption by analyzing the serving patterns of LLMs. Further, we have developed an efficient, low-cost pipeline relinquishing approach for instances of incorrect predictions. Our experiments show that compared with vanilla systems without confidential computing (e.g., vLLM, PEFT, and FlexGen), PipeLLM incurs modest overhead ( © 2025 ACM.

关键词： Modeling languages

A Hardware-Software Co-Design for Efficient Secure Containers 25

学校读者我要写书评

暂无评论

A Hardware-Software Co-Design for Efficient Secure Container...

20th European Conference on Computer systems, EuroSys 2025, co-located 30th ACM International Conference on Architectural Support for Programming Languages and Operating systems, ASPLOS 2025

作者： Shi, Jiacheng Yu, Yang Gu, Jinyu Xia, Yubin Institute of Parallel and Distributed Systems SEIEE Shanghai Jiao Tong University Engineering Research Center for Domain-specific Operating Systems Ministry of Education China

ISBN: (纸本)9798400711961

VM-level containers provide strong isolation by running each container with its own kernel in a VM. However, they rely on virtualization hardware designed for general-purpose VMs, causing non-negligible performance overhead compared to OS-level containers. This performance gap widens dramatically in nested virtualization scenarios, where secure containers run inside a VM. This paper proposes CKI (Container Kernel Isolation), a hardware-software co-design for efficient secure containers, based on two insights. First, Protection Keys for Supervisor (PKS) facilitates constructing a new privilege level for securely collocating multiple container kernels within the host kernel, without involving non-root ring-0. Second, the general-purpose virtualization mechanisms used by secure containers provide unnecessary features that exceed the actual isolation requirements of containers, especially the two-stage address translation, which is not required for container isolation, introducing avoidable performance overhead. Thus, CKI avoids using the virtualization hardware for running container kernels and removes the unnecessary virtualization mechanism like two-stage address translation. Instead, it uses PKS to construct a new privilege level for isolating multiple container kernels, which allows more efficient cross-privilege interaction;it also uses single-stage address translation for each container while monitoring the page table updates in a lightweight way to ensure cross-container memory isolation. Our experiments on real-world applications demonstrate the efficiency of CKI, reducing the latency of memory-intensive applications by up to 72% and 47% compared with state-of-the-art hardware-assisted virtualization (HVM) and software-based virtualization (PVM), respectively. © 2025 Copyright held by the owner/author(s).

关键词： Virtualization

Wireless-Aware TSN Engineering: Implications for 5G and Upcoming 6G Networks

学校读者我要写书评

暂无评论

IEEE Network 2025年第3期39卷 99-107页

作者： Egger, Simon Durr, Frank Varga, Balazs De Andrade, Marilet Sharma, Gourav Prateek Sachs, Joachim Harmatos, Janos Gross, James University of Stuttgart Institute for Parallel and Distributed Systems Stuttgart70569 Germany Ericsson Research Budapest1117 Hungary Ericsson Research Stockholm16440 Sweden KTH Royal Institute of Technology School of Electrical Engineering and Computer Science Stockholm114 28 Sweden Ericsson Research Aachen52134 Germany

The importance of time- and mission-critical applications is increasing as industries and society are advancing in their digitalization. As such applications introduce stringent quality of service (QoS) requirements on the underlying network infrastructure, open standards like time-sensitive networking (TSN) and DetNet are being developed to provide dependable and time-sensitive communication in bridged LAN networks and IP networks, respectively. More recently, the extension to support wireless time-sensitive connectivity has received special interest due to improved network deployment flexibility with mobile devices. These efforts have led to the standardized support of TSN/DetNet in 5G, allowing the 5G system and its mobile devices to be integrated transparently into TSN/DetNet networks while conforming to their user and control plane protocols. Nevertheless, the induced packet delays– and in particular the packet delay variations (PDVs)– of 5G are significantly larger than in their wired counterparts. In this work, we illustrate that established methods to configure TSN/DetNet networks insufficiently address major runtime uncertainties, and that they would consequently result in QoS impairments or scalability issues. We therefore advocate for two systematic approaches to improve traffic shaping in this setting: first, we propose Packet Delay Correction where the PDV is corrected within the 5G system. Second, we introduce "wireless-aware" TSN scheduling to account for the packet delay characteristics of 5G. We demonstrate that wireless-aware TSN scheduling and Packet Delay Correction are complementary in their design and can be combined to provide formal end-to-end reliability guarantees at scale. In addition, being a key aspect to support wireless-aware TSN scheduling in 5G and future 6G networks, we introduce a data-driven framework to predict the packet delay characteristics of the wireless systems over a finite time horizon. Finally, we identify open research

关键词： Quality of service

A HPX Communication Benchmark: distributed FFT using Collectives

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Strack, Alexander Pflüger, Dirk Institute of Parallel and Distributed Systems University of Stuttgart Stuttgart70569 Germany

Due to increasing core counts in modern processors, several task-based runtimes emerged, including the C++ Standard Library for Concurrency and parallelism (HPX). Although the asynchronous many-task runtime HPX allows implicit communication via an Active Global Address Space, it also supports explicit collective operations. Collectives are an efficient way to realize complex communication patterns. In this work, we benchmark the TCP, MPI, and LCI communication backends of HPX, which are called parcelports in HPX terms. We use a distributed multi-dimensional FFT application relying on collectives. Furthermore, we compare the performance of the HPX all-to-all and scatter collectives against an FFTW3 reference based on MPI+X on a 16-node cluster. Of the three parcelports, LCI performed best for both scatter and all-to-all collectives. Furthermore, the LCI parcelport was up to factor 3 faster than the MPI+X reference. Our results highlight the potential of message abstractions and the parcelports of HPX. Copyright © 2025, The Authors. All rights reserved.

关键词： C++ (programming language)

Proximity-based service discovery for distributed digital twin systems

学校读者我要写书评

暂无评论

Discover Internet of Things 2025年第1期5卷 1-24页

作者： Rothermel, Kurt Herzog, Otthein Zhiqiang, Siegfried Wu Institute Of Parallel and Distributed Systems University of Stuttgart Universitaetsstr. 38 Stuttgart70569 Germany College of Architecture and Urban Planning Tongji University 1239 Siping Road Shanghai200092 China Department of Mathematics and Computer Science University of Bremen Bibliothekstrasse 5 Bremen23884 Germany

Over the past decade, there has been a significant increase in interest in digital twin (DT) technology in a variety of domains. While research on DTs of single assets was initially prevalent, there has been a notable shift towards distributed systems of DTs, which connect to each other to collaborate. Typically, collaboration is enabled by DTs providing services that can be consumed by other DTs. In service-oriented systems, a service is typically identified by type information. However, this is not sufficient in distributed DT systems, where DTs associated with different physical entities may provide the same type of service. Consequently, selecting the appropriate service depends not only on the service type, but also on the associated physical entity. However, requiring DTs to know the mapping of services to their physical environment is not feasible for large dynamic systems. This paper presents a novel proximity-based service discovery method that allows DTs to select services based on service type and their proximity to other objects. That is, service specifications are fully abstracted from the mapping of services to physical objects, relieving DTs from maintaining information about this mapping. Furthermore, service discovery is robust to changes in the physical environment and service population. The proposed service discovery method has been implemented on top of a spatial DBMS. We argue that this implementation is optimal in terms of network utilization and latency, and perform comprehensive evaluations to show the performance of discovery queries as a function of their complexity. © The Author(s) 2025.

关键词： Location based services

Accelerating Large Language Models through Partially Linear Feed-Forward Network

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Hu, Gansen Wang, Zhaoguo Wei, Jinglin Huang, Wei Chen, Haibo Institute of Parallel and Distributed Systems SEIEE Shanghai Jiao Tong University China

Large language models (LLMs) demonstrate remarkable capabilities but face deployment challenges due to their massive parameter counts. While existing compression techniques like pruning can reduce model size, it leads to significant accuracy degradation under high compression ratios. We present a novel perspective inspired by constant folding in compiler optimization. Our approach enables parameter reduction by treating activation functions in LLMs as linear functions. However, recent LLMs use complex non-linear activations like GELU that prevent direct application of this technique. We propose TARDIS, which enables optimization of LLMs with non-linear activations by partially approximating them with linear functions in frequently occurring input ranges. For outlier inputs, TARDIS employs an online predictor to dynamically fall back to original computations. Our experiments demonstrate that TARDIS achieves 80% parameter reduction in feed-forward networks, while significantly outperforming state-of-the-art pruning methods Wanda and RIA with up to 65% higher accuracy. In practical deployments for a 7B model, TARDIS achieves 1.6× end-to-end inference speedup when integrated with the vLLM serving system, and 1.4× speedup with the widely adopted HuggingFace implementation, while incurring only a 10.9% accuracy trade-off. © 2025, CC BY.

关键词： Program compilers

Harmonizing Security and Performance in Microkernel File Servers

学校读者我要写书评

暂无评论

Journal of Computer Science and Technology 2025年第2期40卷 464-481页

作者： Li, Wen-Tai Wang, Zi-Xuan Gu, Jin-Yu Xia, Yu-Bin Zang, Bin-Yu Institute of Parallel and Distributed Systems School of Electronic Information and Electrical Engineering Shanghai Jiao Tong University Shanghai200240 China

Microkernel OSes separate OS functionalities, including file systems and device drivers, into different user-level services, which mitigates the problem of lacking isolation in monolithic OSes. Nevertheless, from the perspective of applications, compromised services may still threaten applications’ security. Specifically, attackers can utilize vulnerabilities in file systems and disk drivers to leak or manipulate applications’ file content. The key problem is that de-privileging OS services from the kernel level to the user level does not mean the reduction of applications’ trusted computing base (TCB), and applications still need to trust all the required system services. This paper shows a case for providing the file service to applications with minimum TCB on microkernel OSes. Observing that file services actually do not need to access concrete file content, we propose a mechanism named Mirage, which deprives their privilege of accessing file content while preserving their management capability. Mirage efficiently protects the confidentiality and integrity of application files from untrusted services. The evaluation demonstrates that Mirage outperforms an encryption-based mechanism by up to 128% for IO-intensive workloads. © institute of Computing Technology, Chinese Academy of Sciences 2025.

关键词： Carrier sense multiple access

PipeWeaver: Addressing Data Dynamicity in Large Multimodal Model Training with Dynamic Interleaved Pipeline

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Xue, Zhenliang Hu, Hanpeng Chen, Xing Jiang, Yimin Song, Yixin Mi, Zeyu Zhu, Yibo Jiang, Daxin Xia, Yubin Chen, Haibo Institute of Parallel and Distributed Systems Shanghai Jiao Tong University China StepFun Zenergize AI

Large multimodal models (LMMs) have demonstrated excellent capabilities in both understanding and generation tasks with various modalities. While these models can accept flexible combinations of input data, their training efficiency suffers from two major issues: pipeline stage imbalance caused by heterogeneous model architectures, and training data dynamicity stemming from the diversity of multimodal data. In this paper, we present PipeWeaver, a dynamic pipeline scheduling framework designed for LMM training. The core of PipeWeaver is dynamic interleaved pipeline, which searches for pipeline schedules dynamically tailored to current training batches. PipeWeaver addresses issues of LMM training with two techniques: adaptive modality-aware partitioning and efficient pipeline schedule search within a hierarchical schedule space. Meanwhile, PipeWeaver utilizes SEMU (Step Emulator), a training simulator for multimodal models, for accurate performance estimations, accelerated by spatial-temporal subgraph reuse to improve search efficiency. Experiments show that PipeWeaver can enhance LMM training efficiency by up to 97.3% compared to state-of-the-art systems, and demonstrate excellent adaptivity to LMM training’s data dynamicity. © 2025, CC BY-SA.

关键词：

HeteroLLM: Accelerating Large Language Model Inference on Mobile SoCs platform with Heterogeneous AI Accelerators

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Le Feng, Dahu Feng, Erhu Zhao, Rong Wang, Yingrui Xia, Yubin Chen, Haibo Xu, Pinjie Institute of Parallel and Distributed Systems Shanghai Jiao Tong University China Tsinghua University China SenseTime Research

With the rapid advancement of artificial intelligence technologies such as ChatGPT, AI agents and video generation, contemporary mobile systems have begun integrating these AI capabilities on local devices to enhance privacy and reduce response latency. To meet the computational demands of AI tasks, current mobile SoCs are equipped with diverse AI accelerators, including GPUs and Neural Processing Units (NPUs). However, there has not been a comprehensive characterization of these heterogeneous processors, and existing designs typically only leverage a single AI accelerator for LLM inference, leading to suboptimal use of computational resources and memory bandwidth. In this paper, we first summarize key performance characteristics of mobile SoC, including heterogeneous processors, unified memory, synchronization, etc. Drawing on these observations, we propose different tensor partition strategies to fulfill the distinct requirements of the prefill and decoding phases. We further design a fast synchronization mechanism that leverages the unified memory address provided by mobile SoCs. By employing these techniques, we present HeteroLLM, the fastest LLM inference engine in mobile devices which supports both layer-level and tensor-level heterogeneous execution. Evaluation results show that HeteroLLM achieves 9.99× and 4.36× performance improvement over other mobile-side LLM inference engines: MLC and MNN. © 2025, CC BY.

关键词： Tensors