检索结果-内蒙古大学图书馆

ProMoE: Fast MoE-based LLM Serving using Proactive Caching

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Song, Xiaoniu Zhong, Zihang Chen, Rong Chen, Haibo Institute of Parallel and Distributed Systems Shanghai Jiao Tong University China Key Laboratory of System Software Chinese Academy of Sciences China Zhejiang University China

The promising applications of large language models are often limited by the constrained GPU memory capacity available on edge devices. Mixture-of-Experts (MoE) models help address this issue by activating only a subset of the model’s parameters during computation. This approach allows the unused parameters to be offloaded to host memory, thereby reducing the overall GPU memory demand. However, existing cache-based offloading solutions handle cache misses reactively, which significantly impacts system performance. In this paper, we introduce ProMoE, a novel proactive caching system that utilizes intermediate results to predict subsequent expert usage. By proactively fetching experts in advance, ProMoE eliminates passive cache misses, removes loading time from the critical path, and reduces the performance overhead associated with offloading. Our evaluations demonstrate that ProMoE achieves an average speedup of 2.20× (up to 3.21×) and 2.07× (up to 5.02×) in the prefill and decode stages, respectively, compared to existing offloading solutions. © 2024, CC BY.

关键词： Graphics processing unit

Meshless method stencil evaluation with machine learning 45

学校读者我要写书评

暂无评论

Meshless method stencil evaluation with machine learning

45th Jubilee International Convention on Information, Communication and Electronic Technology, MIPRO 2022

作者： Rot, Miha Rashkovska, Aleksandra Jozef Stefan International Postgraduate School Jamova cesta 39 Ljubljana1000 Slovenia Jožef Stefan Institute Parallel and Distributed Systems Laboratory Jamova cesta 39 Ljubljana1000 Slovenia

ISBN: (纸本)9789532331035

Meshless methods are an active and modern branch of numerical analysis with many intriguing benefits. One of the main open research questions related to local meshless methods is how to select the best possible stencil - a collection of neighbouring nodes - to base the calculation on. In this paper, we describe the procedure for generating a labelled stencil dataset and use a variation of pointNet - a deep learning network based on point clouds - to create a classifier for the quality of the stencil. We exploit features of pointNet to implement a model that can be used to classify differently sized stencils and compare it against models dedicated to a single stencil size. The model is particularly good at detecting the best and the worst stencils with a respectable area under the curve (AUC) metric of around 0.90. There is much potential for further improvement and direct application in the meshless domain. © 2022 Croatian Society MIPRO.

关键词： Classification (of information)

Implicit-Explicit Error Indicator based on Approximation Order

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Jančič, Mitja Strniša, Filip Kosec, Gregor Parallel and Distributed Systems Laboratory Jožef Stefan Institute Jožef Stefan International Postgraduate School Ljubljana Slovenia Parallel and Distributed Systems Laboratory Jožef Stefan Institute Ljubljana Slovenia

With the immense computing power at our disposal, the numerical solution of partial differential equations (PDEs) is becoming a day-to-day task for modern computational scientists. However, the complexity of real-life problems is such that tractable solutions do not exist. This makes it difficult to validate the numerically obtained solution, so good error estimation is crucial in such cases. It allows the user to identify problematic areas in the computational domain that may affect the stability and accuracy of the numerical method. Such areas can then be remedied by either h-or p-adaptive procedures. In this paper, we propose to estimate the error of the numerical solution by solving the same governing problem implicitly and explicitly, using a different approximation order in each case. We demonstrate the newly proposed error indicator on the solution of a synthetic twodimensional Poisson problem with tractable solution for easier validation. We show that the proposed error indicator has good potential for locating areas of high error. Copyright © 2022, The Authors. All rights reserved.

关键词： Poisson equation

Accelerating Extra Dimensional Page Walks for Confidential Computing

学校读者我要写书评

暂无评论

Accelerating Extra Dimensional Page Walks for Confidential C...

IEEE/ACM International Symposium on Microarchitecture (MICRO)

作者： Dong Du Bicheng Yang Yubin Xia Haibo Chen Institute of Parallel and Distributed Systems Shanghai Jiao Tong University Engineering Research Center for Domain-Specific Operating Systems (MoE) Shanghai China Institute of Parallel and Distributed Systems Shanghai Jiao Tong University Shanghai AI Laboratory Engineering Research Center for Domain-Specific Operating Systems (MoE) Shanghai China

To support highly scalable and fine-grained computing paradigms such as microservices and serverless computing better, modern hardware-assisted confidential computing systems, such as Intel TDX and ARM CCA, introduce permission table to achieve fine-grained and scalable memory isolation among different domains. However, it also adds an extra dimension to page walks besides page tables, leading to significantly more memory references (e.g., 4 12 for RISC-V Sv39) 1 . We observe that most costs (about 75%) caused by the extra dimension of page walks are used to validate page table pages. Based on this observation, this paper proposes HPMP (Hybrid Physical Memory Protection), a hardware-software co-design (on RISC-V) that protects page table pages using segment registers and normal pages using permission tables to balance scalability and performance. We have implemented HPMP and Penglai-HPMP (a TEE system based on HPMP) on FPGA with two RISC-V cores (both in-order and out-of-order). Evaluation results show that HPMP can reduce costs by 23.1%–73.1% on BOOM and significantly improve performance on real-world applications, including serverless computing (FunctionBench) and Redis.

关键词：

Computational Electromagnetics with the RBF-FD Method

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Kolar-Požun, Andrej Kosec, Gregor Parallel and Distributed Systems Laboratory ’Jožef Stefan’ Institute Ljubljana Slovenia Faculty of Mathematics and Physics University of Ljubljana Ljubljana Slovenia

One of the most popular methods employed in computational electromagnetics is the Finite Difference Time Domain (FDTD) method. We generalise it to a meshless setting using the Radial Basis Function generated Finite Di... 详细信息

关键词： Computational electromagnetics

Transactional Indexes on (RDMA or CXL-based) Disaggregated Memory with Repairable Transaction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wei, Xingda Wang, Haotian Wang, Tianxia Chen, Rong Gu, Jinyu Zuo, Pengfei Chen, Haibo Institute of Parallel and Distributed Systems SEIEE Shanghai Jiao Tong University China Shanghai AI Laboratory China Huawei Cloud China

The failure atomic and isolated execution of clients operations is a default requirement for a system that serve multiple loosely coupled clients at a server. However, disaggregated memory breaks this requirement in remote indexes because a client operation is disaggregated to multiple remote reads/writes. Current indexes focus on performance improvements and largely ignore tolerating client failures. We argue that a practical DM index should be transactional: each index operation should be failure atomic and isolated in addition to being concurrency isolated. We present repairable transaction (rTX), a lightweight primitive to execute DM index operations. Each rTX can detect other failed rTXes on-the-fly with the help of concurrency control. Upon detection, it will repair their non-atomic updates online with the help of logging, thus hiding their failures from healthy clients. By further removing unnecessary logging and delegating concurrency control to existing carefully-tuned index algorithms, we show that transactional indexes can be built at a low performance overhead on disaggregated memory. We have refactored two state-of-the-art DM indexes, RaceHashing and Sherman (B+Tree), with rTX. Evaluations show that rTX is 1.2–2× faster than other alternatives, e.g., distributed transaction. Meanwhile, its overhead is up to 42% compared to non-fault-tolerant indexes. © 2023, CC BY.

关键词： Concurrency control

Computational Electromagnetics with the RBF-FD Method

学校读者我要写书评

暂无评论

Computational Electromagnetics with the RBF-FD Method

International Multidisciplinary Conference on Computer and Energy Science (SpliTech)

作者： Andrej Kolar-Požun Gregor Kosec Parallel and Distributed Systems Laboratory ’Jožef Stefan’ Institute Ljubljana Slovenia Faculty of Mathematics and Physics University of Ljubljana Ljubljana Slovenia

One of the most popular methods employed in computational electromagnetics is the Finite Difference Time Domain (FDTD) method. We introduce its generalisation to a meshless setting using the Radial Basis Function gene...

关键词：

A hybrid RBF-FD and WLS mesh-free strong-form approximation method 7

学校读者我要写书评

暂无评论

A hybrid RBF-FD and WLS mesh-free strong-form approximation ...

7th International Conference on Smart and Sustainable Technologies, SpliTech 2022

作者： Jancic, Mitja Kosec, Gregor JoÅ½ef Stefan Institute Parallel and Distributed Systems Laboratory Jamova cesta 39 Ljubljana1000 Slovenia JoÅ½ef Stefan International Postgraduate School Jamova cesta 39 Ljubljana1000 Slovenia

ISBN: (纸本)9789532901160

Since the advent of mesh-free methods as a tool for the numerical analysis of systems of Partial Differential Equations (PDEs), many variants of differential operator approximation have been proposed. In this work, we propose a local mesh-free strong-form method that combines the stability of Radial Basis Function-Generated Finite Differences (RBF-FD) with the computational effectiveness of Diffuse Approximation Method (DAM), forming a so-called hybrid method. To demonstrate the advantages of a hybrid method, we evaluate its computational complexity and accuracy of the obtained numerical solution by solving a two-dimensional Poisson problem with an exponentially strong source in the computational domain. Finally, we employ the hybrid method to solve a three-dimensional Boussinesq's problem on an isotropic half-space and show that the implementation overhead can be justified. © 2022 University of Split, FESB.

关键词： Mesh generation

Spatially-varying meshless approximation method for enhanced computational efficiency

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Jančič, Mitja Rot, Miha Kosec, Gregor Jožef Stefan Institute Parallel and Distributed Systems Laboratory Ljubljana Slovenia Jožef Stefan International Postgraduate School Ljubljana Slovenia

In this paper, we address a way to reduce the total computational cost of meshless approximation by reducing the required stencil size through spatially varying computational node regularity. Rather than covering the entire domain with scattered nodes, only regions with geometric details are covered with scattered nodes, while the rest of the domain is discretized with regular nodes. A simpler approximation using solely monomial basis can be used in regions covered by regular nodes, effectively reducing the required stencil size and computational cost compared to the approximation on scattered nodes where a set of polyharmonic splines is added to ensure convergent behaviour. The performance of the proposed hybrid scattered-regular approximation approach, in terms of computational efficiency and accuracy of the numerical solution, is studied on natural convection driven fluid flow problems. We start with the solution of the de Vahl Davis benchmark case, defined on a square domain, and continue with two- and three-dimensional irregularly shaped domains. We show that the spatial variation of the two approximation methods can significantly reduce the computational demands, with only a minor impact on the accuracy. Copyright © 2023, The Authors. All rights reserved.

关键词： Computational efficiency