检索结果-内蒙古大学图书馆

Serverless execution of scientific workflows: Experiments with HyperFlow, AWS Lambda and Google cloud functions

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2020年 110卷 502-514页

作者： Malawski, Maciej Gajek, Adam Zima, Adam Balis, Bartosz Figiela, Kamil AGH Univ Sci & Technol Dept Comp Sci Krakow Poland

Scientific workflows consisting of a high number of interdependent tasks represent an important class of complex scientific applications. Recently, a new type of serverless infrastructures has emerged, represented by such services as Google cloud functions and AWS Lambda, also referred to as the Function-as-a-Service model. In this paper we take a look at such serverless infrastructures, which are designed mainly for processing background tasks of Web and Internet of Things applications, or event-driven stream processing. We evaluate their applicability to more compute- and data-intensive scientific workflows and discuss possible ways to repurpose serverless architectures for execution of scientific workflows. We have developed prototype workflow executor functions using AWS Lambda and Google cloud functions, coupled with the HyperFlow workflow engine. These functions can run workflow tasks in AWS and Google infrastructures, and feature such capabilities as data staging to/from S3 or Google cloud Storage and execution of custom application binaries. We have successfully deployed and executed the Montage astronomy workflow, often used as a benchmark, and we report on initial results of its performance evaluation. Our findings indicate that the simple mode of operation makes this approach easy to use, although there are costs involved in preparing portable application binaries for execution in a remote environment. While our solution is an early prototype, we find the presented approach highly promising. We also discuss possible future steps related to execution of scientific workflows in serverless infrastructures. Finally, we perform a cost analysis and discuss implications with regard to resource management for scientific applications in general. (C) 2017 Elsevier B.V. All rights reserved.

关键词： Scientific workflows cloud functions Serverless architectures FaaS

来源：评论

学校读者我要写书评

暂无评论

Challenges for Scheduling Scientific Workflows on cloud functions 11

Challenges for Scheduling Scientific Workflows on Cloud Func...

引用

11th IEEE International Conference on cloud Computing (cloud) Part of the IEEE World Congress on Services

作者： Kijak, Joanna Martyna, Piotr Pawlik, Maciej Balis, Bartosz Malawski, Maciej AGH Univ Sci & Technol Dept Comp Sci Krakow Poland

ISBN: (纸本)9781538672358

Serverless computing, also known as Function-asa- Service (FaaS) or cloud functions, is a new method of running distributed applications by executing functions on the infrastructure of cloud providers. Although it frees the developers from managing servers, there are still decisions to be made regarding selection of function configurations based on the desired performance and cost. The billing model of this approach considers time of execution, measured in 100ms units, as well as the size of the memory allocated per function. In this paper, we look into the problem of scheduling scientific workflows, which are applications consisting of multiple tasks connected into a dependency graph. We discuss challenges related to workflow scheduling and propose the Serverless Deadline-Budget Workflow Scheduling (SDBWS) algorithm adapted to serverless platforms. We present preliminary experiments with a small-scale Montage workflow run on the AWS Lambda infrastructure.

关键词： FaaS serverless computing cloud functions scientific workflow task scheduling

来源：评论

学校读者我要写书评

暂无评论

Benchmarking Heterogeneous cloud functions 1

引用

23rd International Conference on Parallel and Distributed Computing (Euro-Par)

作者： Malawski, Maciej Figiela, Kamil Gajek, Adam Zima, Adam AGH Univ Sci & Technol Dept Comp Sci Krakow Poland

ISBN: (数字)9783319751788

ISBN: (纸本)9783319751788;9783319751771

cloud functions, often called Function-as-a-Service (FaaS), pioneered by AWS Lambda, are an increasingly popular method of running distributed applications. As in other cloud offerings, cloud functions are heterogeneous, due to different underlying hardware, run-time systems, as well as resource management and billing models. In this paper, we focus on performance evaluation of cloud functions, taking into account heterogeneity aspects. We developed a cloud function benchmarking framework, consisting of one suite based on Serverless Framework, and one based on HyperFlow. We deployed the CPU-intensive benchmarks: Mersenne Twister and Linpack, and evaluated all the major cloud function providers: AWS Lambda, Azure functions, Google cloud functions and IBM OpenWhisk. We make our results available online and continuously updated. We report on the initial results of the performance evaluation and we discuss the discovered insights on the resource allocation policies.

关键词： cloud computing FaaS cloud functions Performance evaluation

来源：评论

学校读者我要写书评

暂无评论

Vexless: A Serverless Vector Data Management System Using cloud functions

引用

Proceedings of the ACM on Management of Data 2024年第3期2卷 1-26页

作者： Yongye Su Yinqi Sun Minjia Zhang Jianguo Wang Purdue University West Lafayette IN USA University of Illinois Urbana-Champaign Urbana IL USA

cloud functions, exemplified by AWS Lambda and Azure functions, are emerging as a new computing paradigm in the cloud. They provide elastic, serverless, and low-cost cloud computing, making them highly suitable for bursty and sparse workloads, which are quite common in practice. Thus, there is a new trend in designing data systems that leverage cloud functions. In this paper, we focus on vector databases, which have recently gained significant attention partly due to large language models. In particular, we investigate how to use cloud functions to build high-performance and cost-efficient vector databases. This presents significant challenges in terms of how to perform sharding, how to reduce communication overhead, and how to minimize cold-start *** this paper, we introduce Vexless, the first vector database system optimized for cloud functions. We present three optimizations to address the challenges. To perform sharding, we propose a global coordinator (orchestrator) that assigns workloads to cloud function instances based on their available hardware resources. To overcome communication overhead, we propose the use of stateful cloud functions, eliminating the need for costly communications during synchronization. To minimize cold-start overhead, we introduce a workload-aware cloud function lifetime management strategy. Vexless has been implemented using Azure functions. Experimental results demonstrate that Vexless can significantly reduce costs, especially on bursty and sparse workloads, compared to cloud VM instances, while achieving similar or higher query performance and accuracy.

关键词： cloud functions serverless computing serverless databases vector databases

来源：评论

学校读者我要写书评

暂无评论

Performance evaluation of heterogeneous cloud functions

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2018年第23期30卷 e4792.1-e4792.16页

作者： Figiela, Kamil Gajek, Adam Zima, Adam Obrok, Beata Malawski, Maciej AGH Univ Sci & Technol Dept Comp Sci Krakow Poland

cloud functions, often called Function-as-a-Service (FaaS), pioneered by AWS Lambda, are an increasingly popular method of running distributed applications. As in other cloud offerings, cloud functions are heterogeneous due to variations in underlying hardware, runtime systems, as well as resource management and billing models. In this paper, we focus on performance evaluation of cloud functions, taking into account heterogeneity aspects. We developed a cloud function benchmarking framework, consisting of one suite based on Serverless Framework and one based on HyperFlow. We deployed the CPU-intensive benchmarks: Mersenne Twister and Linpack. We measured the data transfer times between cloud functions and storage, and we measured the lifetime of the runtime environment. We evaluated all the major cloud function providers: AWS Lambda, Azure functions, Google cloud functions, and IBM cloud functions. We made our results available online and continuously updated. We report on the results of the performance evaluation, and we discuss the discovered insights into resource allocation policies.

关键词： cloud computing cloud functions FaaS performance evaluation serverless

来源：评论

学校读者我要写书评

暂无评论

CSPOT: Portable, Multi-scale functions-as-a-Service for IoT 19

CSPOT: Portable, Multi-scale Functions-as-a-Service for IoT

引用

4th ACM/IEEE Symposium on Edge Computing (SEC)

作者： Wolski, Rich Krintz, Chandra Bakir, Fatih George, Gareth Lin, Wei-Tsung Univ Calif Santa Barbara Dept Comp Sci Santa Barbara CA 93106 USA

ISBN: (纸本)9781450367332

In this paper, we present CSPOT, a distributed runtime system implementing a functions-as-service (FaaS) programming model for the "Internet of Things" (IoT). With FaaS, developers express arbitrary computations as simple functions that are automatically invoked and managed by a cloud platform in response to events. We extend this FaaS model so that it is suitable for use in all tiers of scale for IoT - sensors, edge devices, and cloud - to facilitate robust, portable, and low-latency IoT application development and deployment. To enable this, we combine the use of Linux containers and namespaces for isolation and portability, an append-only object store for robust persistence, and a causal event log for triggering functions and tracking event dependencies. We present the design and implementation of CSPOT, detail its abstractions and APIs, and overview examples of its use. We empirically evaluate the performance of CSPOT using different devices and applications and find that it implements function invocation with significantly lower latency than other FaaS offerings, while providing portability across tiers and similar data durability characteristics.

关键词： IoT serverless cloud functions append-only portability distributed systems

来源：评论

学校读者我要写书评

暂无评论

NanoLambda: Implementing functions as a Service at All Resource Scales for the Internet of Things. 5

NanoLambda: Implementing Functions as a Service at All Resou...

引用

5th IEEE/ACM Symposium on Edge Computing (SEC)

作者： George, Gareth Bakir, Fatih Wolski, Rich Krintz, Chandra Univ Calif Santa Barbara Dept Comp Sci Santa Barbara CA 93106 USA

ISBN: (纸本)9781728159430

Internet of Things (IoT) devices are becoming increasingly prevalent in our environment, yet the process of programming these devices and processing the data they produce remains difficult. Typically, data is processed on device, involving arduous work in low level languages, or data is moved to the cloud, where abundant resources are available for functions as a Service (FaaS) or other handlers. FaaS is an emerging category of flexible computing services, where developers deploy self-contained functions to be run in portable and secure containerized environments;however, at the moment, these functions are limited to running in the cloud or in some cases at the "edge" of the network using resource rich, Linux-based systems. In this paper, we present NanoLambda, a portable platform that brings FaaS, high-level language programming, and familiar cloud service APIs to non-Linux and microcontroller-based IoT devices. To enable this, NanoLambda couples a new, minimal Python runtime system that we have designed for the least capable end of the IoT device spectrum, with API compatibility for AWS Lambda and S3. NanoLambda transfers functions between IoT devices (sensors, edge, cloud), providing power and latency savings while retaining the programmer productivity benefits of high-level languages and FaaS. A key feature of NanoLambda is a scheduler that intelligently places function executions across multi-scale IoT deployments according to resource availability and power constraints. We evaluate a range of applications that use NanoLambda to run on devices as small as the ESP8266 with 64KB of ram and 512KB flash storage.

关键词： IoT serverless cloud functions edge computing microcontrollers portability

来源：评论

学校读者我要写书评

暂无评论

I/O resource isolation of public cloud serverless function runtimes for data-intensive applications

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2020年第3期23卷 2249-2259页

作者： Kim, Jeongchul Lee, Kyungyong Kookmin Univ Dept Comp Sci Seoul South Korea

Serverless computing and a function execution model, Function-as-a-Service (FaaS), are currently receiving considerable attention from both academia and industry. One of the reasons for the success of serverless computing is its straightforward interface that abstracts complex internals of cloud computing resource usage and configurations. However, this approach may result in hiding too much information about how underlying cloud resources would work, entailing that users cannot predict how their applications will perform, especially for IO-heavy ones. To address this issue, we evaluate several aspects of network and disk IO performance with realistic workloads using public FaaS systems. Our analysis reveals that current public FaaS systems do not provide appropriate levels of IO performance differentiation, and the ability to isolate network resource allocation during concurrent execution is rarely offered by service providers. Based on the results presented in this paper, we insist that it must be mandatory for network and disk IO resource performance of FaaS to be more visible and predictable, as is the case for memory and CPU, in order to expand serverless computing applications to data-intensive ones.

关键词： FaaS Serverless computing Resource isolation cloud functions Data-intensive application I O resource

来源：评论

学校读者我要写书评

暂无评论

Edge-adaptable serverless acceleration for machine learning Internet of Things applications

引用

SOFTWARE-PRACTICE & EXPERIENCE 2021年第9期51卷 1852-1867页

作者： Zhang, Michael Krintz, Chandra Wolski, Rich Univ Calif Santa Barbara Dept Comp Sci Santa Barbara CA 93106 USA

Serverless computing is an emerging event-driven programming model that accelerates the development and deployment of scalable web services on cloud computing systems. Though widely integrated with the public cloud, serverless computing use is nascent for edge-based, Internet of Things (IoT) deployments. In this work, we present STOIC (serverless teleoperable hybrid cloud), an IoT application deployment and offloading system that extends the serverless model in three ways. First, STOIC adopts a dynamic feedback control mechanism to precisely predict latency and dispatch workloads uniformly across edge and cloud systems using a distributed serverless framework. Second, STOIC leverages hardware acceleration (e.g., GPU resources) for serverless function execution when available from the underlying cloud system. Third, STOIC can be configured in multiple ways to overcome deployment variability associated with public cloud use. We overview the design and implementation of STOIC and empirically evaluate it using real-world machine learning applications and multitier IoT deployments (edge and cloud). Specifically, we show that STOIC can be used for training image processing workloads (for object recognition)-once thought too resource-intensive for edge deployments. We find that STOIC reduces overall execution time (response latency) and achieves placement accuracy that ranges from 92% to 97%.

关键词： cloud functions GPU IoT Scheduling Serverless

来源：评论

学校读者我要写书评

暂无评论

SplitServe: Efficiently Splitting Apache Spark Jobs Across FaaS and IaaS 20

SplitServe: Efficiently Splitting Apache Spark Jobs Across F...

引用

21st International Middleware Conference (Middleware)

作者： Jain, Aman Baarzi, Ata F. Kesidis, George Urgaonkar, Bhuvan Alfares, Nader Kandemir, Mahmut Penn State Univ University Pk PA 16802 USA

ISBN: (纸本)9781450381536

Due to their lower startup latencies and finer-grain pricing than virtual machines (VMs), Amazon Lambdas and other cloud functions (CFs) have been identified as ideal candidates for handling unexpected spikes in simple, stateless workloads. However, it is not immediately clear if CFs would be similarly effective in autoscaling complex workloads involving significant state transfer across distributed application components. We have found that, through careful design, currently available CFs can indeed be useful even for complex workloads. To demonstrate this, we design and implement SplitServe, an enhancement of Apache Spark. If not enough executors on existing VMs are available for a newly arriving latency-sensitive job, SplitServe is able to use CFs to quickly bridge this shortfall in VMs, so avoiding the startup latencies of newly requested VMs. If desirable in terms of performance or cost, when newly requested VMs, or executors on existing VMs, do become available, SplitServe is able to move ongoing work from CFs to them. Our experimental evaluation of SplitServe using four different workloads (either on a mixture of VM-based executors and CFs or just CFs) shows that it improves execution time by up to (a) 55% for workloads with small to modest amount of shuffling, and (b) 31% in workloads with large amounts of shuffling, when compared to only VM-based autoscaling.

关键词： cloud Computing cloud functions Virtual Machines Apache Spark

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：