The proceedings contain 13 papers. The special focus in this conference is on Coordination models and languages. The topics include: Formal Methods for Socio-technical Security: (Formal and Automated Analysis of ...
ISBN:
(纸本)9783031081453
The proceedings contain 13 papers. The special focus in this conference is on Coordination models and languages. The topics include: Formal Methods for Socio-technical Security: (Formal and Automated Analysis of Security Ceremonies);soft Concurrent Constraint programming with Local Variables;a Synthesis Tool for Optimal Monitors in a Branching-Time Setting;A Monitoring Tool for Linear-Time μ HML;Model-Driven Generation of Microservice Interfaces: From LEMMA Domain models to Jolie APIs;MIMOS: A Deterministic Model for the Design and Update of Real-Time Systems;A Sound Up-to- n, δ Bisimilarity for PCTL;Extensible 3D Simulation of Aggregated Systems with FCPP;towards Reinforcement Learning-based Aggregate computing;sibilla: A Tool for Reasoning about Collective Systems;space-Fluid Adaptive Sampling: A Field-Based, Self-organising Approach;formal Choreographic languages.
Presented in this paper are the outcomes from the evaluation of a distributed aircraft design environment, based on microservices and cloud computing. The evaluation was performed on a representative airframe-engine o...
详细信息
ISBN:
(数字)9781624106996
ISBN:
(纸本)9781624106996
Presented in this paper are the outcomes from the evaluation of a distributed aircraft design environment, based on microservices and cloud computing. The evaluation was performed on a representative airframe-engine optimization case study, including the engine, wing aero-structural geometry, and high-lift devices. The (computational) design process involved multiple distributed design teams and design tools. The latter were implemented with different programminglanguages and deployed on the Azure cloud service. As a benchmark, the same case study was performed using the traditional email/document-based approach to design collaboration. Compared with the traditional collaboration, the cloud-based approach substantially reduced the time for design iterations between the design teams. This was mainly due to the fast remote access of models/tools on the cloud and automation of data exchange. Also, the exercise indicated that the cloud-based approach is more flexible with regard to orchestrating the computational workflows and optimization studies, while protecting the Intellectual Property (IP) of the collaborating partners.
Fine-tuning on cheap commodity GPU servers makes large-scale deep learning models benefit more people. However, the low interGPU communication bandwidth and pressing communication contention on the commodity GPU serve...
详细信息
ISBN:
(纸本)9781450399166
Fine-tuning on cheap commodity GPU servers makes large-scale deep learning models benefit more people. However, the low interGPU communication bandwidth and pressing communication contention on the commodity GPU server obstruct training efficiency. In this paper, we present Mobius, a communication-efficient system for fine tuning large-scale models on commodity GPU servers. The key idea is a novel pipeline parallelism scheme enabling heterogeneous memory for large-scale model training, while bringing fewer communications than existing systems. Mobius partitions the model into stages and carefully schedules them between GPU memory and DRAM to overlap communication with computation. It formulates pipeline execution into a mixed-integer program problem to find the optimal pipeline partition. It also features a new stage-to-GPU mapping method termed cross mapping, to minimize communication contention. Experiments on various scale models and GPU topologies show that Mobius significantly reduces the training time by 3.8-5.1x compared with the prior art.
In this paper, we present new software for modeling distributed systems using the Petri net apparatus. Described main capabilities of software as follows: modeling extended Petri net models, such as nets with inhibito...
详细信息
This paper proposes ElasticFlow, an elastic serverless training platform for distributed deep learning. ElasticFlow provides a serverless interface with two distinct features: (i) users specify only the deep neural ne...
详细信息
ISBN:
(纸本)9781450399166
This paper proposes ElasticFlow, an elastic serverless training platform for distributed deep learning. ElasticFlow provides a serverless interface with two distinct features: (i) users specify only the deep neural network (DNN) model and hyperparameters for a job, but not the number of GPUs;(ii) users specify the deadline for a job, but not the amount of time to occupy GPUs. In contrast to existing servercentric platforms, ElasticFlow provides performance guarantees in terms of meeting deadlines while alleviating tedious, low-level, and manual resource management for deep learning developers. The characteristics of distributed training introduce two challenges. First, the training throughput scales non-linearly with the number of GPUs. Second, the scaling efficiency is affected byworker placement. To address these challenges, we propose Minimum Satisfactory Share to capture the resource usage of training jobs to meet deadlines, and ElasticFlow performs admission control based on it. We develop a greedy algorithm that dynamically allocates resources to admitted jobs based on diminishing returns. We apply buddy allocation to worker placement to eliminate the effect of topology. Evaluation results on a cluster of 128 GPUs show that ElasticFlow increases the number of jobs that can meet their deadlines by 1.46-7.65x compared to existing solutions.
Jupyter is a web-based, interactive computing environment that supports many commonly-used programminglanguages. It has been widely adopted in the CS education community and is now rapidly expanding to other STEM dis...
详细信息
ISBN:
(纸本)9798400705328
Jupyter is a web-based, interactive computing environment that supports many commonly-used programminglanguages. It has been widely adopted in the CS education community and is now rapidly expanding to other STEM disciplines due to the growing integration of programming in STEM education. However, unlike other educational platforms, there is currently no integrated way to capture, analyze, and visualize student interaction data in Jupyter notebooks. This means that teachers have limited to no visibility into student activity, preventing them from drawing insights from these data and providing timely interventions on the fly. In this paper, we present Jupyter Analytics, an end-to-end solution for teachers to collect, analyze, and visualize both synchronous and asynchronous learning activities in Jupyter. The Jupyter Analytics system consists of two JupyterLab extensions connected via a cloud-based backend. On the student side, we introduce the Jupyter Analytics Telemetry extension to anonymously capture students' interaction activity with more structure and higher granularity than log data. On the teacher side, we introduce the Jupyter Analytics Dashboard extension, which visualizes real-time student data directly in the notebook interface. The Jupyter Analytics system was developed through an iterative co-design process with university instructors and teaching assistants, and has been implemented and tested in several university STEM courses. We report two use cases where Jupyter Analytics impacted teaching and learning in the context of exercise sessions, and discuss the potential value of our tools for CS education.
computing homotopy groups of spheres has long been a fundamental objective in algebraic topology. Various theoretical and algorithmic approaches have been developed to tackle this problem. In this paper we take a step...
详细信息
computing homotopy groups of spheres has long been a fundamental objective in algebraic topology. Various theoretical and algorithmic approaches have been developed to tackle this problem. In this paper we take a step towards the goal of comprehending the group-theoretic structure of the generators of these homotopy groups by leveraging the power of machine learning. Specifically, in the simplicial group setting of Wu's formula, we reformulate the problem of generating simplicial cycles as a problem of sampling from the intersection of algorithmic datasets related to Dyck languages. We present and evaluate language modelling approaches that employ multi-label information for input sequences, along with the necessary group-theoretic toolkit and non-neural baselines.
We consider the communication complexity of some fundamental convex optimization problems in the point-to-point (coordinator) and blackboard communication models. We strengthen known bounds for approximately solving l...
详细信息
ISBN:
(纸本)9798400703836
We consider the communication complexity of some fundamental convex optimization problems in the point-to-point (coordinator) and blackboard communication models. We strengthen known bounds for approximately solving linear regression, p -norm regression (for 1 <= p <= 2), linear programming, minimizing the sum of finitely many convex nonsmooth functions with varying supports, and low rank approximation;for a number of these fundamental problems our bounds are optimal, as proven by our lower bounds. For example, for solving least squares regression in the coordinator model with B servers, n examples, d dimensions, and coefficients specified using at most L bits, we improve the prior communication bound of Vempala, Wang, and Woodruff (SODA, 2020) from (O) over tilde (sd(2)L) to (O) over tilde (sdL + d(2)epsilon L-1), which is optimal up to logarithmic factors. We also study the problem of solving least squares regression in the coordinator model to high accuracy, for which we provide an algorithm with a communication complexity of (O) over tilde (sd(L + log k) log (epsilon(-1)) + d(2)L), matching our improved lower bound for well-conditioned matrices up to a log (epsilon(-1)) factor. Among our techniques, we use the notion of block leverage scores, which have been relatively unexplored in this context, as well as dropping all but the "middle" bits in Richardson-style algorithms. We also introduce a new communication problem for accurately approximating inner products and establish a lower bound using the spherical Radon transform. Our lower bound can be used to show the first separation of linear programming and linear systems in the distributed model when the number of constraints is polynomial, addressing an open question in prior work.
Language models can serve as a valuable tool for software developers to increase productivity. Large generative models can be used for code generation and code completion, while smaller encoder-only models are capable...
详细信息
Implicitly parallel programming systems must solve the joint problems of dependence analysis and coherence to ensure apparently-sequential semantics for applications run on distributed memory machines. Solving these p...
详细信息
暂无评论