The technology of formal quantitative estimation of the conformity of mathematical models to the available dataset is presented. The main purpose of the technology is to make the model selection decision-making proces...
详细信息
The technology of formal quantitative estimation of the conformity of mathematical models to the available dataset is presented. The main purpose of the technology is to make the model selection decision-making process easier for the researcher. The method is a combination of approaches from the areas of data analysis, optimization and distributed computing including: cross validation and regularization methods, algebraic modeling in optimization and methods of optimization, automatic discretization of differential and integral equations, and optimization REST-services. The technology is illustrated by a demo case study. A general mathematical formulation of the method is presented. It is followed by a description of the main aspects of algorithmic and software implementation. The list of success stories of the presented approach is substantial. Nevertheless, the domain of applicability and important unresolved issues are discussed.
Widely used data processing platforms use distributed systems to process huge data efficiently. The aim of this article is to optimize the platform services by tuning only the relevant, tunable, system parameters and ...
详细信息
Widely used data processing platforms use distributed systems to process huge data efficiently. The aim of this article is to optimize the platform services by tuning only the relevant, tunable, system parameters and to identify the relation between the software quality metrics. The system parameters of data platforms based on the service level agreements can be defined and customized. In the first stage, the most significant parameters are identified and shortlisted using various feature selection approaches. In the second stage, the iterative runs of applications are executed for tuning these shortlisted parameters to identify the optimal value and to understand the impact of individual input parameters on the system output parameter. The empirical results imply significant improvement in performance and with which it is possible to render the proposed work optimizing the services offered by these data platforms.
Urgent computing workloads are time critical, unpredictable, and highly dynamic. Whilst efforts are on-going to run these on traditional HPC machines, another option is to leverage the computing power donated by volun...
详细信息
ISBN:
(纸本)9781665463348
Urgent computing workloads are time critical, unpredictable, and highly dynamic. Whilst efforts are on-going to run these on traditional HPC machines, another option is to leverage the computing power donated by volunteers. Volunteer computing, where members of the public donate some of their CPU time to large scale projects has been popular for many years because it is a powerful way of delivering compute for specific problems, with the public often eager to contribute to a good cause with societal benefits. However, traditional volunteer computing has required user installation of specialist software which is a barrier to entry, and the development of the software itself by the projects, even on-top of existing frameworks, is nontrivial. As such, the number of users donating CPU time to these volunteer computing projects has decreased in recent years, and this comes at a time when the frequency of disasters, often driven by climate change, are rising fast. We believe that an alternative approach, where visitors to websites donate some of their CPU time whilst they are browsing, has the potential to address these issues. However, web-based distributed computing is an immature field and there are numerous questions that must be answered to fully understand the viability of leveraging the large scale parallelism that website visitors represent. In this paper we describe our web-based distributed computing framework, Panther, and perform in-depth performance experiments for two benchmarks using real world hardware and real world browsing habits for the first time. By exploring the performance characteristics of our approach we demonstrate that this is viable for urgent workloads, but there are numerous caveats, not least the most appropriate visitor patterns to a website, that must be considered.
The evolution of telecommunication networks toward the fifth generation of mobile services (5G), along with the increasing presence of cloud-native applications, and the development of Cloud and Mobile Edge computing ...
详细信息
The evolution of telecommunication networks toward the fifth generation of mobile services (5G), along with the increasing presence of cloud-native applications, and the development of Cloud and Mobile Edge computing (MEC) paradigms, have opened up new opportunities for the monitoring and management of logistics and transportation. We address the case of distributed streaming platforms with multiple message brokers to develop an optimisation model for the real-time assignment and load balancing of event streaming generated data traffic among Edge computing facilities. The performance indicator function to be optimised is derived by adopting queuing models with different granularity (packet- and flow-level) that are suitably combined. A specific use case concerning a logistics application is considered and numerical results are provided to show the effectiveness of the optimisation procedure, also in comparison to a "static" assignment proportional to the processing speed of the brokers.
Introducing undergraduate students to key concepts of distributed computing has become almost essential as the world continues to embrace cloud-based solutions to daily problems and as research continues to grow in sc...
详细信息
ISBN:
(纸本)9781665473675
Introducing undergraduate students to key concepts of distributed computing has become almost essential as the world continues to embrace cloud-based solutions to daily problems and as research continues to grow in scale requiring distributed re-sources. Although distributed computing is an important part of the computer science curriculum, it can be difficult to introduce at some institutions. We explore some key challenges associated with introducing distributed computing into the computer science curriculum at a small, liberal arts college. We focus on an initial failure introducing a specialized distributed computing course too soon and relay the successes and failures experienced over a one year span of incorporating key distributed computing concepts across multiple systems-level courses. We discuss lessons learned from our first foray into teaching distributed computing and provide recommendations for new adopters of distributed computing curriculum based on our experiences.
Aiming at the lack of multi-fragment collision detection for complex targets in damage assessment simulation,a technical solution is proposed to use open source scene graphics OSG and Bullet physics engine to combine ...
详细信息
Aiming at the lack of multi-fragment collision detection for complex targets in damage assessment simulation,a technical solution is proposed to use open source scene graphics OSG and Bullet physics engine to combine distributed computing to solve collision detection of different fragments.A self-developed bounding box technology is proposed to construct the bounding box of multi-fragment damage elements,and the axial bounding box technology is selected to construct the bounding box of complex *** on the traditional collision algorithm,a collision detection algorithm relying on distributed computing is proposed,which realizes the collision detection between tens of thousands of fragments and the target,and visualizes the damage effect after the fragments collide with the *** simulation results show that the optimized solution can successfully achieve the target collision detection and damage assessment under the action of multiple fragments,which effectively improves the calculation efficiency and has a good rendering effect.
In order to reduce the communication load in general distributed computing frameworks such as MapReduce Li et al. introduced coded distributed computing (CDC). The authors in [1] proposed a scheme for cascaded coded d...
详细信息
In order to reduce the communication load in general distributed computing frameworks such as MapReduce Li et al. introduced coded distributed computing (CDC). The authors in [1] proposed a scheme for cascaded coded distributed computing on general networks. Although the numbers of input files and output functions in [1] are much less than those in [2], large numbers of input files and output functions are still required. In this paper, we present a new CDC scheme which 1) designs the assignments of input file and output function; 2) requires less number of input files and output functions compared with those in the scheme derived by [1]. Meanwhile, the ratio of communication load to that of the scheme in [1] is less than $\mathbf{the}\frac{4}{3}$ .
It is possible to develop intelligent and self-adaptive application on the edge nodes with rapid increase in computational capability of Internet of Things (IoT) devices. With the rapid growth of cloud technologies, t...
详细信息
It is possible to develop intelligent and self-adaptive application on the edge nodes with rapid increase in computational capability of Internet of Things (IoT) devices. With the rapid growth of cloud technologies, the demand for hybrid architecture with cloud and IoT has also been boosted as well. To satisfy the critical and comprehensive requirements in the architecture evolution, we proposed a lightweight framework called IoT-Pi to provide a 3-phase (sample, learn, adapt) life cycle management of cloud resources with machine learning prediction working on IoT edge nodes using Raspberry Pi device. Compared to the traditional interference by human beings in the field of system administration, the accuracy rate of machine learning prediction in the proposed technique for some algorithms reached over 70%, which demonstrates the feasibility and effectiveness of running cloud resource management on an IoT devices such as Raspberry Pi.
Coded distributed computing(CDC) has shown great potentials to solve the unexpected delay caused by stragglers and communication load in distributed computing. We propose a novel learning auction to allocate computing...
详细信息
ISBN:
(数字)9781665497923
ISBN:
(纸本)9781665497930
Coded distributed computing(CDC) has shown great potentials to solve the unexpected delay caused by stragglers and communication load in distributed computing. We propose a novel learning auction to allocate computing resource efficiently in a CDC scenario. The user demand types are usually het-erogeneous according to different variation trends of the value with finish time and workload, which can be modeled by deep learning. As the goal of social welfare maximizationthe platform would allocate computing resources according to inferred value functions of users. Due to the uncertain finish time and nonlinear structures of deep learning models, the considered optimization problem is non-convex. We then reformulate the non-convex optimization problem into a mixed integer program(MIP). After analyzing the inference error caused by deep learning, a payment rule referred to VCG is designed to achieve incentive alignment and individual rationality. Besides, experiments have been performed to show the superiority of our mechanism.
In this paper, we propose an Android-based distributed computing framework for accelerating DNN inference on Android edge devices. We experimentally demonstrate that the proposed distributed framework can reduce CPU u...
详细信息
ISBN:
(数字)9781665486118
ISBN:
(纸本)9781665486125
In this paper, we propose an Android-based distributed computing framework for accelerating DNN inference on Android edge devices. We experimentally demonstrate that the proposed distributed framework can reduce CPU utilization by 24 % (making the the CPU utilization close to that of idle status), reduce power consumption by 59.8 % to 71.8 %, without leading to high-bandwidth througput. The proposed framework can be applied to various Android devices to enable cooperation among edge devices in a distributed computing manner, accelerate DNN inference, and enrich the functionality of Android devices to enhance user experience.
暂无评论