Scientific workflows are increasingly gaining attention as both data and compute resources are getting bigger, heterogeneous, and distributed. Many scientific workflows are both compute intensive and data intensive an...
详细信息
ISBN:
(纸本)9780769551173
Scientific workflows are increasingly gaining attention as both data and compute resources are getting bigger, heterogeneous, and distributed. Many scientific workflows are both compute intensive and data intensive and use distributed resources. This situation poses significant challenges in terms of real-time remote analysis and dissemination of massive datasets to scientists across the community. These challenges will be exacerbated in the exascale era. parallel jobs in scientific workflows are common, and such parallelism can be exploited by scheduling parallel jobs among multiple execution sites for enhanced performance. Previous scheduling algorithms such as heterogeneous earliest finish time (HEFT) did not focus on scheduling thousands of jobs often seen in contemporary applications. Some techniques, such as task clustering, have been proposed to reduce the overhead of scheduling a large number of jobs. However, scheduling massively parallel jobs in distributed environments poses new challenges as data movement becomes a nontrivial factor. We propose efficient parallel execution models through pipelined execution of data transfer, incorporating network bandwidth and reserved resources at an execution site. We formally analyze those models and suggest the best model with the optimal degree of parallelism. We implement our model in the Swift parallel scripting paradigm using GridFTP. Experiments on real distributed computing resources show that our model with optimal degrees of parallelism outperform the current parallel execution model by as much as 50% reduction of total execution time.
With the growing popularity of the Internet of Things (IoT), energy efficiency has been a critical concern during the design and development of IoT service systems. Meanwhile, edge computing has drawn significant atte...
详细信息
ISBN:
(纸本)9781538637906
With the growing popularity of the Internet of Things (IoT), energy efficiency has been a critical concern during the design and development of IoT service systems. Meanwhile, edge computing has drawn significant attention as a burgeoning computing paradigm. This paper studies the energy efficiency issue of IoT systems by proposing a joint scheme of resource allocation and task scheduling under the edge computing paradigm. Specifically, dynamic processes of the IoT services and system are formulated by generalized queueing network models, based on which quantitative analyses of performance and energy consumption are conducted. The resource management and task scheduling are formulated by Markov Decision Process (MDP), which can balance the tradeoff between energy costs and QoS requirements. To attack the challenge of MDP search space explosion due to the large scale of IoT systems, Ordinal Optimization (OO) techniques are applied to the MDP algorithms, which are able to significantly narrow the search of MDP by slightly softening the optimization objective to a good enough subset. Finally, we conduct simulation experiments based on real-world IoT data. Evaluations and comparisons demonstrate that our approach is effective and efficient in practice.
In this paper we develop a distributed algorithm, using probes stamped with dynamic priorities, for detecting deadlocks in the single resource model. The correctness of the algorithm is proven. A dynamic priority is a...
详细信息
ISBN:
(纸本)1932415262
In this paper we develop a distributed algorithm, using probes stamped with dynamic priorities, for detecting deadlocks in the single resource model. The correctness of the algorithm is proven. A dynamic priority is a pair (p, c) where p is a process and c is the counter value in p when p initiates a deadlock detection computation. The pair is called a detector when it is referenced in a process or a resource manager It is maintained in a process and a resource manager updated dynamically upon certain events, and used as the priority of a probe in a deadlock detection computation. Introduction of a dynamic priority to the probes, combined with a dynamically updated detector in a process and resource manager make the algorithm more efficient and less complex to implement than the existing ones. Our algorithm has an O(1) space complexity for each process. In certain cases, the algorithm uses n messages to detect a deadlock and has an O(n) detection delay where n is the number of processes and resource managers that are involved in the deadlock.
This paper presents a scalable design and implementation of the molecular docking application DOCK for a large-scale high performance computing system, the Sunway TaihuLight supercomputer, which provisions a heterogen...
详细信息
ISBN:
(纸本)9781538637906
This paper presents a scalable design and implementation of the molecular docking application DOCK for a large-scale high performance computing system, the Sunway TaihuLight supercomputer, which provisions a heterogeneous, manycore processor architecture that consists of management processing elements (MPEs) and clusters of computing processing elements (CPEs). The key innovation is a novel refactoring of DOCK on the CPEs. Optimization techniques for data redundancy minimization to fit data in cache, software-controlled prefetching into scratchpads, memory access coalescing, software caches, vectorization and loop unrolling are employed to improve the exploitation of the computational resources. For a single docking process, the refactored version using both the MPE and CPE cluster achieved 260x to 402x speedup compared against the original ported version using MPE only. To scale the DOCK to the full Sunway Taihulight system with 10,649,600 cores (including all MPE and CPE cores), we present an MPI communication domain partition scheme as well. For docking 9 million small compounds to a Zika virus target protein, we manage to scale to 131,072 MPEs, and 8,388,608 CPEs, with a total of 8,519,680 cores.
Knot detection in a distributed graph is an important problem and finds applications in several areas such as packet switching, distributed simulation, and distributed database systems. This paper presents a distribut...
详细信息
ISBN:
(纸本)0769516777
Knot detection in a distributed graph is an important problem and finds applications in several areas such as packet switching, distributed simulation, and distributed database systems. This paper presents a distributed algorithm to efficiently detect the existence of a knot in a distributed graph. The algorithm requires 2e messages and a delay of 2(d+1) message hops to detect if a node in a distributed graph is in a knot (e is the number of edges in the reachable part of the distributed graph and d is its diameter). A significant advantage of this algorithm is that it not only detects if a node is in a knot but also finds exactly which nodes are involved in the knot.
As the Internet of Vehicles (IoV) becomes flourishing and the data generated by sensors be ubiquitous, there exist various kinds of IoV applications with different performance requirements. Hence, different distribute...
详细信息
ISBN:
(纸本)9781538637906
As the Internet of Vehicles (IoV) becomes flourishing and the data generated by sensors be ubiquitous, there exist various kinds of IoV applications with different performance requirements. Hence, different distributed data processing systems (DDPS) clusters will coexist, e.g., a stream processing system cluster for real-time tasks and a batch one for statistics based data mining tasks, to meet the requirements of such IoV applications. However, it is not an economical or convenient way to maintain varied systems clusters, as the developers and/or administrators have to be familiar with all of these DDPSs, and of course, the deployment of multiple DDPS means a waste of resources compared to the deployment of one DDPS. Based on these observations, this paper proposes the TDAG as a solution. TDAG allows users to adjust the data processing from the streaming style to the batch style by encapsulating the input data with specific packing strategies. We have implemented TDAG in a prototype called TStream. The experimental tests show that our TStream is both effective and efficient.
Advantages are shown by using the DSP for the real time online control of hydraulic axis. The interrupt latency time is the most important factor for equidistant position samples to control the velocity as accurately ...
详细信息
ISBN:
(纸本)1932415262
Advantages are shown by using the DSP for the real time online control of hydraulic axis. The interrupt latency time is the most important factor for equidistant position samples to control the velocity as accurately as possible. The trajectory calculations for the velocity and position are presented as well as their derivation for S-curvatures. The control of the velocity, as a function of the force, is the key factor for the exact cold molding processes. Other applications are discussed, for example, the use of Newton algorithm to determine the dead point of the hydraulic axis movement and the control of two different hydraulic paths.
Characteristics and way of behavior of attacks and infiltrators on computer networks are very difficult and need an expert. In addition;the advancement of computer networks, the number of attacks and infiltrations is ...
详细信息
ISBN:
(纸本)9781538637906
Characteristics and way of behavior of attacks and infiltrators on computer networks are very difficult and need an expert. In addition;the advancement of computer networks, the number of attacks and infiltrations is also increasing. In fact, the knowledge coming from expert will lose its value over time and must be updated and made available to the system and this makes the need for expert person always felt. In machine learning techniques, knowledge is extracted from the data itself which has diminished the role of the expert. Various methods used to detect intrusions, such as statistical models, safe system approach, neural networks, etc., all weaken the fact that it uses all the features of an information packet rotating in the network for intrusion detection. Also, the huge volume of information and the unthinkable state space is also an important issue in the detection of intrusion. Therefore, the need for automatic identification of new and suspicious patterns in attempt for intrusion with the use of more efficient methods (Lower cost and higher performance) is needed more than before. The purpose of this paper is to provide a new method based on intrusion detection systems and its various architectures aimed at increasing the accuracy of intrusion detection in cloud computing.
A fundamental issue affecting the performance of parallelapplications running on distributed systems is the assignment of tasks to processors. This paper shows the effectiveness in scheduling strategies derived from ...
详细信息
ISBN:
(纸本)0769512607
A fundamental issue affecting the performance of parallelapplications running on distributed systems is the assignment of tasks to processors. This paper shows the effectiveness in scheduling strategies derived from the use of the temporal behaviour of tasks included in the new TTIG (Temporal Task Interaction Graph) model. Experimentation was performed for a set of C+PVM applications running in a PVM platform. These applications were on the one hand synthetic programs whose communication topology matches certain well-known regular graph families such as trees, pipes and meshes and programs with irregular communication patterns. Additionally, a real image processing application was modelled and executed. The TTIG model has been shown to be effective in all cases compared with the classical TIG (Task Interaction Graph) and with the PVM default allocation scheme, and facilitates the development of new more efficient scheduling strategies.
DSP chip appears on the basis of progress of three technologies. These technologies are technology of the computer, technology of signal treatment and micro-electric techni ue. A review on characteristics, application...
详细信息
ISBN:
(纸本)9781932415582
DSP chip appears on the basis of progress of three technologies. These technologies are technology of the computer, technology of signal treatment and micro-electric techni ue. A review on characteristics, applications and the prospects of DSP is provided in this paper. The development process of characteristics, applications and the prospects of DSP are introduced briefly. The main problems which exist in the applications of DSP bring into focus. The advantages and disadvantages of characteristics of DSP are comprehensively analy ed. Finally the development future trends are presented. It is believed that DSP will be applied in more and more science and technology field.
暂无评论