A new tool and web portal are presented for deployment of High Performance Computing applications on distributed heterogeneous computing platforms. This tool relies on the decentralized environment P2PDC and the OMF a...
详细信息
ISBN:
(纸本)9781479927289
A new tool and web portal are presented for deployment of High Performance Computing applications on distributed heterogeneous computing platforms. This tool relies on the decentralized environment P2PDC and the OMF and OML multithreaded control, instrumentation and measurement libraries. Deployment on PlanetLab of a numerical simulation application is studied. A first series of computational results is displayed and analyzed.
A scheme to partition and allocate computational modules for distributed multimedia processing on wirelessly netwoked handheld devices (WNHHD) is presented. The minimization of battery utilization on the HHD is achiev...
详细信息
ISBN:
(纸本)1932415262
A scheme to partition and allocate computational modules for distributed multimedia processing on wirelessly netwoked handheld devices (WNHHD) is presented. The minimization of battery utilization on the HHD is achieved through minimization of computation on HHD device. The end-users machines with multiple potential heterogeneous idle is linked to achieve resource management. It also allows the multiple servers with idle retrieval bandwidth to help out end-user machines that are low power. The results show that the scheme provides battery management through resource scheduling.
Decision support systems are characterized by large data sets and complex queries with multi-way joins, aggregation and nesting. SQLmpp is a highly parallel database server designed to efficiently support these applic...
详细信息
Decision support systems are characterized by large data sets and complex queries with multi-way joins, aggregation and nesting. SQLmpp is a highly parallel database server designed to efficiently support these applications. SQLmpp's query processing strategy is driven by three principal goals: 1) data parallelism in all operations, 2) maximal use of any relevant indexes, and 3) minimal processing of complete relations. This paper describes the general query processingtechniques and the specific operations used to achieve these goals.
techniques of customizing and extending operating systems (OSs) have a growing impact on system architectures in the field of distributed computing and parallel programming. Even if traditional methods of adaption hav...
详细信息
ISBN:
(纸本)0780342291
techniques of customizing and extending operating systems (OSs) have a growing impact on system architectures in the field of distributed computing and parallel programming. Even if traditional methods of adaption have been limited to the user-level, modern OSs cannot do without kernel support. Hence concepts and structures of microkernel architectures must be redefined to meet the requirements of today's and future applications. In this paperwork we are proposing a new customizable low-level OS architecture - the Dycos kernel. In a first part we will discuss customization demands on microkernels. In a second part we are passing over to describe the basic kernel concept. Dycos is an object-based approach providing a toolbox of operations to build user-definable compositions of kernel structures. The Dycos approach has been evaluated on a Solaris 2.5.1 platform.
For a stream processing system that uses checkpoints as a fault-tolerant method, selecting the appropriate checkpoint period is the key to ensuring the efficient operation of streaming applications. State-of-art strea...
详细信息
ISBN:
(纸本)9781665473156
For a stream processing system that uses checkpoints as a fault-tolerant method, selecting the appropriate checkpoint period is the key to ensuring the efficient operation of streaming applications. State-of-art stream processing systems currently only support fixed-cycle checkpoints, which is difficult to make a good trade-off between fault-tolerant processing and the cost of failure recovery in dynamically changing streaming application scenarios. Moreover, in a complex distributed streaming application environment, the dynamic environmental indicators (e.g., the values of workloads and failure rates) are not in coincidence with the model assumptions, such as the dynamics of Twitter's hot events data changing quickly. In this paper, we consider the dynamic changes of environmental indicators and adaptively optimize the processing delay and fault recovery time. Then, we propose a dynamic adjustment method for the checkpoint interval by reinforcement learning, which is named DACM. DACM adaptively optimizes the processing delay and fault recovery time, while avoiding the overall environment modeling of streaming applications. The experiments conducted on the Flink platform show that DACM reduces the processing delay by 10% and the failure recovery time by 37% compared with the existing checkpoint interval optimization models.
With the rapid development of the Internet, applications in distributedprocessing systems have attracted more and more attention. The privacy protection is one of the most important factors to ensure the security of ...
详细信息
ISBN:
(纸本)9781538637906
With the rapid development of the Internet, applications in distributedprocessing systems have attracted more and more attention. The privacy protection is one of the most important factors to ensure the security of underlying networks in a distributed computing system. Users expect to use a service (e.g., a computing service) anonymously, while service providers want to make sure that the users are trusted and their behavior can be accountable. Given that these two factors are contradictory, how to provide an efficient solution to achieve trusted anonymous access to the services in distributed computing networks is a challenging task. In a distributed computing system, when a task needs to be computed by a worker (or server), the transmitted data over the underlying network to perform this computing service might include many different contents, and even lots of different flows. To this end, this paper presents a new and efficient architecture to help users use services anonymously in distributed computing networks, without loss of trustworthiness. The experimental results show that the proposed architecture introduces acceptable overheads, and can achieve the goal of balancing anonymity and trust in distributed computing networks.
作者：
Shires, Dale R.Henz, Brian J.
High Performance Computing Division Aberdeen Proving Ground MD 21005 United States
Scientific computing has historically been performed using simple, efficient languages supporting function-oriented design. This can be seen in the legacy codes written in FORTRAN 77 as well as the newer codes written...
详细信息
ISBN:
(纸本)1892512416
Scientific computing has historically been performed using simple, efficient languages supporting function-oriented design. This can be seen in the legacy codes written in FORTRAN 77 as well as the newer codes written in C and Fortran 90/95. However, these applications suffer the same problems found in other software written in this fashion. Code reuse is inefficient and prone to error, and changing or modifying the algorithms employed is a highly problematic and time-consuming operation. Accordingly, a new approach has been developed employing the object-oriented design methodology for a large class of computational engineering problems that use the finite element method. The framework has been designed to deliver portability, modularity, extensibility, and efficiency through parallelism.
parallel and distributedprocessing is employed to accelerate training for many deep-learning applications with large models and inputs. As it reduces synchronization and communication overhead by tolerating stale gra...
详细信息
ISBN:
(纸本)9781538610428
parallel and distributedprocessing is employed to accelerate training for many deep-learning applications with large models and inputs. As it reduces synchronization and communication overhead by tolerating stale gradient updates, asynchronous stochastic gradient descent (ASGD), derived from stochastic gradient descent (SGD), is widely used. Recent theoretical analyses show ASGD converges with linear asymptotic speedup over SGD. Oftentimes glossed over in theoretical analysis are communication overhead and practical learning rates that are critical to the performance of ASGD. After analyzing the communication performance and convergence behavior of ASGD using the Downpour algorithm as an example, we demonstrate the challenges for ASGD to achieve good practical speedup over SGD. We propose a distributed, bulk-synchronous stochastic gradient descent algorithm that allows for sparse gradient aggregation from individual learners. The communication cost is amortized explicitly by a gradient aggregation interval, and global reductions are used instead of a parameter server for gradient aggregation. We prove its convergence and show that it has superior communication performance and convergence behavior over popular ASGD implementations such as Downpour and EAMSGD for deep-learning applications.
Deploying a single application on stand-along server is a solved problem. However, real life system solution typically requires multiple layers of middleware running on multiple servers, and deploying such an applicat...
详细信息
ISBN:
(纸本)1601320841
Deploying a single application on stand-along server is a solved problem. However, real life system solution typically requires multiple layers of middleware running on multiple servers, and deploying such an application is still an open problem. Considerable domain expertise is required to manage the complexities at all levels, making such deployment a frustrating and lengthy exercise for the service engineers as well as the customers. This paper describes a tooling framework to address this problem. The scope of the framework spans from modeling the distributed application, to creating a deployment package, then to deploying and debugging the application. This framework has been used to deploy several large scale distributedapplications at IBM.
Anomaly diagnosis for distributed service plays an important role in communication network information system. Log analysis is the main method to undertake anomaly detection. In order to reduce the manual detection, w...
详细信息
ISBN:
(纸本)9781538637906
Anomaly diagnosis for distributed service plays an important role in communication network information system. Log analysis is the main method to undertake anomaly detection. In order to reduce the manual detection, we propose an anomaly detection method based on the time-weighted control flaw graph model. The border is split by a discrete degree strategy based on analyzing the time interval distribution and the time weight is selected to be k-means. Experiments show that our algorithm has good precision and recall in anomaly diagnosis. In real-world scenarios, it has a precision of 80% and a recall rate of 65% on average.
暂无评论