MapReduce programming model is a popular model to simplify but speed up data parallel applications. However, it is not efficient for iterative applications because of its repeated data transmission with HDFS (Hadoop D...
详细信息
MapReduce programming model is a popular model to simplify but speed up data parallel applications. However, it is not efficient for iterative applications because of its repeated data transmission with HDFS (Hadoop Distributed File system). Conch, a cyclic MapReduce model, is designed for efficient processing of iterative applications. In order to minimize network overhead, shared data is cached locally and a "map-shuffle" phase is presented with a combined transmission mechanism. Meanwhile, a prediction scheduler for iterative applications is brought out to achieve better data locality in terms of runtime information. The experiments show that Conch can support iterative applications transparently and efficiently. Compared with Hadoop and HaLoop in single-job environment, Conch can achieve 13%-17% improvements on K-Means and fuzzy C-Means. Especially in multi-job environment, 63.6% and 28.6% improvements can be obtained compared with Hadoop and HaLoop.
This paper provides a new data fusion mechanism based on regulation and reliability to solve the data conflict problems of multi-source heterogeneous data fusion in traffic information engineering. This mechanism eval...
详细信息
This paper provides a new data fusion mechanism based on regulation and reliability to solve the data conflict problems of multi-source heterogeneous data fusion in traffic information engineering. This mechanism evaluates each data source on historical reliability and data source QoS (Quality of Service), and then gives its reliability result. Userspsila reliability on the data source is calculated and dynamically adjusted, then new reliability data of the data source is given, finally the conflict data is fused. The validation of experiment result indicates that the method enhances accuracy and adoption ratio of the collected data in real time greatly.
Automated trust negotiation (ATN) is an important means to establish trust between strangers through the exchange of digital credentials and access control policies specifying what credentials a stranger must submit i...
详细信息
Automated trust negotiation (ATN) is an important means to establish trust between strangers through the exchange of digital credentials and access control policies specifying what credentials a stranger must submit in the open and distributed environment. While ATN brings convenience, there are still two issues unsolved successfully: 1) to protect the transmitting message; 2) to prevent the sensitive information leakage. Hidden credential can provide high security level to protect sensitive resources, policies and credentials from being attacked. Zero-knowledge is an important means to avoid leakage and can be used to protect the user's private context and sensitive information from unauthorized inferences. If hidden credentials and zero-knowledge are well-combined and redesigned, they can jointly make ATN better. Based on these, a hidden credential based oblivious automated trust negotiation model (HBOA) is proposed in this paper. In the model, the negotiation information is carried by hidden credentials, which have the same secure level with elliptic curve. Zero-knowledge protocol is provided by Pedersen commitment scheme, which ensures that no sensitive information can be attained by authorized users. A practical example is given to prove its usability.
Peer-to-peer (P2P) technology encounters serious methodological limitations to guarantee the quality of services (QoS) of P2P networks with very few peers. In this paper, we propose multiple-access channel model (MACM...
详细信息
Peer-to-peer (P2P) technology encounters serious methodological limitations to guarantee the quality of services (QoS) of P2P networks with very few peers. In this paper, we propose multiple-access channel model (MACM) to merge different P2P networks as a whole to share their data-transfer capability. The model abstracts the interlaced data links between peers as logical data channels according to their spatial-temporal relations. The channels can be subdivided based on multiplex technology and serve different applications at same time. MACM redefines P2P data-packet structure and organizes data packets into time-related groups according to cascade-packet rule to support controllable data-projection between data pools of peers. MACM-based P2P networks can be created as virtual sub-network or virtual-peer bridging strategy so as to improve the performance of P2P networks with few peers but high priorities within Internet protocol television (IPTV) and wireless sensor network (WSN) applications.
During the study of grid projects, a real grid environment or simulated grid is built up for development and experiment normally. But there are some disadvantages such as high cost, complex deployment and configuratio...
详细信息
During the study of grid projects, a real grid environment or simulated grid is built up for development and experiment normally. But there are some disadvantages such as high cost, complex deployment and configuration, hard management, even conflict among people. A novel virtual grid experiment method with HAL (Hardware Abstraction Level) virtualization technology is presented, which can conquer the disadvantages above and has been used in MedImGrid for the development of medical grid applications. The experiment shows that the method is feasible with low cost, easy configuration and convenient management, so as to increase research efficiency and shorten developing cycle. Furthermore, it can be applied not only in MedImGrid but also in some other grid projects.
In this paper, we propose a scalable service scheme for secure group communication in grid. In the service scheme, a series of methods and strategies are presented, such as the initialization methods for group member,...
详细信息
In this paper, we propose a scalable service scheme for secure group communication in grid. In the service scheme, a series of methods and strategies are presented, such as the initialization methods for group member, administrative domain and virtual organization, the key distribution strategy and the rekeying strategy. In order to improve the scalability of this service scheme, the services for a group are logically divided into two hierarchical levels, which is in accordance with the characteristics of group communication in grid. In addition, in order to show the efficiency and the scalability of the service scheme, simulation experiments are done. The results show that the service scheme is efficient and scalable. Thus, the service scheme can satisfy the requirement of people in large-scale, dynamic grid environment.
Recently, convolutional networks have achieved great successes in the field of computer vision. In order to improve the efficiency of convolutional networks, large amount of solutions focusing on training algorithms a...
详细信息
ISBN:
(纸本)9781479989386
Recently, convolutional networks have achieved great successes in the field of computer vision. In order to improve the efficiency of convolutional networks, large amount of solutions focusing on training algorithms and parallelism strategies have been proposed. In this paper, a novel algorithm based on look-up table is proposed to speed up convolutional networks with small filters by applying GPU. By transforming multiplication operations in the convolution computation to some table-based summation operations, the overhead of convolution computation can be reduced largely. The process of creating table and looking up table is very appropriate for parallelization on GPU. Experiment results show that the proposed approaches can improve the speed of convolution computation by 20%-30%, compared with state-of-the-art existing works.
Cell broadband engine (Cell BE) is a state-of-the-art heterogeneous multi-core processor. It is an accelerator-based multi-core architecture, which contains a powerful 64-bit dual-threaded PowerPC core and eight high ...
详细信息
ISBN:
(纸本)9780769533063
Cell broadband engine (Cell BE) is a state-of-the-art heterogeneous multi-core processor. It is an accelerator-based multi-core architecture, which contains a powerful 64-bit dual-threaded PowerPC core and eight high efficient single-instruction-multiple-data (SIMD) cores. Apart from traditional parallel systems, the users must explicitly manage the communication, scheduling and load-balancing to achieve cellpsilas greatest performance. In this paper, a novel heterogeneous data parallel computational model on cell BE is proposed. This aggressive model could not only exploit computing power of SPE but also that of PPE and aggregate them together to achieve high performance. We investigate the performance of this model with naive ray tracing algorithm. The preliminary experimental results validate the efficiency of this model.
In this paper, we propose a scheme that manages the computational resource of virtual machines that are used to host high performance computing applications. Different from the static configuration methodology employe...
详细信息
In this paper, we propose a scheme that manages the computational resource of virtual machines that are used to host high performance computing applications. Different from the static configuration methodology employed by the state-of-art virtual machine monitors, in our scheme, the virtual machines are automatically configured according to the actual load generated by the applications. NPB, HPL and kernel compilation are chosen as representative high performance computing applications to run inside the virtual machine constructed using our scheme, and the performance of such applications are compared with that obtained from the statically configured virtual machines. The comparison indicates that besides the great flexibility it brings, the performance penalty resulted by our scheme is below 5% in most cases, and the performance of the application running inside the automatically configured virtual machine is even better than that running inside the statically configured ones in some cases.
Deterministic multithreading (DMT) system is well-known to eliminate the harmful program behaviors caused by nondeterminism, i.e., always proceeding the program execution into the same thread schedule for the same giv...
详细信息
Deterministic multithreading (DMT) system is well-known to eliminate the harmful program behaviors caused by nondeterminism, i.e., always proceeding the program execution into the same thread schedule for the same given input. To achieve this goal, two kinds of schedules are enforced by existing DMT systems. 1) A mem-based schedule ensures the determinism with the total order of the shared memory accesses, and 2) A sync-based schedule makes it by only enforcing the total order of the synchronization operations. Mem-schedule achieves full determinism but suffers from prohibitive overhead; while sync-schedule mitigates this overhead but cannot ensure the determinism for the race schedules, i.e., part determinism. Much recent research is devoted to the hybrid schedule combining the determinism of mem-schedule and efficiency of sync-schedule. However, they suffer from the practicability and scalability problems due to the defects of their technical characteristics, such as trace collection in advance and huge schedule memoization. To address the above problem, this paper proposes esDMT, an efficient and scalable DMT system using a new technique of memory isolation. It can improve the efficiency by proceeding the execution of each thread in parallel within its private virtual memory, and defers the determinism guarantee by updating private memory into shared memory in a deterministic order according to deterministic lock algorithm, thus further reducing the overhead of inter-thread waiting. In contrast to the previous hybrid work avoiding the nondeterminism of race schedules offline based on the enormous historical records, our key insight is to eliminate the nondeterminism of race schedules online at runtime. Our experimental results on PARSEC benchmarks show that esDMT eliminates the nondeterminism successfully, almost gains the same performance as the sync-schedule (with
暂无评论