The paper aims to propose a distributed clustering method for High performance computing (HPC) models and, its application for medical image processing. The communication cost is one of the great challenges, which min...
详细信息
The paper aims to propose a distributed clustering method for High performance computing (HPC) models and, its application for medical image processing. The communication cost is one of the great challenges, which minimizes the scalability of parallel and distributed computing models. Indeed, it reduces significantly the performance of HPC systems where these models are assigned to be implemented. In this paper, we present a new distributedk-means method which integrates virtual parallel distributed computing model with a low communication cost mechanism. The k-means method is performed as a distributed service within a cooperative micro-services team which uses asynchronous communication mechanism based on AMQP protocol. We design and implement a parallel and distributed HPC application for MRI image segmentation assigned to be deployed on cloud. Experimental results show that the proposed method (DSCM) and its assigned model reach high degree of scalability. We expect this clustering approach to provide scalable HPC applications for big data clustering.
distributed clustering algorithms have proven to be effective in dramatically reducing execution time. However, distributed environments are characterized by a high rate of failure. Nodes can easily become unreachable...
详细信息
distributed clustering algorithms have proven to be effective in dramatically reducing execution time. However, distributed environments are characterized by a high rate of failure. Nodes can easily become unreachable. Furthermore, it is not guaranteed that messages are delivered to their destination. As a result, fault tolerance mechanisms are of paramount importance to achieve resiliency and guarantee continuous progress. In this paper, a fault-tolerant distributed k-means algorithm is proposed on a grid of commodity machines. Machines in such an environment are connected in a peer-to-peer fashion and managed by a gossip protocol with the actor model used as the concurrency model. The fact that no synchronization is needed makes it a good fit for parallel processing. Using the passive replication technique for the leader node and the active replication technique for the workers, the system exhibited robustness against failures. The results showed that the distributed k-means algorithm with no fault-tolerant mechanisms achieved up to a 34% improvement over the Hadoop-based k-meansalgorithm, while the robust one achieved up to a 12% improvement. The experiments also showed that the overhead, using such techniques, was negligible. Moreover, the results indicated that losing up to 10% of the messages had no real impact on the overall performance.
暂无评论