Large-scale graph analytics has gained attention during the past few years. As the world is going to be more connected by appearance of new technologies and applications such as social networks, Web portals, mobile de...
详细信息
ISBN:
(纸本)9781509024537
Large-scale graph analytics has gained attention during the past few years. As the world is going to be more connected by appearance of new technologies and applications such as social networks, Web portals, mobile devices, Internet of things, etc, a huge amount of data are created and stored every day in the form of graphs consisting of billions of vertices and edges. Many graph processing frameworks have been developed to process these large graphs since Google introduced its graph processing framework called Pregel in 2010. On the other hand, cloud computing which is a new paradigm of computing that overcomes restrictions of traditional problems in computing by enabling some novel technological and economical solutions such as distributed computing, elasticity and pay-as-you-go models has improved service delivery features. In this paper, we present iGiraph, a cost-efficient Pregel-like graph processing framework for processing large-scale graphs on public clouds. iGiraph uses a new dynamic re-partitioning approach based on messaging pattern to minimize the cost of resource utilization on public clouds. We also present the experimental results on the performance and cost effects of our method and compare them with basic Giraph framework. Our results validate that iGiraph remarkably decreases the cost and improves the performance by scaling the number of workers dynamically.
The high costs of acquiring and maintaining high-performance computing (HPC) resources pose significant barriers for medium-sized enterprises and educational institutions, often forcing them to rely on expensive cloud...
详细信息
The high costs of acquiring and maintaining high-performance computing (HPC) resources pose significant barriers for medium-sized enterprises and educational institutions, often forcing them to rely on expensive cloud-based solutions with recurring costs. This paper introduces CommC, a multi-purpose commodity hardware cluster designed to reduce operational expenses and extend hardware lifespan by repurposing underutilized computing resources. By integrating virtualization (KVM and Proxmox) and containerization (Kubernetes and Docker), CommC creates a scalable, secure, and cost-efficient computing environment. The proposed system enables seamless resource sharing, ensuring high availability and fault tolerance for both containerized and virtualized workloads. To demonstrate its versatility, we deploy big data engines like Apache Spark alongside traditional web services, showcasing CommC's ability to support diverse workloads efficiently. Our cost analysis reveals that CommC reduces computing expenses by up to 77.93% compared to cloud-based alternatives while also mitigating e-waste accumulation by extending the lifespan of existing hardware. This significantly improves environmental sustainability compared to cloud providers, where frequent hardware turnover contributes to rising carbon emissions. This research contributes to the fields of cloud computing, resource management, and sustainable IT infrastructure by providing a replicable, adaptable, and financially viable alternative to traditional cloud-based solutions. Future work will focus on automating resource allocation, enhancing real-time monitoring, and integrating advanced security mechanisms to further optimize performance and usability.
暂无评论