检索结果-内蒙古大学图书馆

International Conference on Cloud Computing and Big data (CCBD)

作者： Liu, Chao Yao, Hong Tang, Zhengwang Zeng, Deze Hu, Chengyu Liang, Qingzhong China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China

ISBN: (纸本)9781479966219

How to effectively process massive graph data is an intractable challenging issue. In this paper, two types of parallel computation approaches were compared: MapReduce and MyBSP. MyBSP is our open source implementation which adopts the Bulk Synchronous Parallel (BSP) programming model to support iterative processing. The MapReduce-based and MyBSP-based PageRank algorithms were implemented respectively. The experimental studies were conducted to evaluate and compare the performance and scalability of our MyBSP prototype system with MapReduce model. The results revealed that the MyBSP approach outperforms MapReduce approach for iterative graph data processing with vary size of datasets.

关键词： Bulk Synchronous Parallel MapReduce cloud computing graph data processing

来源：评论

学校读者我要写书评

暂无评论

An overview and an Approach for graph data processing using Hadoop MapReduce 2

An overview and an Approach for Graph Data Processing using ...

引用

2nd International Conference on Computing Methodologies and Communication (ICCMC)

作者： Talan, Pooja P. Sharma, Kartik U. PRMCEAM Comp Sci & Engn Badnera India

ISBN: (纸本)9781538634523

A very large quantity of data which traditional applications fail to process, leads the world to the era of Big data. With the increase in opportunity and technology scope, Big data also leads to many challenges such as data capture, storage, transfer, update, analysis, sharing, search, visualization, privacy of data etc. In order to deal with all these challenges there is a need of proper framework which will not only process the data but also provide a meaningful analysis so as to take proper decision in critical situations either related to industry, healthcare, social network, science, telecom, environment, business etc. The contribution of this paper is to analyze literature related to Big data & Hadoop framework and provide architecture to process graph data. Additionally, it provides online source code in order to understand big data for beginners.

关键词： Big data Hadoop MapReduce graph data processing

来源：评论

学校读者我要写书评

暂无评论

Research on Knowledge Storage and Query Technology Based on General graph data processing Framework 13

Research on Knowledge Storage and Query Technology Based on ...

引用

13th IEEE International Conference on Communication Software and Networks (ICCSN)

作者： Yu, Bihui Zhang, Yabiao Sun, Huajun Chinese Acad Sci Shenyang Inst Comp Technol Shenyang Peoples R China Univ Chinese Acad Sci Beijing Peoples R China Shenyang Univ Chem Technol Shenyang Peoples R China

ISBN: (纸本)9781665431828

With the development of the Semantic Web, more and more data is currently managed in the form of knowledge graphs. Different knowledge storage and query modes have their own advantages, but also have shortcomings, and there is no unified standard. Aiming at the current deficiencies in knowledge storage and knowledge query technology, this paper proposes a knowledge storage and query scheme based on TinkerPop graph computing framework, a general graph data processing framework that combines Neo4j massive graph data storage capabilities and SPARQL semantic query capabilities.

关键词： knowledge storage query technology graph data processing

来源：评论

学校读者我要写书评

暂无评论

Kylin: An Efficient and Scalable graph data processing System

Kylin: An Efficient and Scalable Graph Data Processing Syste...

引用

IEEE International Conference on Big data (Big data)

作者： Ho, Li-Yung Li, Tsung-Han Wu, Jan-Jan Liu, Pangfeng Natl Taiwan Univ Dept Comp Sci & Informat Engn Taipei 10764 Taiwan Acad Sinica Informat Technol Innovat Res Ctr Inst Informat Sci Taipei Taiwan Natl Taiwan Univ Grad Inst Networking & Multimedia Dept Comp Sci & Informat Engn Taipei Taiwan

ISBN: (纸本)9781479912926;9781479912933

We introduce Kylin, an efficient and scalable graph data processing system. Kylin is based on bulk synchronization processing(BSP) model to process graph data. Although there have been some BSP-based graph processing systems, Kylin is different from these systems in two-fold. First, Kylin cooperates with HBase to achieve scalable data manipulation. Second, We propose three techniques to optimize the performance of Kylin. The proposed techniques are pull messaging, lazy vertex loading and vertex-weighted partitioning. We demonstrate Kylin outperforms other BSP-based systems, i.e. Hama and Giraph, in the experiments.

关键词： graph data processing graph data partition load balancing pull messaging dynamic loading

来源：评论

学校读者我要写书评

暂无评论

An efficient iterative graph data processing framework based on bulk synchronous parallel model

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2020年第3期32卷

作者： Liu, Chao Zeng, Deze Yao, Hong Yan, Xuesong Yu, Linchen Fu, Zhangjie China Univ Geosci Sch Comp Sci Hubei Key Lab Intelligent Geoinformat Proc Wuhan Peoples R China Nanjing Univ Informat Sci & Technol Sch Comp & Software Nanjing Peoples R China

graph data processing has been widely applied in a variety of domains such as industry, science, social network, and so on. It therefore has stimulated many efforts devoted to this area. To embrace the fast development trend of big graph data, graph data processing based on Pregel-like systems has been regarded as one of the most promising ways and has widely attracted the attention of researchers. However, it still remains in its early stage and there still exist many challenges. In Pregel, the superstep synchronization is time consuming as the graph data iteration operation requires multiple synchronizations. Furthermore, the graph data partition strategy adopted by Pregel fails to support load balancing, therefore causing the increase of network I/O overhead as the scale of graph data grows. To address these issues, this paper presents an efficient computational framework for graph data processing based on the bulk synchronous parallel model. The global synchronization control mechanism is improved by determining the start time of the next round of superstep through counting the number of global message files. Furthermore, an improved graph data partition mechanism based on a balanced hash method is proposed to reduce the communication overhead between different partitions of sub-graph computational tasks. We also re-design the PageRank algorithm to verify the effectiveness of the proposed framework. Experimental results on different real-world datasets verify the efficiency of our proposed framework as it outperforms Giraph (an open source Pregel-like system) by 58%-69%, and achieves 10x-17x performance improvement over Hadoop.

关键词： bulk synchronous parallel model graph data processing graph partition global synchronization MapReduce

来源：评论

学校读者我要写书评

暂无评论

Arbor: Efficient Large-Scale graph data Computing Model

<i>Arbor</i>: Efficient Large-Scale Graph Data Computing Mod...

引用

15th IEEE International Conference on High Performance Computing and Communications (HPCC) /11th IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC)

作者： Zhou, Wei Li, Bo Han, Jizhong Xu, Zhiyong Chinese Acad Sci Inst Informat Engn Beijing Peoples R China Suffolk Univ Dept Math & Comp Sci Boston MA 02114 USA

ISBN: (纸本)9780769550886

graph data is the default data organization mechanism used in large-scale Social Network Service (SNS) applications. Traditional graph data computing models are used to dig out useful hidden information inside the data. However, the ever growing data volume is adding more and more pressures. To retrieve and discover the information, the system has to introduce a larger number of data iterations. This makes the data analysis operations becoming slower. To speed up these operations on large-scale graph data, recent research works focus on developing efficient parallel iteration processing strategies. However, the synchronization requirements between successive iterations can severely jeopardize the effectiveness of parallel operations. In this paper, we propose a novel large-scale graph data processing model, Arbor, to address these issues. Arbor substitutes time-constrained synchronization operations with non-time-constrained control message transmissions to increase the degree of parallelism. Furthermore, it develops a new graph data organization format, which can not only save storage space, but also accelerate graph data processing operations. We compare Arbor with other graph processing models using a large-scale experimental graph data, and the results show that it outperforms the state-of-the-art systems.

关键词： graph data graph data processing graph query graph aggregation graph analysis

来源：评论

学校读者我要写书评

暂无评论

High performance GPU primitives for graph-tensor learning operations

引用

JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING 2021年 148卷 125-137页

作者： Zhang, Tao Kan, Wang Liu, Xiao-Yang Shanghai Univ Sch Comp Engn & Sci Shanghai Peoples R China Columbia Univ Dept Elect Engn New York NY 10027 USA Shanghai Univ Shanghai Engn Res Ctr Intelligent Comp Syst Shanghai Peoples R China

graph-tensor learning operations extend tensor operations by taking the graph structure into account, which have been applied to diverse domains such as image processing and machine learning. However, the running time of graph-tensor operations increases rapidly with the number of nodes and the dimension of data on nodes, making them impractical for real-time applications. In this paper, we propose a GPU library called cugraph-Tensor for high-performance graph-tensor learning operations, which consists of eight key operations: graph shift (g-shift), graph Fourier transform (g-FT), inverse graph Fourier transform (inverse g-FT), graph filter (g-filter), graph convolution (g-convolution), graphtensor product (g-product), graph-tensor SVD (g-SVD) and graph-tensor QR (g-QR). cugraph-Tensor supports scalar, vector, and matrix data processing on each graph node. We propose optimization techniques on computing, memory accesses, and CPU-GPU communications that significantly improve the performance of the graph-tensor learning operations. Using the optimized operations, cugraphTensor builds a graph data completion application for fast and accurate reconstruction of incomplete graph data. In the experiments, the proposed graph learning operations achieve up to 142.12x speedups versus CPU-based GSPBOX and CPU MATLAB implementations running on two Xeon CPUs. The graph data completion application achieves up to 174.38x speedups over the CPU MATLAB implementation, and up to 3.82x speedups with better accuracy over the GPU-based tensor completion in the cuTensor-tubal library. (C) 2020 Elsevier Inc. All rights reserved.

关键词： GPU graph-tensor graph operations graph data processing Library

来源：评论

学校读者我要写书评

暂无评论

Survey of external memory large-scale graph processing on a multi-core system

引用

JOURNAL OF SUPERCOMPUTING 2020年第1期76卷 549-579页

作者： Huang, Jianqiang Qin, Wei Wang, Xiaoying Chen, Wenguang Tsinghua Univ Dept Comp Sci & Technol Beijing 100084 Peoples R China Qinghai Univ Dept Comp Technol & Applicat Xining 810016 Qinghai Peoples R China

The fast development of big data computing contributes to the fact that large-scale graph processing has become a basic computing model in both academic and industrial communities, and it has been applied in many actual big data computing works, such as social network analysis, Web search, and product promotion. These computing works include large-scale graphs of billions of vertices and trillions of edges. Such scale has brought many challenges to large-scale graph processing. This paper mainly introduces the essential features and challenges of large-scale graph processing and how we can handle billions of edges on a multi-core machine, for which we represent out-of-core processing system and semi-external memory processing systems. This paper also summarizes the key technologies in graph processing systems and forecasts the future development of large-scale graph processing systems.

关键词： graph data processing Parallel computing Computing model graph algorithms

来源：评论

学校读者我要写书评

暂无评论

Powerful graph neural network for node classification of the IoT network

引用

INTERNET OF THINGS 2024年 28卷

作者： Sejan, Mohammad Abrar Shakil Rahman, Md Habibur Aziz, Md Abdul Tabassum, Rana Baik, Jung-In Song, Hyoung-Kyu Sejong Univ Dept Elect Engn 209 Neungdong Ro Seoul 05006 South Korea Sejong Univ Dept Informat & Commun Engn 209 Neungdong Ro Seoul 05006 South Korea Sejong Univ Dept Convergence Engn Intelligent Drone 209 Neungdong Ro Seoul 05006 South Korea

Internet of Things (IoT) devices are increasingly used in various applications in our daily lives. The network structure for IoT is heterogeneous and can create a complex architecture depending on the application and geographical structure. To efficiently process the information within this diverse and complex relationship, a robust data structure is needed for network operations. graph neural network (GNN) technology is emerging as a capable tool for predicting complex data structures, such as graphs. graphs can be employed to mimic the structure of IoT network and process information from IoT nodes using GNN techniques. In this paper, our goal is explore the effectiveness of GNN in performing the node classification task for a given network. We have generated three different IoT networks with varying network sizes, number nodes, and feature sizes. We then test 12 different GNN algorithms to evaluate their performance in IoT node classification. Each method is examined in detail to observe its training behavior, testing behavior, and resilience against noise. In addition, time complexity and generalization ability of each model have also been studied. The experimental results show that some methods exhibit high resilience against noisy data for IoT node classification accuracy.

关键词： graph data processing graph convolutional network Internet of Things Node classification Machine learning

来源：评论

学校读者我要写书评

暂无评论

An Experimental Comparison of Pregel-like graph processing Systems

引用

PROCEEDINGS OF THE VLDB ENDOWMENT 2014年第12期7卷 1047-1058页

作者： Han, Minyang Daudjee, Khuzaima Ammar, Khaled Oezsu, M. Tamer Wang, Xingfang Jin, Tianqi Univ Waterloo David R Cheriton Sch Comp Sci Waterloo ON Canada

The introduction of Google's Pregel generated much interest in the field of large-scale graph data processing, inspiring the development of Pregel-like systems such as Apache Giraph, GPS, Mizan, and graphLab, all of which have appeared in the past two years. To gain an understanding of how Pregel-like systems perform, we conduct a study to experimentally compare Giraph, GPS, Mizan, and graphLab on equal ground by considering graph and algorithm agnostic optimizations and by using several metrics. The systems are compared with four different algorithms (PageRank, single source shortest path, weakly connected components, and distributed minimum spanning tree) on up to 128 Amazon EC2 machines. We find that the system optimizations present in Giraph and graphLab allow them to perform well. Our evaluation also shows Giraph 1.0.0's considerable improvement since Giraph 0.1 and identifies areas of improvement for all systems.

关键词： data handling Amazon ec2 Experimental comparison graph data processing graph processing Minimum spanning trees Single source shortest paths

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：