检索结果-内蒙古大学图书馆

A Comparative Survey of big data computing and HPC: From a Parallel Programming Model to a Cluster Architecture

INTERNATIONAL JOURNAL OF PARALLEL PROGRAMMING 2022年第1期50卷 27-64页

作者： Yin, Fei Shi, Feng Beijing Inst Technol Sch Comp Sci Beijing 100081 Peoples R China

With the rapid growth of artificial intelligence (AI), the Internet of Things (IoT) and big data, emerging applications that cross stacks with different techniques bring new challenges to parallel computing systems. These cross-stack functionalities require one system to possess multiple characteristics, such as the ability to process data under high throughput and low latency, the ability to carry out iterative and incremental computation, transparent fault tolerance, and the ability to perform heterogeneous tasks that evolve dynamically. However, high-performance computing (HPC) and big data computing, as two categories of parallel computing architecture, are incapable of meeting all these requirements. Therefore, by performing a comparative analysis of HPC and big data computing from the perspective of the parallel programming model layer, middleware layer, and infrastructure layer, we explore the design principles of the two architectures and discuss a converged architecture to address the abovementioned challenges.

关键词： High-performance computing big data computing Parallel computing architecture Iterative computation Heterogeneous tasks Converged architecture Cross-stack functionality

来源：评论

学校读者我要写书评

暂无评论

Sublinear-time reductions for big data computing

引用

THEORETICAL COMPUTER SCIENCE 2022年 932卷 1-12页

作者： Gao, Xiangyu Li, Jianzhong Miao, Dongjing Harbin Inst Technol Dept Comp Sci & Technol Harbin Peoples R China Chinese Acad Sci Shenzhen Inst Adv Technol Dept Comp Sci & Control Engn Shenzhen Peoples R China

The inefficiency of polynomial-time (PTIME) algorithms in the context of big data indicates a departure from the traditional view on tractability. In recent years, sublineartime algorithms have been regarded as the new standard for tractability of big data computing problems. Based on the prior work on sublinear-time complexity classes[1], this paper focuses on designing appropriate reductions specialized for big data computing problems. In particular, a series of pseudo-sublinear-time reductions are proposed, and their properties are systematically investigated. It is proved thatPsTis properly contained inP, and anyPsT-complete problem and PsPLi-complete problem that is notP-complete must be a witness in P\NC. Several examples are also given to illustrate the usefulness of the proposed reductions. Then, to cope with problems that are intractable in sublinear time even after a PTIME preprocessing, the L-reduction is extended to pseudo-sublinear time. Finally, it is proved that there is noPPL-complete problem under DLOGTIME reduction. (C) 2022 Elsevier B.V. All rights reserved.

关键词： big data computing Sublinear-time tractability Reduction techniques Completeness

来源：评论

学校读者我要写书评

暂无评论

Sublinear-Time Reductions for big data computing 15th

Sublinear-Time Reductions for Big Data Computing

引用

15th Annual International Conference on Combinatorial Optimization and Applications (COCOA)

作者： Gao, Xiangyu Li, Jianzhong Miao, Dongjing Harbin Inst Technol Dept Comp Sci & Technol Harbin Peoples R China Chinese Acad Sci Fac Comp Sci & Control Engn Shenzhen Inst Adv Technol Shenzhen Peoples R China

ISBN: (纸本)9783030926816;9783030926809

With the rapid popularization of big data, the dichotomy between tractable and intractable problems in big data computing has been shifted. Sublinear time, rather than polynomial time, has recently been regarded as the new standard of tractability in big data computing. This change brings the demand for new methodologies in computational complexity theory in the context of big data. Based on the prior work for sublinear-time complexity classes [9], this paper focuses on sublinear-time reductions specialized for problems in big data computing. First, the pseudo-sublinear-time reduction is proposed and the complexity classes P and PsT are proved to be closed under it. To establish PsT-intractability for certain problems in P, we find the first problem in P\PsT. Using the pseudo-sublinear-time reduction, we prove that the nearest edge query is in PsT but the algebraic equation root problem is not. Then, the pseudo-polylog-time reduction is introduced and the complexity class PsPL is proved to be closed under it. The PsT-completeness under it is regarded as an evidence that some problems can not be solved in polylogarithmic time after a polynomial-time preprocessing, unless PsT = PsPL. We prove that all PsT-complete problems are also P-complete, which gives a further direction for identifying PsT-complete problems.

关键词： big data computing Sublinear-time tractability Reduction techniques Preprocessing

来源：评论

学校读者我要写书评

暂无评论

Recognizing the tractability in big data computing

引用

THEORETICAL COMPUTER SCIENCE 2020年 838卷 195-207页

作者： Gao, Xiangyu Li, Jianzhong Miao, Dongjing Liu, Xianmin Harbin Inst Technol Harbin 150001 Heilongjiang Peoples R China

Due to the limitation on computational power of existing computers, the polynomial time does not work for identifying the tractable problems in big data computing. This paper adopts the sublinear time as the new standard to recognize the tractability in big data computing and study tractable problems in terms of computational complexity theory. The random-access Turing machine is used as the computational model to characterize the problems that are tractable on big data. First, pure tractable class PT is proposed and two important classes within it, PPL and PDP, are studied. The structures of this two pure tractable classes are deeply investigated and they are proved PPLi subset of PPLi+1 , PPL subset of PT and PDPk+1 subset of PDPk subset of PT. Then, pseudo-tractable class PsT is proposed and is partitioned into two classes, PsTR and PsTE, according to preprocessing techniques. The relations among pseudo-tractable classes and other complexity classes are investigated and they are proved that PsT subset of P and Pi'T-Q(0) subset of PsTR. Finally, we show that PPL is closed under DLOGTIME reduction. (C) 2020 Elsevier B.V. All rights reserved.

关键词： big data computing Tractability Sublinear Complexity theory

来源：评论

学校读者我要写书评

暂无评论

Self-organization Scheme of WSNs with Mobile Sensors and Mobile Multiple Sinks for big data computing

引用

KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS 2020年第3期14卷 943-961页

作者： Shin, Ahreum Ryoo, Intae Kim, Seokhoon Kyung Hee Univ Dept Comp Sci & Engn Seoul South Korea Soonchunhyang Univ Dept Comp Software Engn Asan South Korea

With the advent of IoT technology and big data computing, the importance of WSNs (Wireless Sensor Networks) has been on the rise. For energy-efficient and collection-efficient delivery of any sensed data, lots of novel wireless medium access control (MAC) protocols have been proposed and these MAC schemes are the basis of many IoT systems that leads the upcoming fourth industrial revolution. WSNs play a very important role in collecting big data from various IoT sensors. Also, due to the limited amount of battery driving the sensors, energy-saving MAC technologies have been recently studied. In addition, as new IoT technologies for big data computing emerge to meet different needs, both sensors and sinks need to be mobile. To guarantee stability of WSNs with dynamic topologies as well as frequent physical changes, the existing MAC schemes must be tuned for better adapting to the new WSN environment which includes energy-efficiency and collection-efficiency of sensors, coverage of WSNs and data collecting methods of sinks. To address these issues, in this paper, a self-organization scheme for mobile sensor networks with mobile multiple sinks has been proposed and verified to adapt both mobile sensors and multiple sinks to 3-dimensional group management MAC protocol. Performance evaluations show that the proposed scheme outperforms the previous schemes in terms of the various usage cases. Therefore, the proposed self-organization scheme might be adaptable for various computing and networking environments with big data.

关键词： WSN (Wireless Sensor Network) multi-sink IoT (Internet of Things) Mobile Sensor Network big data computing

来源：评论

学校读者我要写书评

暂无评论

Experimenting with big data computing for scaling data quality-aware query processing

引用

EXPERT SYSTEMS WITH APPLICATIONS 2021年 178卷 114858-114858页

作者： Cisneros-Cabrera, Sonia Michailidou, Anna-Valentini Sampaio, Sandra Sampaio, Pedro Gounaris, Anastasios Univ Manchester Dept Comp Sci Manchester Lancs England Aristotle Univ Thessaloniki Dept Informat Thessaloniki Greece Univ Manchester Alliance Manchester Business Sch Manchester Lancs England

Combining query processing techniques with data quality management approaches enables enforcement of quality constraints, such as timeliness, accuracy and completeness, as part of ad-hoc query specification and execution, improving the quality of query results. Despite the emergence of novel data quality processing tools, there is a dearth of studies assessing performance and scalability in the execution of data quality assessment tasks during query processing. This paper reports on an empirical study aiming to investigate the extent to which a big data computing framework (Spark) can offer significant gains in performance and scalability when executing data quality querying tasks over a range of computational platforms including a single commodity multi-core machine and a cluster-based platform for a wide range of workloads. Our results show that substantial performance and scalability gains can be obtained by using optimized data science libraries combined with the parallel and distributed capabilities of big data computing. We also provide guidelines on choosing the appropriate computational infrastructure for executing DQ-aware queries.

关键词： data quality-aware queries big data computing Empirical evaluation

来源：评论

学校读者我要写书评

暂无评论

Temporal, Functional and Spatial big data computing Framework for Large-Scale Smart Grid

引用

IEEE TRANSACTIONS ON EMERGING TOPICS IN computing 2019年第3期7卷 369-379页

作者： Hou, Weigang Ning, Zhaolong Guo, Lei Zhang, Xu Northeastern Univ Sch Comp Sci & Engn Shenyang 110819 Liaoning Peoples R China Dalian Univ Technol Sch Software Dalian 116024 Peoples R China

With the deployment of monitoring devices, the smart grid is collecting large amounts of energy-related data at an unprecedented speed. The smart grid has become data-driven, which necessitates extracting meaningful data from a large dataset. The traditional approach of data extraction improves the computing efficiency in temporal dimension, but it is made for only one task in the smart grid. Moreover, the existing solutions neglect the geographical distribution of computing capacity in a large-scale smart grid. The future large-scale smart grid will run over the internet of energy where the dataset will be sent to a specific destination along power routers hop-by-hop. Consequently, we design a novel temporal, functional and spatial big data computing framework for large-scale smart grid. In functional dimension, we divide every dataset into sub-groups, each of which has data items shared by different tasks. In spatial dimension, we determine which location the power router should be placed to harvest computing resources used for extracting the sub-group of data items. Our method achieves a promising computing efficiency approaching to the optimal solution with 95 percent convergence ratio, and it saves the in-path bandwidth with 81 percent improvement ratio over benchmarks.

关键词： Smart grids big data computing

来源：评论

学校读者我要写书评

暂无评论

Applying SDN based data network on HPC big data computing - Design, Implementation, and Evaluation

Applying SDN based data network on HPC Big Data Computing - ...

引用

IEEE International Conference on big data (big data)

作者： Chen, Hsing-bung Qiao, Zhi Fu, Song Los Alamos Natl Labs HPC Des Grp Los Alamos NM 87545 USA Univ North Texas CSE Dept Denton TX 76203 USA

ISBN: (纸本)9781728108582

Large scale storage data networks are difficult to configure and tricky to maintain [1] [2] [3]. As storage data volumes grow and the pace of change accelerates, it can be a struggle to keep up the integrity of a large scale storage network. Software Defined Networking (SDN) [5] [6] [7] [8] [10] [11] provides a method to centrally configure and manage physical and virtual network devices such as routers, switches, and gateways in HPC datacenter. Current HPC computer cluster and data centers are using homogeneous data network technologies such as the Infiniband network (QDR, FDR, EDR, and HDR). Due to the connector, cable, and bandwidth backward compatibility issues, once an aged HPC computing system was retired, we have to demolish all established computer clusters, data network, and data storage. Eventually we lost all costly investment on the data network and storage. Those current deployment approaches do not provide a feasible and compatible growing path and cannot meet the need for future extreme scale HPC computing systems [4] [12] [13].

关键词： big data computing Convergence Network Deep buffer switch Software Defined Network

来源：评论

学校读者我要写书评

暂无评论

bigNoC: Accelerating big data computing with Application-Specific Photonic Network-on-Chip Architectures

引用

IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 2018年第11期29卷 2402-2415页

作者： Chittamuru, Sai Vineel Reddy Dang, Dharanidhar Pasricha, Sudeep Mahapatra, Rabi Colorado State Univ Dept Elect & Comp Engn Ft Collins CO 80523 USA Texas A&M Univ Dept Comp Sci & Engn College Stn TX 77843 USA

In the era of big data, high performance data analytics applications are frequently executed on large-scale cluster architectures to accomplish massive data-parallel computations. Often, these applications involve iterative machine learning algorithms to extract information and make predictions from large data sets. Multicast data dissemination is one of the major performance bottlenecks for such data analytics applications in cluster computing, as terabytes of data need to be distributed frequently from a single data source to hundreds of computing nodes. To overcome this bottleneck for big data applications, we propose bigNoC, a manycore chip platform with a novel application-specific photonic network-on-chip (PNoC) fabric. bigNoC is designed for big data computing and exploits multicasting in photonic waveguides. For high performance data analytics applications, bigNoC improves throughput by up to 9: 9x while reducing latency by up to 88 percent and energy-per-bit by up to 98 percent over two state-of-the-art PNoC architectures as well as a broadcast-optimized electrical mesh NoC architecture, and a traditional electrical mesh NoC architecture.

关键词： Photonic networks-on-chip photonic channel sharing big data computing multicasting

来源：评论

学校读者我要写书评

暂无评论

Performance Analysis of Storm in a Real-World big data Stream computing Environment 13th

Performance Analysis of Storm in a Real-World Big Data Strea...

引用

13th European-Alliance-for-Innovation (EAI) International Conference on Collaborative computing - Networking, Applications and Worksharing (CollaborateCom)

作者： Yan, Hongbin Sun, Dawei Gao, Shang Zhou, Zhangbing China Univ Geosci Sch Informat Engn Beijing 100083 Peoples R China Deakin Univ Sch Informat Technol Geelong Vic 3216 Australia

ISBN: (纸本)9783030009168;9783030009151

As an important distributed real-time computation system, Storm has been widely used in a number of applications such as online machine learning, continuous computation, distributed RPC, and more. Storm is designed to process massive data streams in real time. However, there have been few studies conducted to evaluate the performance characteristics clusters in Storm. In this paper, we analyze the performance of a Storm cluster mainly from two aspects, hardware configuration and parallelism setting. Key factors that affect the throughput and latency of the Storm cluster are identified, and the performance of Storm's fault-tolerant mechanism is evaluated, which help users use the computation system more efficiently.

关键词： Storm Performance analysis Stream computing big data computing big data

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：