检索结果-内蒙古大学图书馆

Data Analysis and parallel database Construction of Cloud Platform

Data Analysis and Parallel Database Construction of Cloud Pl...

2019 2nd International Conference on Mechanical Engineering,Industrial Materials and Industrial Electronics(MEIMIE 2019)

作者： Yan Wei Information Centre of HuaiHai Institute of Technology

With the arrival of the era of big data, people have higher and higher requirements for data processing. Clustering analysis is the key to data processing, which can facilitate people to describe and understand data objects, and collect the intrinsic information of large data hidden in cloud platforms. It is urgent to study a reasonable parallel clustering method for large data in cloud platforms. The Internet has taken root in people's lives and spread to every corner. The development of scientific research informationization makes people deal with things more and more conveniently and synchronization more and more timely. However, the emergence of new things will always bring new problems and usher in a new pattern. When we use the Internet for information transmission and interaction and it will produce a lot of data information. However, with the rapid growth of data storage and transmission, the existing traditional computer mechanisms and technologies can no longer meet the current computing needs, which prompt people to seek new technologies to process and analyze large amounts of data and extract potential information and value from data.

关键词： Cloud Platform Data Analysis parallel database Construction Hadoop

来源：评论

学校读者我要写书评

暂无评论

A New Oracle parallel database Optimization Model

A New Oracle Parallel Database Optimization Model

引用

2017 4th ICMIBI International Conference on Training, Education, and Management (ICMIBI-TEM 2017)

作者： Lei Yu Nanjing Institute of Technology

This manuscript proposes the new Oracle parallel database optimization model. There are many factors that affect the operational efficiency of the Oracle databases, which together constrain the performance of the general Oracle databases. Specifically, the influencing factors are as follows. The use of the transaction can make developers work more easily. In basic Java applications use transactions that can reduce the possibility of a basic multi-user environment data error. However, transaction control mechanism is very complex and its operation requires additional processing and the processing will degrade the performance of the system. Using stored procedures and storage functions, the application does not have to include multiple SQL statements of network operations to perform database server operations, but simply call stored and stored functions, and only on the network call the name of the process and output the results. The proposed model can deal with the mentioned challenges well which will be verified in the future research.

关键词： Oracle parallel database Optimization model Novel methodology

来源：评论

学校读者我要写书评

暂无评论

A Provenance Storage Method Based On parallel database 2

A Provenance Storage Method Based On Parallel Database

引用

2nd International Conference on Information Science and Control Engineering

作者： Wang, Zhuo Xu, Yao Suo, Bo Wang, Zhong Xu, Jian Chen, Qun Li, Zhanhuai NPU Sch Comp Xian Peoples R China

ISBN: (纸本)9781467368506

When faced with dirty data fused from different sources, modern data management generally trace origins using data provenance technical to repair it fundamentally. Unfortunately, the state-of-art data provenance approaches can only deal with small amount of data using annotation or inverse process. What's worse, these approaches just work under stand-alone mode, resulting in low efficiencies. In this paper, we raise a distributed provenance storage strategy DPSM under the environment of parallel data-base combined with the small-amount-storage advantage of "Provenance Tree". The model optimizes both query efficiencies and storage cost by using a detached method to store different provenance information.

关键词： data provenanc provenance tree parallel database distributed storage component

来源：评论

学校读者我要写书评

暂无评论

Accelerating Columnar Storage Based on Asynchronous Skipping Strategy

引用

BIG DATA RESEARCH 2023年 31卷

作者： Li, Wenhai Yang, Zheng Deng, Lingfeng Cheng, Zhiling Wen, Weidong He, Yanxiang Wuhan Univ Wuhan 430072 Hubei Peoples R China Pengcheng Lab Shenzhen Guangdong Peoples R China

Many database applications, such as OnLine Analytical Processing (OLAP), web-based information extraction or scientific computation, need to select a subset of fields based on several user-defined filters. Developers of these applications require effective assembly methods for on-demand filtering and aggregation, which raises new challenges in deploying parallel computing components on top of columnar storage. To efficiently generate qualified records, an asynchronous skipping strategy is presented to speed up filtering and decoding in the column-based storage. Concentrating on filtering-pushdown in parallel analytical workloads, we offer in-depth analysis on record assembly. We highlight the bottleneck of traditional record-wise assembling methods in the cases of evaluating analytical tasks on a nested schema. With a concurrent queue structure, an asynchronous skipping strategy is presented to evaluate column scan separately by a software pipeline involving an optionally different set of threads. We show how to intensively read the sequential blocks of each column, and how to effectively eliminate invalid payloads by integrating filtering-pushdown in an asynchronous I/O stack. We implement a columnar storage supporting filtering-pushdown in nested schema. Our experiments are conducted on a de-facto standard benchmark using both variant-selectivity scans and ad-hoc queries. The results revealed that in parallel I/O-intensive workloads, our implementation improved the I/O performance of the state-of-the arts by 1.3X similar to 2.7X. Coupling the asynchronous strategy with filtering-pushdown, our implementation remarkably outperforms its competitors with heavyweight coding workloads on both HDD and SSD.(c) 2022 Elsevier Inc. All rights reserved.

关键词： Columnar storage parallel database Optimization

来源：评论

学校读者我要写书评

暂无评论

TIIS: A Two-level Inverted-index Scheme for Large-scale Data Processing in the parallel database System

TIIS: A Two-level Inverted-index Scheme for Large-scale Data...

引用

International Conference on Mechatronic Sciences, Electric Engineering and Computer (MEC)

作者： Yu, Lei Fu, Ge Tan, Huaiyuan Jin, Yan Zhang, Hong Liu, Xinran Xiang, Xiaojia Chinese Acad Sci Inst Comp Technol Beijing Peoples R China Natl Comp Network Emergency Response Tech Team Co Beijing Peoples R China Changan Commun Technol Co Ltd Beijing Peoples R China

ISBN: (纸本)9781479925650

Based on Service-Oriented Architecture, an inexpensive solution, parallel database middleware gather the standalone database instance to provide users with highly scalable relational data management platform. However, with the advent of the era of large-scale data, such platform has posed a serious challenge in the context of text data retrieval. Motivated by this observation, a parallel database middleware based on semi-structure data is firstly designed to support text retrieval. Then, a two-level inverted-index scheme called TIIS is designed for full-text query. The advantages of TIIS are that it can quickly locate the result data from large cluster distributed database storing large-scale data, and it can greatly reduce the network I/O and disk I/O. Experimental results show that, comparing with Hive using Hadoop Distributed File System in same environment of hardware, our system performs typical TPC-H data analysis, consuming of full-text query is declined by 90% on 2GB commercial data in average.

关键词： parallel database query optimization full text retrieval inverted index

来源：评论

学校读者我要写书评

暂无评论

引用

INFORMATION SYSTEMS 2020年 88卷 101455-000页

作者： Kim, Taewoo Li, Wenhai Behm, Alexander Cetindil, Inci Vernica, Rares Borkar, Vinayak Carey, Michael J. Li, Chen Univ Calif Irvine Irvine CA 92697 USA Wuhan Univ Wuhan Hubei Peoples R China

Similarity query processing is becoming increasingly important in many applications such as data cleaning, record linkage, Web search, and document analytics. In this paper we study how to provide end-to-end similarity query support natively in a parallel database system. We discuss how to express a similarity predicate in its query language, how to build indexes, how to answer similarity queries (selections and joins) efficiently in the runtime engine, possibly using indexes, and how to optimize similarity queries. One particular challenge is how to incorporate existing similarity join algorithms, which often require a series of steps to achieve a high efficiency, including collecting token frequencies, finding matching record id pairs, and reassembling result records based on id pairs. We present a novel approach that uses existing runtime operators to implement such complex join algorithms without reinventing the wheel;doing so positions the system to automatically benefit from future improvements to those operators. The approach includes a technique to transform a similarity join plan into an efficient operator-based physical plan during query optimization by using a template expressed largely in the system's user-level query language;this technique greatly simplifies the specification of such a transformation rule. We use Apache AsterixDB, a parallel Big Data management system, to illustrate and validate our techniques. We conduct an experimental study using several large, real datasets on a parallel computing cluster to assess the similarity query support. We also include experiments involving three other parallel systems and report the efficacy and performance results. (C) 2019 Elsevier Ltd. All rights reserved.

关键词： Similarity query parallel database Optimization

来源：评论

学校读者我要写书评

暂无评论

Analysis and Research on the Balanced Distribution of the Network-Based Data in parallel database

Analysis and Research on the Balanced Distribution of the Ne...

引用

4th International Conference on Machine Vision (ICMV) - Computer Vision and Image Analysis - Pattern Recognition and Basic Technologies

作者： He, JunHua Huangshi Inst Technol Sch Comp Sci & Technol Huangshi Hubei Peoples R China

ISBN: (纸本)9780819490261

The rapid development of parallel computer systems, making parallel operating environment gradually mature and widely used in scientific computing and research in many fields, thus parallel database of research becomes more and more attention and research has become an important database field of study. This network-based parallel cluster of characteristics and the current parallel computer system new trends, analyzes the network parallel clusters of workstations, parallel database data skew problem in data distribution characteristics of the environment is proposed with the ability to adapt to the dynamic data balanced distribution programs.

关键词： Network parallel database data distribution

来源：评论

学校读者我要写书评

暂无评论

Cost Effective Load-Balancing Approach for Range-Partitioned Main-Memory Resident Data 29th

Cost Effective Load-Balancing Approach for Range-Partitioned...

引用

29th International Conference on database and Expert Systems Applications (DEXA)

作者： Belayadi, Djahida Hidouci, Khaled-Walid Bellatreche, Ladjel Ordonez, Carlos Ecole Natl Super Informat LCSI Lab Algiers Algeria LIAS ISAE ENSMA F-86960 Futuroscope France Univ Houston Houston TX USA

ISBN: (纸本)9783319988122;9783319988115

Due to the availability of larger RAM capacity, there is a new trend bringing parallel main memory database systems with higher performance, compared to traditional DBMSs. In parallel database systems the most critical aspect is data partitioning, which significantly impacts query processing time. Specifically, unbalanced data partitioning introduces data skew, which ends up decreasing query performance if not managed. In this work, we focus on optimizing range queries, widely used in P2P, decision support systems and spatio-temporal databases. We improve the communication complexity of the state-of-the-art previous algorithm based on skip graphs, which required O(log p) messages between 2 nodes to rebalance load, resulting in a high complexity O(p log p) to rebalance load on the p nodes. With such high cost in mind, we propose to create a global view of data distribution among all processing nodes and database clients. Our main contribution is the Approximate Partitioning Vector (APV), which provides a global approximate view of data distribution to both processing nodes and database clients. A new data balancing algorithm, following a ring topology, reduces communication to 2 messages per node pair, resulting in O(1) communication complexity per node pair and O(p) globally among the p nodes. Experiments analyze the tradeoff between adjusting load balance and query performance.

关键词： Data skew Load-balancing parallel database Range-partitioning Main memory databases P2P databases parallel query processing

来源：评论

学校读者我要写书评

暂无评论

Advantages and Disadvantages of Introducing Third-party Monitoring Mechanism into Students' Physique

Advantages and Disadvantages of Introducing Third-party Moni...

引用

作者： Yunlin Deng Shengjun Li Jiangxi Vocational College of Ahead Software

The results of many surveys conducted by the National Committee of the Political Consultative Conference of the People's Republic of China(CPPCC) and the Ministry of Education(MOE) and other ministries for 25 consecutive years show that some important indicators of the physical fitness of Chinese adolescents are on the decline,which is *** physical health of students in our country has attracted much attention from the Party,the government and the *** grasp the healthy development of students' physical health,it is necessary to integrate the quality of students' physical health testing into effective *** present,there are some problems in the process of monitoring adolescents' physical health,such as inadequate monitoring,formalism of monitoring,low quality of physical health testing and so *** order to improve the current situation of monitoring mechanism,improve the enthusiasm of students' monitoring,and improve students' physical health,the government,society and other relevant departments play an important role in the study of student physical health monitoring *** government,society,third-party monitoring institutions,schools,individual students and family-related persons are formed into a "six-in-one" monitoring *** above contents all reflect the importance of the third-party monitoring mechanism to improve the current health of students.

关键词： Cloud platform Data analysis parallel database Construction Hadoop

来源：评论

学校读者我要写书评

暂无评论

Towards Interactive Analytics and Visualization on One Billion Tweets 16

Towards Interactive Analytics and Visualization on One Billi...

引用

24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL GIS)

作者： Jia, Jianfeng Li, Chen Zhang, Xi Li, Chen Carey, Michael J. Su, Simon Univ Calif Irvine Dept Comp Sci Irvine CA 92717 USA US Army Res Lab Arlington VA USA

ISBN: (纸本)9781450345897

We present a system called "Cloudberry" that allows users to interactively query, analyze, and visualize large amounts of data with temporal, spatial, and textual dimensions. As a general-purpose full-stack solution, it has a friendly UI, intelligent middleware, and a powerful big data management backend running Apache AsterixDB to enable big data analytics and visualization. We will demonstrate the system using Twitter data on a computer cluster.

关键词： Spatial-temporal-text data data analytics visualization social networks parallel database Apache AsterixDB

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：