检索结果-内蒙古大学图书馆

IEEE International Conference on Big Data (Big Data)

作者： Misev, Dimitar Baumann, Peter Bellos, Dimitris Wiehle, Stefan Jacobs Univ Comp Sci & Elect Engn Bremen Germany Cloudeo AG Munich Germany German Aerosp Ctr DLR Remote Sensing Technol Inst SAR Signal Proc Bremen Germany

ISBN: (纸本)9781728108582

The European Copernicus programme generates massive amounts of Earth Observation (EO) data, with the goal of improving our environmental understanding and management on local, regional, and global level. An initiative of this magnitude comes with a set of challenges, especially pertaining to effective service management once the huge volumes of raster data files are distributed across open data centers and commercial companies. The BigDataCube project responds to this challenge through the concept of federated, analysis-ready datacubes, exposed via the open OGC geo standard interfaces 14 interoperable access and processing, WMS, WCS, and WCPS. Such federated datacube services have been established in the project between the public German Copernicus hub, CODE-DE, and a commercial cloud provider, cloudeo AG, altogether offering access to more than 500 TB. Both data pools are federated in a location-transparent manner, establishing a common information space where users can query and combine datacubes without knowing their location of storage. In this nucleus of a growing federation recently the Alfred Wegener maritime research institute has joined, and further data centers are in the line. The spectrum of functionality available is showcased through several realistic use cases, including a value-adding Sentinel-I SAR product on sea state and wind speed in the North Sea. The platform used is the pioneer datacube engine and OGC reference implementation, rasdaman. We report on the outcomes of the project, lessons learned, and further work foreseen in this area.

关键词： datacube array database rasdaman security federation WCS WCPS SQL/MDA

来源：评论

学校读者我要写书评

暂无评论

Accelerating Scientific Algorithms in array databases with GPUs 5

Accelerating Scientific Algorithms in Array Databases with G...

引用

IEEE International Conference on Big Data (IEEE Big Data)

作者： Marcin, Simon Csillaghy, Andre Univ Appl Sci North Western Switzerland FHNW Inst Technol 4D CH-5210 Windisch Switzerland

ISBN: (纸本)9781538627150

In data science, interactive data analysis allows to very efficiently interpret what information is in the data. However, with increasing amounts of data available, the data crunching and data presentation becomes a more complex and resource-demanding task. array databases allow to mitigate this difficulty by distributing the workload on a cluster. But there are two major challenges. Firstly, they rely on CPUs to process the data. Secondly, it is difficult to represent complex scientific algorithms in terms of native database operations. A way to integrate GPU accelerated algorithms is needed. In this paper, we study and test how to run scientific algorithms on GPUs entirely as native array database operators. We present the implementation of a specific scientific algorithm as exemplary use case. The computation of the Differential Emission Measure (DEM) calculates the distribution of plasma density in the solar corona at specific temperatures. DEM uses image series of the NASA spacecraft Solar Dynamic Observatory (SDO). We stack these images in a 3-dimensional array and process the data on GPUs, distributed in an array database cluster. For our algorithm, we measure a decrease of the overall runtime by a factor of 10700. We discuss different strategies used to minimize the overhead of GPUs and parameters used to scale the array database to a cluster.

关键词： AIA array database CUDA DEM GPU User Defined Operator SciDB

来源：评论

学校读者我要写书评

暂无评论

Architecture and prototypical implementation of a semantic querying system for big Earth observation image bases

引用

EUROPEAN JOURNAL OF REMOTE SENSING 2017年第1期50卷 452-463页

作者： Tiede, Dirk Baraldi, Andrea Sudmanns, Martin Belgiu, Mariana Lang, Stefan Univ Salzburg Dept Geoinformat Z GIS Schillerstr 30 A-5020 Salzburg Austria Univ Naples Federico II Dept Agr & Food Sci Portici Italy

Spatiotemporal analytics of multi-source Earth observation (EO) big data is a pre-condition for semantic content-based image retrieval (SCBIR). As a proof of concept, an innovative EO semantic querying (EO-SQ) subsystem was designed and prototypically implemented in series with an EO image understanding (EO-IU) subsystem. The EO-IU subsystem is automatically generating ESA Level 2 products (scene classification map, up to basic land cover units) from optical satellite data. The EO-SQ subsystem comprises a graphical user interface (GUI) and an array database embedded in a client server model. In the array database, all EO images are stored as a space-time data cube together with their Level 2 products generated by the EO-IU subsystem. The GUI allows users to (a) develop a conceptual world model based on a graphically supported query pipeline as a combination of spatial and temporal operators and/or standard algorithms and (b) create, save and share within the client-server architecture complex semantic queries/decision rules, suitable for SCBIR and/or spatiotemporal EO image analytics, consistent with the conceptual world model.

关键词： Big data Earth observation Level 2 product spatiotemporal objects and events array database semantic content-based image retrieval

来源：评论

学校读者我要写书评

暂无评论

Visual Analytics with Unparalleled Variety Scaling for Big Earth Data 5

Visual Analytics with Unparalleled Variety Scaling for Big E...

引用

IEEE International Conference on Big Data (IEEE Big Data)

作者： Yu, Lina Rilee, Michael L. Pan, Yu Zhu, Feiyu Kuo, Kwo-Sen Yu, Hongfeng Univ Nebraska Lincoln NE 68588 USA NASA GSFC Greenbelt MD USA Rilee Syst Technol Derwood MD USA Bayes LLC Bowie MD USA Univ Maryland College Pk MD 20742 USA

ISBN: (纸本)9781538627150

We have devised and implemented a key technology, SpatioTemporal Adaptive-Resolution Encoding (STARE), in an array database management system, i.e. SciDB, to achieve unparalleled variety scaling for Big Earth Data, enabling rapid-response visual analytics. STARE not only serves as a unifying data representation homogenizing diverse varieties of Earth Science Datasets, but also supports spatiotemporal data placement alignment of these datasets to optimize a major class of Earth Science data analyses, i.e. those requiring spatiotemporal coincidence. Using STARE, we tailor a data partitioning and distribution strategy for the data access patterns of our scientific analysis, leading to optimal use of distributed resources. With STARE, rapid-response visual analytics are made possible through a high-level query interface, allowing geoscientists to perform data exploration visually, intuitively and interactively. We envision a system based on these innovations to relieve geoscientists of most laborious data management chores so that they may focus better on scientific issues and investigations. A significant boost in scientific productivity may thus be expected. We demonstrate these advantages with a prototypical system including comparisons to alternatives.

关键词： STARE SciDB array database variety data analysis load balancing indexing GIS

来源：评论

学校读者我要写书评

暂无评论

VisualCloud Demonstration: A DBMS for Virtual Reality 17

VisualCloud Demonstration: A DBMS for Virtual Reality

引用

ACM International Conference on Management of Data

作者： Haynes, Brandon Minyaylov, Artem Balazinska, Magdalena Ceze, Luis Cheung, Alvin Univ Washington Seattle WA 98195 USA

ISBN: (纸本)9781450341974

We demonstrate VisualCloud, a database management system designed to efficiently ingest, store, and deliver virtual reality (VR) content at scale. VisualCloud targets both live and prerecorded spherical panoramic (a.k.a. 360 degrees) VR videos. It persists content as a multidimensional array that utilizes both dense (e.g., space and time) and sparse (e.g., bitrate) dimensions. VisualCloud uses orientation prediction to reduce data transfer by degrading out-of-view portions of the video. Content delivered through VisualCloud requires up to 60% less bandwidth than existing methods and scales to many concurrent connections. This demonstration will allow attendees to view both live and prerecorded VR video content served through VisualCloud. Viewers will be able to dynamically adjust tuning parameters (e.g., bitrates and path prediction) and observe changes in visual fidelity.

关键词： virtual reality in-memory database array database

来源：评论

学校读者我要写书评

暂无评论

Spatio-temporal change detection from multidimensional arrays: Detecting deforestation from MODIS time series

引用

ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING 2016年 117卷 227-236页

作者： Lu, Meng Pebesma, Edzer Sanchez, Alber Verbesselt, Jan Univ Munster Inst Geoinformat Heisenbergstr 2 D-48149 Munster Germany Natl Inst Space Res INPE Av Astronautas 1758 BR-12227007 Sao Jose Dos Campos Brazil Wageningen Univ Lab Geoinformat Sci & Remote Sensing Droevendaalsesteeg 3 NL-6708 PB Wageningen Netherlands

Growing availability of long-term satellite imagery enables change modeling with advanced spatio-temporal statistical methods. Multidimensional arrays naturally match the structure of spatio-temporal satellite data and can provide a clean modeling process for complex spatio-temporal analysis over large datasets. Our study case illustrates the detection of breakpoints in MODIS imagery time series for land cover change in the Brazilian Amazon using the BFAST (Breaks For Additive Season and Trend) change detection framework. BFAST includes an Empirical Fluctuation Process (EFP) to alarm the change and a change point time locating process. We extend the EFP to account for the spatial autocorrelation between spatial neighbors and assess the effects of spatial correlation when applying BFAST on satellite image time series. In addition, we evaluate how sensitive EFP is to the assumption that its time series residuals are temporally uncorrelated, by modeling it as an autoregressive process. We use arrays as a unified data structure for the modeling process, R to execute the analysis, and an array database management system to scale computation. Our results point to BFAST as a robust approach against mild temporal and spatial correlation, to the use of arrays to ease the modeling process of spatio-temporal change, and towards communicable and scalable analysis. (C) 2016 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

关键词： BFAST Time series analysis Spatial correlation Temporal correlation array database Spatio-temporal change modeling

来源：评论

学校读者我要写书评

暂无评论

Integration and management of massive remote-sensing data based on GeoSOT subdivision model

引用

JOURNAL OF APPLIED REMOTE SENSING 2016年第3期10卷

作者： Li, Shuang Cheng, Chengqi Chen, Bo Meng, Li Peking Univ Inst Remote Sensing & Geog Informat Syst 5 Yiheyuan Rd Beijing 100871 Peoples R China Peking Univ Coll Engn 5 Yiheyuan Rd Beijing 100871 Peoples R China

Owing to the rapid development of earth observation technology, the volume of spatial information is growing rapidly;therefore, improving query retrieval speed from large, rich data sources for remote-sensing data management systems is quite urgent. A global subdivision model, geographic coordinate subdivision grid with one-dimension integer coding on 2(n)-tree, which we propose as a solution, has been used in data management organizations. However, because a spatial object may cover several grids, ample data redundancy will occur when data are stored in relational databases. To solve this redundancy problem, we first combined the subdivision model with the spatial array database containing the inverted index. We proposed an improved approach for integrating and managing massive remote-sensing data. By adding a spatial code column in an array format in a database, spatial information in remote-sensing meta-data can be stored and logically subdivided. We implemented our method in a Kingbase Enterprise Server database system and compared the results with the Oracle platform by simulating worldwide image data. Experimental results showed that our approach performed better than Oracle in terms of data integration and time and space efficiency. Our approach also offers an efficient storage management system for existing storage centers and management systems. (C) 2016 Society of Photo-Optical Instrumentation Engineers (SPIE)

关键词： geographic coordinate subdivision grid with one-dimension integer coding on 2n-tree remote-sensing data array database inverted index

来源：评论

学校读者我要写书评

暂无评论

Addressing the Big-Earth-Data Variety Challenge with the Hierarchical Triangular Mesh 4

Addressing the Big-Earth-Data Variety Challenge with the Hie...

引用

4th IEEE International Conference on Big Data (Big Data)

作者： Rilee, Michael L. Kuo, Kwo-Sen Clune, Thomas Oloso, Amidu Brown, Paul G. Yu, Hongfeng NASA Goddard Space Flight Ctr Greenbelt MD 20771 USA Rilee Syst Technol Derwood MD USA Bayesics LLC Bowie MD USA Univ Maryland College Pk MD 20742 USA SSAI Greenbelt MD USA Paradigm4 Inc Waltham MA USA Univ Nebraska Lincoln NE USA

ISBN: (纸本)9781467390057

We have implemented an updated Hierarchical Triangular Mesh (HTM) as the basis for a unified data model and an indexing scheme for geoscience data to address the variety challenge of Big Earth Data. In the absence of variety, the volume challenge of Big Data is relatively easily addressable with parallel processing. The more important challenge in achieving optimal value with a Big Data solution for Earth Science (ES) data analysis, however, is being able to achieve good scalability with variety. With HTM unifying at least the three popular data models, i.e. Grid, Swath, and Point, used by current ES data products, data preparation time for integrative analysis of diverse datasets can be drastically reduced and better variety scaling can be achieved. HTM is also an indexing scheme, and when applied to all ES datasets, data placement alignment (or co-location) on the shared nothing architecture, which most Big Data systems are based on, is guaranteed and better performance is ensured. With HTM most geospatial set operations become integer interval operations with further performance advantages.

关键词： SciDB HTM array database shared nothing architecture variety DAAC remote sensing data analysis data fusion load balancing indexing geographic metadata GIS

来源：评论

学校读者我要写书评

暂无评论

Skew-Aware Join Optimization for array databases 15

Skew-Aware Join Optimization for Array Databases

引用

ACM SIGMOD International Conference on Management of Data

作者： Duggan, Jennie Papaemmanouil, Olga Battle, Leilani Stonebraker, Michael Northwestern Univ Evanston IL 60208 USA Brandeis Univ Waltham MA 02254 USA MIT Cambridge MA 02139 USA

ISBN: (纸本)9781450327589

Science applications are accumulating an ever-increasing amount of multidimensional data. Although some of it can be processed in a relational database, much of it is better suited to array-based engines. As such, it is important to optimize the query processing of these systems. This paper focuses on efficient query processing of join operations within an array database. These engines invariably "chunk" their data into multidimensional tiles that they use to efficiently process spatial queries. As such, traditional relational algorithms need to be substantially modified to take advantage of array tiles. Moreover, most n-dimensional science data is unevenly distributed in array space because its underlying observations rarely follow a uniform pattern. It is crucial that the optimization of array joins be skew-aware. In addition, owing to the scale of science applications, their query processing usually spans multiple nodes. This further complicates the planning of array joins. In this paper, we introduce a join optimization framework that is skew-aware for distributed joins. This optimization consists of two phases. In the first, a logical planner selects the query's algorithm (e.g., merge join), the granularity of the its tiles, and the reorganization operations needed to align the data. The second phase implements this logical plan by assigning tiles to cluster nodes using an analytical cost model. Our experimental results, on both synthetic and real-world data, demonstrate that this optimization framework speeds up array joins by up to 2.5X in comparison to the baseline.

关键词： distributed computation skew array database join optimization

来源：评论

学校读者我要写书评

暂无评论

On the Efficient Evaluation of array Joins 3

On the Efficient Evaluation of Array Joins

引用

IEEE International Conference on Big Data

作者： Baumann, Peter Merticariu, Vlad Jacobs Univ D-28759 Bremen Germany Rasdaman GmbH D-28759 Bremen Germany

ISBN: (纸本)9781479999255

array databases close a gap in the database ecosystem by adding modeling, storage, and processing support on multi-dimensional arrays. Declarative queries provide processing of arrays of regularly massive size, such as Tera- to Petabyte datacubes, while allowing internal degrees of freedom in partitioning the large arrays into tractable subarrays. Among the important new operations is the array Theta-Join, such as overlaying two images. Evaluation of such joins is complicated by the fact that the participating arrays likely do not align in their partitioning schemes. This can lead to inefficient multiple reads of sub-arrays. We introduce array joins and present an efficient way of pairing corresponding sub-arrays. As a by product, this technique delivers information on optimal data placement for parallel join evaluation. The method is implemented in the array DBMS rasdaman which is in operational use at data centers and mapping agencies.

关键词： array database scientific database join rasdaman

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：