检索结果-内蒙古大学图书馆

10th International Workshop on Analytics for Big Geospatial data

作者： Werner, Martin Li, Hao Tech Univ Munich Big Geospatial Data Management Munich Germany

ISBN: (纸本)9781450395311

The last decade witnesses a fast development in geospatial application of artificial intelligence (GeoAI). However, due to the misalignment with wider computer science progresses, the geospatial community, for a long time, keeps working with powerful and over-sophisticated tools and software, whose functionality goes far beyond the actual basic need of GeoAI tasks. This fact, to a certain extent, hinders our steps towards establishing future sustainable and replicable GeoAI models. In this paper, we aim to address this challenge by introducing an efficient big data framework based on the modern HDF5 technology, called AtlasHDF, in which we designed lossless data mappings (immediate mapping and analysis-ready mapping) from OpenStreetMap (OSM) vector data into a single HDF5 data container to facilitate fast and flexible GeoAI applications learnt from OSM data. Since the HDF5 is included as a default dependency in most GeoAI and high performance computing (HPC) environments, the proposed AtlasHDF provides a cross-platformm and single-techonology solution of handling heterogeneous big geodata for GeoAI. As a case study, we conducted a comparative analysis of the AtlasHDF framework with three commonly-used data formats (i.e., PBF, Shapefile and GeoPackage) using the latest OSM data from the city of Berlin (Germany), then elaborated on the advantages of each data format w.r.t file size, querying, rending, dependency, data extendability. Given a wide range of GeoAI tasks that can potentially benefit from our framework, our future work will focus on extending the framework to heterogeneous big geodata (vector and raster) to support seamless and fast data integration without any geospatial software dependency until the training stage of GeoAI. A reference implementation of the framework developed in this paper is provided to the public at: https://***/tumbgd/hdf4water.

关键词： GeoAI hierarchical data format Immediate mapping Big data OpenStreetMap

来源：评论

学校读者我要写书评

暂无评论

A Violently Tornadic Supercell Thunderstorm Simulation Spanning a Quarter-Trillion Grid Volumes: Computational Challenges, I/O Framework, and Visualizations of Tornadogenesis

引用

ATMOSPHERE 2019年第10期10卷 578-578页

作者： Orf, Leigh Univ Wisconsin CIMSS Madison WI 53706 USA

Tornadoes remain an active subject of observational and numerical research due to the damage and fatalities they cause worldwide as well as poor understanding of their behavior, such as what processes result in their genesis and what determines their longevity and morphology. A numerical model executed on a supercomputer run at high resolution can serve as a powerful tool for exploring the rapidly evolving tornado-scale features within a simulated storm, but saving large amounts data for meaningful analysis can result in unacceptably slow model performance, an unwieldy amount of saved data, and saved data spread across millions of files. In this paper, a system for efficiently saving and managing hundreds of terabytes of compressed model output is described to support a supercomputer simulation of a tornadic supercell thunderstorm. The challenges of managing a simulation spanning over a quarter-trillion grid volumes across the Blue Waters supercomputer are also described. The simulated supercell produces a long-track EF5 tornado, and the near-tornado environment is described during tornadogenesis, where single upward-growing vortex undergoes several vortex mergers before transitioning into a multiple-vortex tornado.

关键词： tornadogenesis numerical simulation volume rendering visualization supercell thunderstorm hierarchical data format ZFP compression VAPOR3

来源：评论

学校读者我要写书评

暂无评论

FIVE GOOD REASONS TO USE THE hierarchical data format

引用

COMPUTING IN SCIENCE & ENGINEERING 2010年第5期12卷 84-90页

作者： Poinot, Marc Onera the French Aerosp. Labs. France

Practical experience migrating to HDF for a numerical simulation library illustrates the storage system's benefits as well as how numerical simulation codes contribute to an open system. In this article describe the hierarchical data format the computational fluid dynamics (CFD) data storage for the CFD General Notation Standard (CGNS).

关键词： CFD General Notation Standard CFD General Notation System (CGNS) Computational Fluid Dynamics HDF hierarchical data format (HDF) Numerical simulation computational fluid dynamic data storage computational fluid dynamics hierarchical data format mathematics computing numerical simulation codes numerical simulation library open system open systems open systems physics computing scientific computing storage system tree data structure tree data structures

来源：评论

学校读者我要写书评

暂无评论

Mechanic: The MPI/HDF code framework for dynamical astronomy

引用

NEW ASTRONOMY 2015年 34卷 98-107页

作者： Slonina, Mariusz Gozdziewski, Krzysztof Migaszewski, Cezary Nicholas Copernicus Univ Torun Ctr Astron PL-87100 Torun Poland

We introduce the Mechanic, a new open-source code framework. It is designed to reduce the development effort of scientific applications by providing unified API (Application Programming Interface) for configuration, data storage and task management. The communication layer is based on the well-established Message Passing Interface (MPI) standard, which is widely used on variety of parallel computers and CPU-clusters. The data storage is performed within the hierarchical data format (HDF5). The design of the code follows core-module approach which allows to reduce the user's codebase and makes it portable for single- and multi-CPU environments. The framework may be used in a local user's environment, without administrative access to the cluster, under the PBS or Slurm job schedulers. It may become a helper tool for a wide range of astronomical applications, particularly focused on processing large data sets, such as dynamical studies of long-term orbital evolution of planetary systems with Monte Carlo methods, dynamical maps or evolutionary algorithms. It has been already applied in numerical experiments conducted for Kepler-11 (Migaszewski et al., 2012) and vOctantis planetary systems (Gozdziewski et al., 2013). In this paper we describe the basics of the framework, including code listings for the implementation of a sample user's module. The code is illustrated on a model Hamiltonian introduced by (Froeschle et al., 2000) presenting the Arnold diffusion. The Arnold web is shown with the help of the MEGNO (Mean Exponential Growth of Nearby Orbits) fast indicator (Gozdziewski et al., 2008a) applied onto symplectic SABA(n) integrators family (Laskar and Robutel, 2001). (C) 2014 Elsevier B.V. All rights reserved.

关键词： Numerical methods Task management Message Passing Interface hierarchical data format

来源：评论

学校读者我要写书评

暂无评论

Optimization of PACS data Persistency Using Indexed hierarchical data

引用

JOURNAL OF DIGITAL IMAGING 2014年第3期27卷 297-308页

作者： Prado, Thiago C. de Macedo, Douglas D. J. Dantas, M. A. R. von Wangenheim, Aldo INE Dept Informat & Stat Florianopolis SC Brazil Univ Fed Santa Catarina Postgrad Program Knowledge Engn & Management PPGE Florianopolis SC Brazil Univ Fed Santa Catarina Dept Informat & Estat Sala 320 BR-88040900 Florianopolis SC Brazil

We present a new approach for the development of a data persistency layer for a Digital Imaging and Communications in Medicine (DICOM)-compliant Picture Archiving and Communications Systems employing a hierarchical database. Our approach makes use of the HDF5 hierarchical data storage standard for scientific data and overcomes limitations of hierarchical databases employing inverted indexing for secondary key management and for efficient and flexible access to data through secondary keys. This inverted indexing is achieved through a general purpose document indexing tool called Lucene. This approach was implemented and tested using real-world data against a traditional solution employing a relational database, in various store, search, and retrieval experiments performed repeatedly with different sizes of DICOM datasets. Results show that our approach outperforms the traditional solution on most of the situations, being more than 600 % faster in some cases.

关键词： DICOM hierarchical data format PACS data Indexing

来源：评论

学校读者我要写书评

暂无评论

h5ebsd: an archival data format for electron back-scatter diffraction data sets

引用

INTEGRATING MATERIALS AND MANUFACTURING INNOVATION 2014年第1期3卷 1-12页

作者： Jackson, Michael A. Groeber, Michael A. Uchic, Michael D. Rowenhorst, David J. De Graef, Marc BlueQuartz Software Springboro OH 45066 USA US Air Force Res Lab Mat & Mfg Directorate Wright Patterson AFB OH 45433 USA US Naval Res Lab Washington DC 20375 USA Dept Mat Sci & Engn 5000 Forbes Ave Pittsburgh PA 15213 USA

We present an archival format for electron back-scatter diffraction ( EBSD) data based on the HDF5 scientific file format. We discuss the differences between archival and data work flow file formats, and present details of the archival file layout for the implementation of h5ebsd, a vendor-neutral EBSD- HDF5 format. Information on sample and external reference frames can be included in the archival file, so that the data is internally consistent and complete. We describe how the format can be extended to include additional experimental modalities, and present some thoughts on the interactions between working files and archival files. The complete file specification as well as an example h5ebsd formatted data set are made available to the reader.

关键词： Electron back-scatter diffraction hierarchical data format HDF5

来源：评论

学校读者我要写书评

暂无评论

Ada 95 bindings for the NCSA hierarchical data format 01

Ada 95 bindings for the NCSA hierarchical data format

引用

Proceedings of the 2001 annual ACM SIGAda international conference on Ada

作者： Bruce R. Barkstrom Atmospheric Sciences NASA Langley Research Center Hampton VA

ISBN: (纸本)9781581133929

This paper describes Ada95 bindings for HDF4 and HDF5, the current versions of the NCSA hierarchical data format (HDF). These self-describing file formats are intended for storage of large, diverse collections of scientific data and for retrieving subsets of these data. The libraries also support data compression, chunking of large arrays, and automatic conversion of vendor-specific binary formats for a variety of data types.

关键词： File formats Self-documenting files HDF hierarchical data format HDF4 HDF5

来源：评论

学校读者我要写书评

暂无评论

Optimalizace distribuovaného I/O subsystému projektu k-Wave

Optimalizace distribuovaného I/O subsystému projektu k-Wav...

引用

作者： Vysocký, Ondřej Brno University of Technology

Práce se zabývá řešením efektivního paralelního zápisu a čtení dat pro nástroj k-Wave, provádějící simulací šíření ultrazvuku. Tento nás... 详细信息

Práce se zabývá řešením efektivního paralelního zápisu a čtení dat pro nástroj k-Wave, provádějící simulací šíření ultrazvuku. Tento nástroj je superpočítačovou aplikací, proto je spouštěn na souborovém systému Lustre a vyžaduje paralelní zpracování pomocí MPI a zápis ve formátu vhodném pro velké množství dat (HDF5). V rámci této práce byly navrženy metody efektivního způsobu zápisu dat dle potřeb k-Wave, pomocí kumulace dat a přerozdělování. Všechny metody zrychlily nativní zápis a vedly až k rychlosti zápisu 13,6GB/s. Popsané metody jsou použitelné pro všechny aplikace s distribuovanými daty a častým zápisem.

关键词： k-Wave paralelní programování MPI HDF5 Lustre superpočítač I/O zápis optimalizace k-Wave Parallel Programming MPI HDF5 hierarchical data format Lustre Supercomputer I/O Write Optimization Text

来源：评论

学校读者我要写书评

暂无评论

Optimalizace distribuovaného I/O subsystému projektu k-Wave

Optimalizace distribuovaného I/O subsystému projektu k-Wav...

引用

作者： Vysocký, Ondřej Brno University of Technology

Práce se zabývá řešením efektivního paralelního zápisu velkých objemů dat na souborovém systému Lustre. Cílový program je navržen pro projekt k-Wave simuluj&... 详细信息

Práce se zabývá řešením efektivního paralelního zápisu velkých objemů dat na souborovém systému Lustre. Cílový program je navržen pro projekt k-Wave simulující šíření akustických a ultrazvukových vln. Tato simulace pro svou výpočetní a datovou náročnost vyžaduje spouštění na superpočítači a implementaci pomocí knihoven pro paralelní zpracování (Open MPI) a pro uložení velkých objemů dat (HDF5). Výsledný program je implementován v jazyce C s využitím zmíněných knihoven. Správným nastavením souborového systému Lustre bylo dosaženo rychlosti 2,5 GB/s, jež odpovídá 5-ti násobnému zrychlení nativního zápisu, který byl následně pomocí techniky agregace dat zrychlen až na 3 GB/s, což naráží na teoretické limity diskového pole superpočítače Anselm.

关键词： k-Wave paralelní programování Open MPI HDF5 Lustre superpočítač I/O zápis velké množství dat optimalizace k-Wave Parallel Programming Open MPI Message Passing Interface HDF5 hierarchical data format Lustre Supercomputer I/O Write Big D

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：