检索结果-内蒙古大学图书馆

Advancing distributed data management for the HydroShare hydrologic information system

ENVIRONMENTAL MODELLING & SOFTWARE 2018年 102卷 233-240页

作者： Yi, Hong Idaszak, Ray Stealey, Michael Calloway, Chris Couch, Alva L. Tarboton, David G. Univ North Carolina Chapel Hill Renaissance Comp Inst RENCI Chapel Hill NC 27599 USA Tufts Univ Medford MA 02155 USA Utah State Univ Logan UT 84322 USA

HydroShare (https://***) is an online collaborative system to support the open sharing of hydrologic data, analytical tools, and computer models. Hydrologic data and models are often large, extending to multi-gigabyte or terabyte scale, and as a result, the scalability of centralized data management poses challenges for a system such as HydroShare. A distributed data management framework that enables distributed physical data storage and management in multiple locations thus becomes a necessity. We use the iRODS (Integrated Rule-Oriented data System) data grid middleware as the distributed data storage and management back end in HydroShare. iRODS provides a unified virtual file system for distributed physical storages in multiple locations and enables data federation across geographically dispersed institutions around the world. In this paper, we describe the iRODS-based distributed data management approaches implemented in HydroShare to provide a practical demonstration of a production system for supporting big data in the environmental sciences. (c) 2017 Elsevier Ltd. All rights reserved.

关键词： distributed data management Big data data sharing Hydrologic information systems Collaborative environment iRODS

来源：评论

学校读者我要写书评

暂无评论

TOWARDS EXASCALE distributed data management

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS 2009年第4期23卷 398-400页

作者： Aloisio, Giovanni Fiore, Sandro Univ Salento Lecce Italy

"Exascale eScience infrastructures" will face important and critical challenges, both from computational and data perspectives. Increasingly complex and parallel scientific codes will lead to the production of a huge amount of data. The large volume of data and the time needed to locate, access, analyze and visualize data will greatly impact on the scientific productivity of scientists and researchers in several domains. Significant improvements in the data management field will increase research productivity in solving complex scientific problems. Next-generation eScience infrastructures will start from the assumption that exascale high-performance computing (HPC) applications (running on million of cores) will generate data at a very high rate (terabytes/s). Hundreds of exabytes of data (distributed across several centers) are expected, by 2020, to be available through heterogeneous storage resources for access, analysis, post-processing and other scientific activities.

关键词： distributed data management data replication metadata management data analysis parallel I/O

来源：评论

学校读者我要写书评

暂无评论

SERVICE-BASED distributed data management AND APPLICATION IN CHINA DIGITAL OCEAN 30

SERVICE-BASED DISTRIBUTED DATA MANAGEMENT AND APPLICATION IN...

引用

30th IEEE International Geoscience and Remote Sensing Symposium (IGARSS) on Remote Sensing - Global Vision for Local Action

作者： Dong, Wen Zhang, Xin Jiang, Bing Chinese Acad Sci Inst Remote Sensing Applicat Beijing 100101 Peoples R China China Natl Marine Data & Informat Serv Tianjin 300171 Peoples R China

ISBN: (纸本)9781424495665

The rapid development of ocean observing instruments makes the rapid increase in the amount and types of marine data, but with the deepening of the marine development, comprehensive utilization and research, traditional data management and use methods have been unable to meet all kinds of users. This paper proposed to implement distributed data management and application through service technology based on the analysis of the situation and lack of marine data management, and designed the system framework and application modes. Finally, this design has been realized in China Digital Ocean information construction.

关键词： distributed data management web service China Digital Ocean

来源：评论

学校读者我要写书评

暂无评论

Analyzing WLCG File Transfer Errors Through Machine Learning: An Automatic Pipeline to Support Post-mortem distributed data management

引用

Computing and Software for Big Science 2022年第1期6卷 1-15页

作者： Clissa, Luca Lassnig, Mario Rinaldi, Lorenzo National Institute for Nuclear Physics Bologna Italy Department of Physics and Astronomy University of Bologna Bologna Italy CERN Geneva Switzerland

The increasingly growing scale of modern computing infrastructures solicits more ingenious and automatic solutions to their management. Our work focuses on file transfer failures within the Worldwide Large Hadron Collider Computing Grid and proposes a pipeline to support distributed data management operations by suggesting potential issues to investigate. Specifically, we adopt an unsupervised learning approach leveraging Natural Language Processing and Machine Learning tools to automatically parse error messages and group similar failures. The results are presented in the form of a summary table containing the most common textual patterns and time evolution charts. This approach has two main advantages. First, the joint elaboration of the error string and the transfer’s source/destination enables more informative and compact troubleshooting, as opposed to inspecting each site and checking unique messages separately. As a by-product, this also reduces the number of errors to check by some orders of magnitude (from unique error strings to unique categories or patterns). Second, the time evolution plots allow operators to immediately filter out secondary issues (e.g. transient or in resolution) and focus on the most serious problems first (e.g. escalating failures). As a preliminary assessment, we compare our results with the Global Grid User Support ticketing system, showing that most of our suggestions are indeed real issues (direct association), while being able to cover 89% of reported incidents (inverse relationship). © 2022, The Author(s).

关键词： Clustering distributed data management Machine learning Text processing WLCG

来源：评论

学校读者我要写书评

暂无评论

SODA-A distributed data management Framework for the Internet of Services

SODA-A Distributed Data Management Framework for the Interne...

引用

第七届网格与协同计算国际会议(Seventh International Conference on Grid and Cooperative Computing GCC 2008)

作者： Peter Paul Beran Gernot Habel Erich Schikuta University of Vienna Faculty of Computer Science Department of Knowledge and Business Engineering Rathausstr.19/9 A-1010 Vienna Austria

ISBN: (纸本)9781424432806;9780769534497

Soda (Service Oriented database Architecture) is a proof-of-concept prototype middleware implementation for data intensive applications, where data of relevance is stored at multiple locations over the Internet. Because of its modularity the used Web Services can be plugged together very easily, almost in a LEGOTMalike manner. In fact these services provide the business logic for distributed query execution, specifically supporting autonomous databases in heterogeneous environments. Operations used in this framework – like selection, projection, join, product or sort – can be orchestrated (used, added and removed) dynamically. The architecture is based on the WSRF standard and thus supports stateful Web Services. For reasons of portability it is implemented in Java and uses the WSRFcompatible Apache Muse library.

关键词： Internet computing Web services SOA/SOI Parallel and distributed database operations Query processing Web Service query analysis Internets Legal Services distributed data management Internet services Portability data management Frameworks

来源：评论

学校读者我要写书评

暂无评论

D2C-DM: distributed-to-Centralized data management for Smart Cities Based on Two Ongoing Case Studies

D2C-DM: Distributed-to-Centralized Data Management for Smart...

引用

Intelligent Systems Conference (IntelliSys)

作者： Sinaeepourfard, Amir Krogstie, John Petersen, Sobah Abbas Norwegian Univ Sci & Technol NTNU Dept Comp Sci Trondheim Norway

ISBN: (纸本)9783030295134;9783030295127

Smart city environments follow different technological management strategies (such as resource management, data management, and so on) between end-users to city planners and technological devices (e.g., sensor, camera surveillance, etc.) to enhance citizens' quality of life through the variety of the smart services. data management is one of the most critical issues in smart cities because data is a core resource in the smart city. Without proper data, no smart services in the smart cities exist to make a connection between end-users and technological devices. A few numbers of distributed-to-centralize data management architectures have been proposed. In addition, there are several different distributed schema by several technological options exist (e.g., cloudlet, fog, etc.) but almost all of the studies used a distributed-to-centralized data management architecture based on fog to cloud technologies. Therefore, the fogto-cloud data management architecture can use both potentials of fog and cloud technologies, including the decrease in communication latencies, organizing distinct policies (e.g., data filtering, data compression, etc.) and so on. In this paper, first, previous studies of distributed-to-centralized data management architectures through two different smart city scenarios have been revisited. Afterward, the easy use and adaptation of the distributed-to-centralized data management architecture to any smart city scenario has been shown. In addition, the advantages of this data management architecture have been highlighted including efficiency rates for the data collection and data storage, and reducing data and network traffic. Finally, a number of the lesson learned from previous case studies has been addressed.

关键词： IoT Smart city data management distributed data management Centralized data management Fog-to-cloud data management Sensor data management

来源：评论

学校读者我要写书评

暂无评论

IDIOMS: Index-powered distributed Object-centric Metadata Search for Scientific data management 24

IDIOMS: Index-powered Distributed Object-centric Metadata Se...

引用

24th IEEE/ACM International Symposium on Cluster, Cloud, and Internet Computing (CCGrid)

作者： Zhang, Wei Tang, Houjun Byna, Suren Lawrence Berkeley Natl Lab Berkeley CA 94720 USA Ohio State Univ Columbus OH USA

ISBN: (纸本)9798350395679;9798350395662

Affix-oriented metadata search is one of the essential fuzzy search capabilities that allow users to find data of interest in their voluminous data set with incomplete query conditions. With the recent transition towards object-centric data management systems in the science community, there is a paramount need for the support of such features in distributed settings. However, existing metadata search solutions either do not support efficient affix-oriented metadata search or do not suit well in a distributed setting of object-centric data management systems. To bridge this gap, we introduce IDIOMS, a metadata search solution underpinned by a distributed metadata index, meticulously designed to enable high-performance affix-oriented metadata search for parallel object-centric storage. One of the standout features of IDIOMS is its efficiency in supporting four distinct types of highly demanded metadata queries. Furthermore, IDIOMS is flexibly catering to both independent and collective metadata search operations. Our experimental comparisons with SoMeta, a state-of-the-art metadata query method, demonstrate more than 400x performance boost for independent queries and up to 300x performance improvements for collective queries, while keeping a small index management overhead.

关键词： Metadata Search Metadata management Parallel data Processing distributed data management Scientific data management

来源：评论

学校读者我要写书评

暂无评论

The network is the database: data management for highly distributed systems 01

The network is the database: Data management for highly dist...

引用

ACM SIGMOD International Conference on management of data

作者： Navas, JC Wynblatt, M Siemens Technol Business Ctr Berkeley CA 94704 USA

ISBN: (纸本)9781581133325

This paper describes the methodology and implementation of a data management system for highly distributed systems, which was built to solve the scalability and reliability problems faced in a wide area postal logistics application developed at Siemens. The core of the approach is to borrow from Internet routing protocols, and their proven scalability and robustness, to build a network-embedded dynamic database index, and to augment schema definition to optimize the use of this index. The system was developed with an eye toward future applications in the area of sensor networks.

关键词： wide-area data management sensor networks logistics distributed data management

来源：评论

学校读者我要写书评

暂无评论

A management Strategy for distributed data in Collaborative Design

A Management Strategy for Distributed Data in Collaborative ...

引用

13th International Conference on Computer Supported Cooperative Work in Design

作者： Zhu, Bin Wang, Yantao Yantai Univ Sch Electromech Automobile Engn Yantai Peoples R China

ISBN: (纸本)9781424435340

This paper presents a distributed data management strategy for collaborative design, which supports the data exchange pattern based on any fine-grained data as well as that based on traditional file service. data are represented using XML and their exchange is achieved through the publishing and,subscription mechanism among individual participants. It is also shown in this paper that how the data replication mechanism, which supports both synchronous and asynchronous mechanism, propagates updates among data copies to provide consistent views of data among many computers, in which the rule-based approach is used to resolve conflict. Based on this strategy, a research prototype system of collaborative design has been implemented and tested in the design of auto-body welding lines.

关键词： Collaborative Design distributed data management data Consistency data Replication

来源：评论

学校读者我要写书评

暂无评论

Coupling Secret Sharing with Decentralized Server-Aided Encryption in Encrypted Deduplication

引用

APPLIED SCIENCES-BASEL 2025年第3期15卷 1245-1245页

作者： Gan, Chuang Wang, Weichun Hu, Yuchong Zhao, Xin Dun, Shi Xiao, Qixiang Wang, Wei Huang, Huadong Huazhong Univ Sci & Technol Sch Comp Sci & Technol Wuhan 430074 Peoples R China Hikvis Digital Technol Co Ltd Hangzhou 310052 Peoples R China

Outsourcing storage to the cloud can save storage costs and is commonly used in businesses. It should fulfill two major goals: storage efficiency and data confidentiality. Encrypted deduplication can achieve both goals via performing deduplication to eliminate the duplicate data within encrypted data. Traditional encrypted deduplication generates the encryption key on the client side, which poses a risk of offline brute-force cracking of the outsourced data. Server-aided encryption schemes have been proposed to strengthen the confidentiality of encrypted deduplication by distributing the encryption process to dedicated servers. Existing schemes rely on expensive cryptographic primitives to provide a decentralized setting on the dedicated servers for scalability. However, this incurs substantial performance slowdown and can not be applied in practical encrypted deduplication storage systems. In this paper, we propose a new decentralized server-aided encrypted deduplication approach for outsourced storage, called ECDedup, which leverages secret sharing to achieve secure and efficient key management. We are the first to use the coding matrix as the encryption key to couple the encryption and encoding processes in encrypted deduplication. We also propose a acceleration scheme to speed up the encryption process of our ECDedup. We prototype ECDedup in cloud environments, and our experimental results based on the real-world backup datasets show that ECDedup can improve the client throughput by up to 51.9% compared to the state-of-the-art encrypted deduplication schemes.

关键词： encrypted deduplication secret sharing server-aided encryption message-locked encryption distributed data management

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：