检索结果-内蒙古大学图书馆

3rd International Conference on Intelligent Technologies and Engineering Systems (ICITES)

作者： Chen, Wenbo Xu, Shungou Jiang, Hai Weng, Tien-Hsiung Marino, Mario Donato Chen, Yi-Siang Li, Kuan-Ching Lanzhou Univ Sch Informat & Technol Lanzhou 730000 Peoples R China Arkansas State Univ Dept Comp Sci Jonesboro AR USA Providence Univ Dept Comp Sci & Informat Engn Taichung Taiwan

ISBN: (纸本)9783319173146;9783319173139

Hadoop is a well-designed approach for handling massive amount of data. Comprised at the core of the Hadoop File System and MapReduce, it schedules the processing by orchestrating the distributed servers, providing redundancy and fault tolerance. In terms of performance, Hadoop is still behind high performance capacity due to CPUs' limited parallelism, though. GPU accelerated computing involves the use of a GPU together with a CPU to accelerate applications to data processing on GPU cluster toward higher efficiency. However, GPU cluster has low level data storage capacity. In this chapter, we exploit the hybrid model of GPU and Hadoop to make best use of both capabilities, and the design and implementation of application using Hadoop and CUDA is presented through two interfaces: Hadoop Streaming and Hadoop Pipes. Experimental results on K-means algorithm are presented as well as their performance results are discussed.

关键词： Hadoop GPU HPC massive data processing

来源：评论

学校读者我要写书评

暂无评论

Implementation of massive data processing Architecture for Electric Enterprise Groups 15

Implementation of Massive Data Processing Architecture for E...

引用

15th IEEE/ACIS International Conference on Computer and Information Science (ICIS)

作者： Wu, Linli Shen, Zhangyi Feng, Xiang Luoyang Normal Univ Informat Technol Coll Luoyang Henan Peoples R China East China Normal Univ Shanghai Engn Res Ctr Shanghai Peoples R China

ISBN: (纸本)9781509008056

In the past three decades, with the fast development of China's economy, the business data of electric power industry also showed rapid growth and the data size is continual expanding. Thus, the management systems of electric enterprise groups encountered the performance bottlenecks of massive data processing. Hadoop is an open-source software framework developed for reliable, scalable, and efficient distributed computing and storage, and is widely used in massive data processing. Based on the study of the proposed massive business data processing of electric industry, this paper introduces a novel architecture, which is based on Hadoop ecosystem, for management systems of electric enterprise groups to crack the present performance bottlenecks. First, the business logic and processing produce of management system is analyzed. Then new architecture with Hadoop based distributed processing model is introduced in detailed. At last, experimental results with actual business scenarios and data prove the architecture can effectively resolve present performance bottlenecks of massive business data processing, and greatly improve the performance and efficiency of the management system of electric enterprise groups.

关键词： massive data processing distributed processing Hadoop Hive

来源：评论

学校读者我要写书评

暂无评论

A massive data processing approach for effective trustworthiness in online learning groups

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2015年第8期27卷 1988-2003页

作者： Miguel, Jorge Caballe, Santi Xhafa, Fatos Prieto, Josep Open Univ Catalonia Dept Comp Sci Multimedia & Telecommun Barcelona Spain Tech Univ Catalonia Dept Languages & Informat Syst Barcelona Spain

This paper proposes a trustworthiness-based approach for the design of secure learning activities in online learning groups. Although computer-supported collaborative learning has been widely adopted in many educational institutions over the last decade, there exist still drawbacks that limit its potential. Among these limitations, we investigate on information security vulnerabilities in learning activities, which may be developed in online collaborative learning contexts. Although security advanced methodologies and technologies are deployed in learning management systems, many security vulnerabilities are still not satisfactorily solved. To overcome these deficiencies, we first propose the guidelines of a holistic security model in online collaborative learning through an effective trustworthiness approach. However, as learners' trustworthiness analysis involves large amount of data generated along learning activities, processing this information is computationally costly, especially if required in real time. As the main contribution of this paper, we eventually propose a parallel processing approach, which can considerably decrease the time of data processing, thus allowing for building relevant trustworthiness models to support learning activities even in real time. Copyright (c) 2014 John Wiley & Sons, Ltd.

关键词： trustworthiness e-Learning activities computer-supported collaborative learning information security parallel processing log files massive data processing Hadoop MapReduce

来源：评论

学校读者我要写书评

暂无评论

processing of massive Rutherford Back-scattering Spectrometry data by artificial neural networks

引用

NUCLEAR INSTRUMENTS & METHODS IN PHYSICS RESEARCH SECTION B-BEAM INTERACTIONS WITH MATERIALS AND ATOMS 2021年 493卷 28-34页

作者： Guimaraes, Renato da S. Silva, Tiago F. Rodrigues, Cleber L. Tabacniks, Manfredo H. Bach, Simon Burwitz, Vassily V. Hiret, Paul Mayer, Matej Univ Sao Paulo Phys Inst Sao Paulo Brazil Max Planck Inst Plasma Phys Garching Germany

Rutherford Backscattering Spectrometry (RBS) is an important technique providing elemental information of the near surface region of samples with high accuracy and robustness. However, this technique lacks throughput by the limited rate of data processing and is hardly routinely applied in research with a massive number of samples (i.e. hundreds or even thousands of samples). The situation is even worse for complex samples. If roughness or porosity is present in those samples the simulation of such structures is computationally demanding. Fortunately, Artificial Neural Networks (ANN) show to be a great ally for massive data processing of ion beam data. In this paper, we report the performance comparison of ANN against human evaluation and an automatic fit routine running on batch mode. 500 spectra of marker layers from the stellarator W7-X were used as study case. The results showed ANN as more accurate than humans and more efficient than automatic fits.

关键词： Rutherford Back-scattering Spectrometry Artificial neural networks massive data processing

来源：评论

学校读者我要写书评

暂无评论

Enhancing data security in massive data sets using blockchain and federated learning: a loosely coupled approach

引用

INTERNATIONAL JOURNAL OF INTERNET PROTOCOL TECHNOLOGY 2024年第1期17卷 31-41页

作者： Kang, Haiyan Wu, Bing Beijing Informat Sci & Technol Univ Dept Informat Secur Beijing Peoples R China

The properties of the blockchain are not suitable for the storage of sensitive privacy data, and the excellent characteristics of the blockchain will be seriously affected by the existence of massive amounts of data on the chain. To address the above issues, propose a Loosely Coupled Local Differential Privacy Blockchain Federated Learning method (LL-BCFL). First of all, a client selection mechanism is put forward to ensure honesty and positivity in joining the training client and the correct effectiveness of the final global model aggregation. Secondly, use federated learning to alleviate the 'data silos' phenomenon and achieve joint training of big data stored in distributed multi-parties. Additionally, a differential privacy method is introduced to act on federated learning networks to avoid inference attacks. Finally, the MNIST dataset was used to confirm the availability of the LL-BCFL method on balanced and unbalanced datasets.

关键词： federated learning blockchain differential privacy massive data processing

来源：评论

学校读者我要写书评

暂无评论

Design and development of a Python-based interface for processing massive data with the LOAD ESTimator (LOADEST)

引用

ENVIRONMENTAL MODELLING & SOFTWARE 2021年 135卷 104897-104897页

作者： Gao, Jungang White, Michael J. Bieger, Katrin Arnold, Jeffrey G. Blackland Res & Extens Ctr Texas A&M AgriLife 702 E Blackland Rd Temple TX 76502 USA ARS USDA 808 E Blackland Rd Temple TX 76502 USA

LOADEST is a program for estimating constituent loads in rivers and streams developed by the U.S. Geological Survey (USGS), but it does not have a Graphical User Interface (GUI) that facilitates processing of large amounts of data. Therefore, we present the LOAD ESTimation (LOADEST) Parallel data processing Interface (LPDPI). LPDPI is unique as it features an easy-to-use workflow for data download and water quality estimations for numerous stations and multiple constituents and is readily applicable to any station with both flow and water quality data available. LPDPI incorporates a parallel module for faster load estimation and can identify and fix errors that occur while running LOADEST by adjusting calibration and estimation data inputs. LPDPI also includes an extension to extract and filter LOADEST output to facilitate further data analysis and use of the data to calibrate hydrologic models. The tool is a standalone executable for Windows and can be readily used without any additional packages or software installation.

关键词： Water quality PyQt5 Graphic user interface massive data processing LOADEST

来源：评论

学校读者我要写书评

暂无评论

Efficient Radio Interferometric Imaging on the GPU 18

Efficient Radio Interferometric Imaging on the GPU

引用

IEEE 18th International Conference on E-Science (E-Science)

作者： Liu, Honghao Luo, Qiong Wang, Feng Hong Kong Univ Sci & Technol Hong Kong Peoples R China Guangzhou Univ Guangzhou Peoples R China

ISBN: (数字)9781665461245

ISBN: (纸本)9781665461245

Radio interferometric imaging refers to the construction of images on visibility data observed by radio telescope arrays. Due to the data and computation intensity, existing imaging algorithms are slow for wide field measurement. To address this problem, we propose cuGridder, a GPU-based CUDA C program that produces the same result as W-gridder, the latest CPU-based imaging algorithm, but the execution is accelerated effectively by the GPU. Our main idea is to divide the imaging workflow into a sequence of steps, including convolution, 2D FFT, 1D DFT, and correction, and parallelize each step as a GPU kernel program. Furthermore, we design a lock-free algorithm for convolution, the most expensive step in the workflow, adopt a simple yet effective mask function for the convolution, and optimize the memory access pattern. As a result, cuGridder outperforms the state-of-the-art CPU-based and GPU-based libraries in running time and achieves high image quality. Our code package is available at https://***/RapidsAtHKUST/cuGridder

关键词： GPU acceleration radio interferometry massive data processing

来源：评论

学校读者我要写书评

暂无评论

Big data Sampling Algorithm Based on Peak Detection 33

Big Data Sampling Algorithm Based on Peak Detection

引用

33rd Chinese Control and Decision Conference (CCDC)

作者： Liu, Mengyu Wang, Yuhang Lin, Ruishi Wang, Shenhang Zheng, Wei Beijing Aerosp Automat Control Inst Beijing 100854 Peoples R China

ISBN: (纸本)9781665440899

Domestic mass data processing system in aerospace field uses big data simple sampling algorithm for data specification in the data preprocessing stage. This paper analyzes the data curve distortion caused by this algorithm, and proposes an optimization method for that. Finally, a big data sampling algorithm based on peak detection is adopted to achieve the purpose of quickly viewing the fidelity and complete picture of massive historical data, while ensuring the correctness of the data interpretation after data preprocessing at the same time. Through the using of real test data for verification, in the data preprocessing stage of the domestic mass data processing system, the large data sampling algorithm based on peak detection is adopted to achieve the high fidelity of the data curve after sampling.

关键词： massive data processing Peak Detection Big data Sampling data Curve Fidelity

来源：评论

学校读者我要写书评

暂无评论

Big data Sampling Algorithm Based on Peak Detection

Big Data Sampling Algorithm Based on Peak Detection

引用

第33届中国控制与决策会议

作者： Mengyu Liu Yuhang Wang Ruishi Lin Shenhang Wang Wei Zheng Beijing Aerospace Automatic Control Institute

Domestic mass data processing system in aerospace field uses big data simple sampling algorithm for data specification in the data preprocessing *** paper analyzes the data curve distortion caused by this algorithm,and proposes an optimization method for ***,a big data sampling algorithm based on peak detection is adopted to achieve the purpose of quickly viewing the fidelity and complete picture of massive historical data,while ensuring the correctness of the data interpretation after data preprocessing at the same *** the using of real test data for verification,in the data preprocessing stage of the domestic mass data processing system,the large data sampling algorithm based on peak detection is adopted to achieve the high fidelity of the data curve after sampling.

关键词： massive data processing Peak Detection Big data Sampling data Curve Fidelity

来源：评论

学校读者我要写书评

暂无评论

On-Demand Urgent High Performance Computing Utilizing the Google Cloud Platform 1

On-Demand Urgent High Performance Computing Utilizing the Go...

引用

1st IEEE/ACM International Workshop on HPC for Urgent Decision Making (UrgentHPC)

作者： Posey, Brandon Deer, Adam Gorman, Wyatt July, Vanessa Kanhere, Neeraj Speck, Dan Wilson, Boyd Apon, Amy Clemson University United States Google TrafficVision Burwood Group United States

ISBN: (纸本)9781728159959

In this paper we describe how high performance computing in the Google Cloud Platform can be utilized in an urgent and emergency situation to process large amounts of traffic data efficiently and on demand. Our approach provides a solution to an urgent need for disaster management using massive data processing and high performance computing. The traffic data used in this demonstration is collected from the public camera systems on Interstate highways in the Southeast United States. Our solution launches a parallel processing system that is the size of a Top 5 supercomputer using the Google Cloud Platform. Results show that the parallel processing system can be launched in a few hours, that it is effective at fast processing of high volume data, and can be de-provisioned in a few hours. We processed 211TB of video utilizing 6,227,593 core hours over the span of about eight hours with an average cost of around $0.008 per vCPU hour, which is less than the cost of many on-premise HPC systems.

关键词： HPC in the commercial cloud Google Cloud Platform disaster management massive data processing traffic management

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：