检索结果-内蒙古大学图书馆

IEEE International Conference on Electrical, Computer, and Energy Technologies (ICECET)

作者： Akinyelu, Andronicus A. Univ Free State Dept Comp Sci & Informat Bloemfontein Free State South Africa

ISBN: (纸本)9781665442312

Today, digital data is exploding at a breakneck speed. Traditional data analytics techniques, unfortunately, are rapidly losing their capabilities and efficiencies when dealing with large datasets. This problem has prompted several researchers to develop more effective, efficient, and fast big data analytics tools. Machine Learning (ML)-based approaches are among the most dependable techniques used to extract usable insights from large datasets. However, some of them cannot efficiently handle large datasets, and their training time grows with dataset size. This paper presents two Nature-Inspired techniques for improving the training time of ML algorithms and the processing time of big datasets. The techniques are evaluated on four ML algorithms and large or medium-scale datasets. Results show that the training time of the four ML algorithms was reduced without a significant drop in classification accuracy. Moreover, the proposed methods are significantly faster than two well-known instance selection methods. Furthermore, statistical analysis reveals that the techniques reduced data size significantly, making them suitable for processing large datasets.

关键词： machine learning bio-inspired algorithm data reduction big data processing

来源：评论

学校读者我要写书评

暂无评论

Fast SQL/Row Pattern Recognition Query processing Using Parallel Primitives on GPUs 32nd

Fast SQL/Row Pattern Recognition Query Processing Using Para...

引用

32nd International Conference on database and Expert Systems Applications (DEXA)

作者： Ohara, Tsubasa Chang, Qiong Miyazaki, Jun Tokyo Inst Technol Sch Comp Tokyo Japan

ISBN: (纸本)9783030864729;9783030864712

SQL/Row Pattern Recognition (SQL/RPR), a row matching query processing for sequence data stored in a database, has been standardized in SQL:2016. So far, many studies have focused on developing technology to perform SQL/RPR for large-scale sequence data, such as stock chart records and system logs. However, due to the large amount of data and complex calculation problems, the processing speeds of current methods are not sufficient. In this paper, we propose a fast SQL/RPR pattern-recognition method that uses parallel primitives on GPUs. This method improves on the processing speed of PostgreSQL, a commonly used RPR method, by 7.1 to 22.6 times in different use cases.

关键词： GPGPU Parallel primitives big data processing SQL/RPR

来源：评论

学校读者我要写书评

暂无评论

Spark-based parallel processing whale optimization algorithm

引用

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2022年第4期34卷

作者： Alshayeji, Mohammad Behbehani, Bader Ahmad, Imtiaz Kuwait Univ Dept Comp Engn Safat Kuwait

Swarm intelligence meta-heuristic optimization algorithms for optimizing engineering applications have become increasingly popular. The whale optimization algorithm (WOA) is a recent and effective swarm intelligence optimization algorithm that mimics humpback whales' behaviors when optimizing a problem. Applying the algorithm to achieve optimal solutions has shown good results compared to most meta-heuristic optimization algorithms. However, complex applications might require the processing of large-scale computations, which results in down-scaling computational throughput of WOA. Apache Spark, a well-known parallel data processing framework, is the most recent distributed computing framework and has been proven to be the most efficient. In this article, we propose a WOA implementation on top of Apache Spark, represented as SBWOA, to enhance its computational performance while providing higher scalability of the algorithm for handling more complex problems. Compared with the recently reported MapReduce WOA (MR-WOA), and serial implementation of WOA, our approach achieves significant enhancements with respect to computational performance for the highest population size with the maximum number of iterations. SBWOA successfully handles higher-complexity problems which require complex computations.

关键词： Apache Spark big data processing meta-heuristic optimization algorithms swarm intelligence whale optimization algorithm

来源：评论

学校读者我要写书评

暂无评论

Smart Management for Smart Cities - Synchronized Solutions 20th

Smart Management for Smart Cities - Synchronized Solutions

引用

20th IFAC Conference on Technology, Culture, and International Stability (TECIS)

作者： Pashchenko, Alexander F. Russian Acad Sci Inst Control Sci Moscow Russia

The paper observes the most updated concepts of Smart Cities. Problem of sustainable development of the cities is presented. An overview of transformation from Smart Cities to Smart Countries is given. Intelligent management system is proposed in the paper enhancing efficiency in the application of intelligent e-Government and Smart City projects in all areas. Integrated Operation Center is introduced - a new concept of operational intelligence, being able to digitize, monitor, predict, act, make better and faster decisions and engage with their citizens. Proposed concept of smart city (smart region, smart country) management system would allow if implemented to make a breakthrough change, improve efficiency in the management and operation of a city or a region. Copyright (C) 2021 The Authors.

关键词： Smart City artificial intelligence internet of things big data processing smart management system sustainable development

来源：评论

学校读者我要写书评

暂无评论

|PB&J| - Easy Automation of data Science/Machine Learning Workflows 8

|PB&J| - Easy Automation of Data Science/Machine Learning Wo...

引用

8th IEEE International Conference on big data (big data)

作者： Ramanan, Buvaneswari Drabeck, Lawrence Woo, Thomas Cauble, Troy Rana, Anil Nokia Bell Labs Murray Hill NJ 07974 USA

ISBN: (纸本)9781728162515

The ability to process large amounts of data efficiently is a must in modern big data, data Science and Machine Learning (ML). Easy and succinct workflow expression with highly efficient scaling is key. Existing systems such as Apache Spark and Beam were created for this type of work but have steep learning curve and tremendous operating complexity. Having a simpler tool with less overhead that is well-matched to the typical workflow definition and execution requirements of data science/ML projects will be beneficial. We propose a new framework PB&J that enables succinct definition and data-parallel scale-out execution of workflows with ease. PB&J is easy to learn as it builds on the native Unix shell, with the addition of just a few operators. PB&J supports Maximal Parallelism with its ability to do true pipelining, and fault recovery with Minimal Redo. It is well-matched to the typical data science/ML processing requirements by leveraging the existing command line executables and Python modules. We illustrate the features and strengths of PB&J with real-life Deep Learning data processing workflows. We compare it to existing frameworks such as Apache Spark, Apache Beam, Swift/T and Apache Airflow in terms of ease of authoring, efficiency, scalability and fault recovery.

关键词： big data processing Distributed computing data Pipelines Pipeline Specification Workflow Management Spark Beam Swift/T Airflow Parallelization Workflow Specification data Parallel Execution

来源：评论

学校读者我要写书评

暂无评论

Study on mismatch losses in large PV plants: data analysis of a case study and modeling approach

Study on mismatch losses in large PV plants: data analysis o...

引用

International Symposium on Power Electronics, Electrical Drives, Automation and Motion (SPEEDAM)

作者： Nobile, Giovanni Vasta, Ester Cacciato, Mario Scarcella, Giuseppe Scelba, Giacomo Di Stefano, Agnese Giuseppa Federica Leotta, Giuseppe Pugliatti, Paola Maria Bizzarri, Fabrizio Univ Catania Dept Elect Elect Engn & Comp Sci Catania Italy Enel Green Power SpA Catania Italy

ISBN: (纸本)9781728170190

The mismatch effect is of fundamental importance in the operation of photovoltaic (PV) power plants because it causes significant losses in energy production. Mismatch originates from several factors such as non-uniform modules aging, shading, dust accumulation, faults in tracker systems and so on. Evaluation of mismatch level is a difficult task because it is necessary to take into account many factors: power configuration, geographical position, weather conditions, quality of measurements provided by dataloggers, etc. Moreover, since different criteria can be applied to calculate the percentage of mismatch, different results might be obtained for the same PV system leading in some cases to wrong conclusions. With the purpose of introducing some approaches for the assessment of mismatch losses in large PV fields, this paper presents a study carried out from data acquired in an existing 300 MW PV plant in Brazil in which an experimental 2.5 MW subfield was realized using 60 kW string inverters whose input stage has more DC/DC input converters. This particular power configuration allows the calculation of mismatch at string level as well as at array level at the DC side input of each string inverter. A multi-criteria approach has been implemented with the purpose to evaluate the coherency of results. A modelling strategy, based on an integrated state-space average model, has been exploited in order to get all the electrical quantities in case of faults, missing or wrong measurements.

关键词： Photovoltaic plants Mismatch Distributed converters State-space modeling methods big data processing

来源：评论

学校读者我要写书评

暂无评论

Using Multi-Dimensional Dynamic Time Warping to Identify Time-Varying Lead-Lag Relationships

引用

SENSORS 2022年第18期22卷 6884-6884页

作者： Stuebinger, Johannes Walter, Dominik Univ Erlangen Nurnberg Dept Stat & Econometr Lange Gasse 20 D-90403 Nurnberg Germany

This paper develops a multi-dimensional Dynamic Time Warping (DTW) algorithm to identify varying lead-lag relationships between two different time series. Specifically, this manuscript contributes to the literature by improving upon the use towards lead-lag estimation. Our two-step procedure computes the multi-dimensional DTW alignment with the aid of shapeDTW and then utilises the output to extract the estimated time-varying lead-lag relationship between the original time series. Next, our extensive simulation study analyses the performance of the algorithm compared to the state-of-the-art methods Thermal Optimal Path (TOP), Symmetric Thermal Optimal Path (TOPS), Rolling Cross-Correlation (RCC), Dynamic Time Warping (DTW), and Derivative Dynamic Time Warping (DDTW). We observe a strong outperformance of the algorithm regarding efficiency, robustness, and feasibility.

关键词： time-varying lead-lag effect dynamic time warping data science big data processing multi-dimensional thermal optimal path simulation study econometric modeling

来源：评论

学校读者我要写书评

暂无评论

Associative Rules-Driven Intelligent Production Schedule Control System for Digital Manufacturing Ecosystem

引用

IFAC-PapersOnLine 2022年第10期55卷 2526-2532页

作者： Alexander Suleykin Natalya Bakhtadze Denis Elpashev Valery Pyatetsky V.A. Trapeznikov Institute of Control Sciences of the Russian Academy of Science 65 Profsoyuznaya Street Moscow 117997 Russia

The paper presents the architecture of a subsystem for processing, storing, visualizing and analyzing of production schedules. It is an important part of an integrated digital manufacturing process control ecosystem. The proposed system allows to process and analyze the production schedule in real time with reference to possible changes in the production situation. The system uses open source big data solutions. The results of the system operation tests with the data from production schedules of a large automotive company are presented. A methodology for updating production schedules based on the realtime predictive models for making control decisions is proposed; the overall system performance is investigated. Identification models for digital twins are offered.

关键词： digital ecosystems big data processing production schedule analysis visualization realtime data streaming

来源：评论

学校读者我要写书评

暂无评论

DRIIS: MapReduce Parameter Optimization of Hadoop Using Genetic Algorithm

引用

INTERNATIONAL JOURNAL OF COOPERATIVE INFORMATION SYSTEMS 2022年第1-2期31卷 2150002-2150002页

作者： Anusha, R. J. RamaParvathy, L. Saveetha Inst Tech & Med Sci Chennai 600077 Tamil Nadu India

In this era of massive knowledge, Hadoop, jointly of the foremost unremarkably used massive processing platforms, features a variety of parameters that are closely associated with resource utilization, particularly mainframe and memory. Calibration of these parameters through improvement may increase Hadoop's resource utilization. Manual calibration of those parameters is just. about not possible, thanks to the time price. Within the massive knowledge business, there's a requirement to mechanically set up parameters and thereby maximize resource usage. The previous automatic calibration strategies take an extended time to realize the optimum configuration, reducing the cluster's overall performance. With the help of a novel perceptive procedure, each genetic programming and a genetic rule are supported with a view to enhancing the performance of a Hadoop MapReduce work, we propose an assistant in nursing the best configuration finder. We have a tendency to use the algorithms on top to search out the simplest values for parameter settings. Experiments were performed on four common applications, WordCount, TeraSort, Index and Grep, and eight virtual machines (VMs) in a very typical Hadoop cluster. Our projected methodology will increase MapReduce job potency by 53.63% for a one GB dataset and by 67.4% for a five GB dataset, in keeping with the findings, and by 73.68% on a 10 GB dataset;additionally, for a TeraSort programmed application, MapReduce job potency will increase by 52.62% for an one GB dataset, 61.2% for a five GB dataset, and 55.17% for a 10GB dataset. MapReduce jobs boost the performance of Grep applications by 44.4% for a one GB dataset, 56.25% for a five GB dataset, and 49.44% for a 10 GB dataset.

关键词： big data processing resource utilization optimization parameter tuning online configuration optimization

来源：评论

学校读者我要写书评

暂无评论

A Parallel Computing Approach to Spatial Neighboring Analysis of Large Amounts of Terrain data Using Spark

引用

SENSORS 2021年第2期21卷 365-365页

作者： Zhang, Jianbo Ye, Zhuangzhuang Zheng, Kai China Univ Geosci Sch Geog Informat Engn Wuhan 430074 Peoples R China

Spatial neighboring analysis is an indispensable part of geo-raster spatial analysis. In the big data era, high-resolution raster data offer us abundant and valuable information, and also bring enormous computational challenges to the existing focal statistics algorithms. Simply employing the in-memory computing framework Spark to serve such applications might incur performance issues due to its lack of native support for spatial data. In this article, we present a Spark-based parallel computing approach for the focal algorithms of neighboring analysis. This approach implements efficient manipulation of large amounts of terrain data through three steps: (1) partitioning a raster digital elevation model (DEM) file into multiple square tile files by adopting a tile-based multifile storing strategy suitable for the Hadoop Distributed File System (HDFS), (2) performing the quintessential slope algorithm on these tile files using a dynamic calculation window (DCW) computing strategy, and (3) writing back and merging the calculation results into a whole raster file. Experiments with the digital elevation data of Australia show that the proposed computing approach can effectively improve the parallel performance of focal statistics algorithms. The results also show that the approach has almost the same calculation accuracy as that of ArcGIS. The proposed approach also exhibits good scalability when the number of Spark executors in clusters is increased.

关键词： spatial neighboring analysis Spark parallel computing big data processing

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：