检索结果-内蒙古大学图书馆

Enhancing Data Analytics in Environmental Sensing Through Cloud IoT Integration

JOURNAL OF CLIMATE CHANGE 2024年第2期10卷 41-45页

作者： Verma, Rohan Taneja, Harsh Singh, Kiran Deep Singh, Prabh Deep Graph Era Dehra Dun Uttarakhand India Chitkara Univ Chitkara Univ Inst Engn & Technol Rajpura Punjab India

Transformational advances in environmental sensing have been made possible by the convergence of Cloud Computing and the Internet of Things (IoT). The potential for these technologies is to work together and improve environmental monitoring data analytics. Integrating IoT in the cloud creates a robust environment for advanced analytics by overcoming previous hurdles in data granularity, real-time monitoring, and geographical coverage. These developments aren't without their own set of difficulties, however, including data security, interoperability, and ethical concerns. This paper explores integrating Cloud IoT with environmental sensing. The paper addresses these issues and emphasises the need to establish ethical data-gathering methods, implement standardised communication protocols, and address privacy concerns. The study concludes by examining the challenges, concerns, and potential of integrating Cloud IoT. The integration of Cloud Computing and IoT not only transforms environmental sensing but also provides a solid foundation for collaborative research, data-driven decision-making, and environmentally conscious management.

关键词： Big data distributed computing Parallel processing Tuning parameters Java virtual machine hadoop Yarn hadoop distributed file system MapReduce

来源：评论

学校读者我要写书评

暂无评论

Faster As Well As Early Measurements from Big Data Predictive Analytics Model 1

Faster As Well As Early Measurements from Big Data Predictiv...

引用

1st International Conference on Technologies for Smart Green Connected Society 2021, ICTSGS 2021

作者： Gupta, Shashi Kant Tiwari, Sharda Jamil, Azlin Abd Singh, Prabhdeep Department of CS&E Integral University UP Lucknow India Department of CS&E Faculty of Engineering and Technology Dr Shakuntala Misra National Rehabilitation University UP Lucknow India Universiti Teknologi Malaysia School of Computer Applications BBD University UP Lucknow India

ISBN: (纸本)9781607685395

Data processing in large scale system and analyses is the key problems of today's distributed systems in real-time and close to real time. These systems should be able, while meeting these constraints, to process large-scale data inputs, to expand to Terabytes or larger. Volume of processing the data over the systems of large scale becomes too large. These systems use parallelization of the data and similar techniques to increase performance. The standards are increasing, though: data analyses are expected almost in real time. Among all the data treatment models Map Reduce is mostly adopted. Due to the processing of data design in this method, once work is finished, the results are returned. The growth of data is not always a preferred batch operating environment: large numbers of applications can benefit from early results. The online aggregation method used in connection databases was first discussed. In this work analysis we discuss techniques that can allow early results to be estimated. We are proposing some changes to the online system for Map Reduce. We show that the properties required for accurate evaluation of the effects of our proposed device design changes. The reduction of data bias algorithm and sampling at the block levels are provided. Consequently, with a series of particular applications and sets of data the implementation of the proposed design of the system is defined and evaluated. © The Electrochemical Society

关键词： Big Data Data processing Data sampling hadoop distributed file system MapReduce framework

来源：评论

学校读者我要写书评

暂无评论

A Novel Political Optimizer-Based Feature Selection with an Optimal Machine Learning Model for Financial Crisis Prediction

引用

INTERNATIONAL JOURNAL OF COOPERATIVE INFORMATION systemS 2024年第3期33卷

作者： Vodithala, Swathy Bhukya, Raghuram Kakatiya Inst Technol & Sci Dept CSE Warangal 505015 Telangana India

In today's digital environment, business intelligence advances make it difficult to stay competitive and up to date on business trends. Decision-making in the financial industry is increasingly being powered by big data and machine learning. A decision-making process may be thought of as any sequence of processes that an individual goes through in order to select the option or course of action that is most suitable to meet their needs and necessities. The ability to anticipate the onset of a financial crisis is a significant economic phenomenon. A nation's economic development and strength can be gauged by its capacity to provide an accurate assessment of the number of failed firms and the frequency with which they fail. The economics of the globe have been ravaged by recent global crises like as the COVID-19 pandemic and other recent environmental, financial, and economic disasters, which have marginalized efforts to construct a maintainable economy and civilization. The health and growth of a nation's economy can be determined by precisely estimating the number of enterprises that will fail and the number that will succeed. Historically, there have been numerous strategies for constructing a successful financial crisis prediction (FPC) method. Effectively predicting business failures is a gauge of a country's economic health. Several strategies are available for effective FCP. Classification performance, forecast accuracy, and legality are insufficient for practical use. Several of the suggested methods work for some issues. The specific dataset is not expandable. To improve classification, design a good prediction model adaptable to several datasets. An effective financial crisis prediction method (FPC) requires the right qualities. ML models can also be used to classify a company's financial health. This research presents political optimizer-based feature selection (POFS) with optimal cascaded deep forest (OCDF) for FCP in big data environments. hadoop Map Reduc

关键词： Financial crisis prediction business intelligence political optimizer feature selection classification big data hadoop distributed file system political optimizer-based feature selection optimal cascaded deep forest

来源：评论

学校读者我要写书评

暂无评论

A HDFS dynamic load balancing strategy using improved niche PSO algorithm in cloud storage

引用

INTERNATIONAL JOURNAL OF AUTONOMOUS AND ADAPTIVE COMMUNICATIONS systemS 2021年第1-2期14卷 163-178页

作者： Jian, Zhiyu Jian, Yiwei Ordos Vocat Coll Ecoenvironm Graph & Text Informat Ctr Ordos 017010 Inner Mongolia Peoples R China Inner Mongolia Normal Univ Coll Comp Sci & Technol Hohhot 010000 Inner Mongolia Peoples R China

A hadoop distributed file system (HDFS) NameNode dynamic load balancing strategy (NDLBT) using improved niche particle swarm optimisation (PSO) is proposed, which is used to solve the load balancing problem of urban surveillance video big data. Firstly, the relationship between bandwidth consumption and the bit rate of video files, the size of data blocks and access hotspots of online video files are analysed. Then, adaptive backup is realised by dynamic multiple copies of heterogeneous nodes. The dynamic distribution of metadata is implemented to ensure the performance of metadata server clusters. Finally, we propose a new improved niche PSO algorithm to achieve load balancing scheduling. In the experimental scenario where high bandwidth consumption video files are used as service access hotspots, the proposed method, which can reduce the bandwidth peak value of bottleneck nodes in data node clusters by 20%, is superior to the original load balancing method in 90% scenarios.

关键词： cloud storage urban surveillance video improved niche PSO particle swarm optimisation dynamic load balancing big data HDFS hadoop distributed file system

来源：评论

学校读者我要写书评

暂无评论

Kafka-Shield: Kafka Streams-based distributed detection scheme for IoT traffic-based DDoS attacks

引用

SECURITY AND PRIVACY 2024年第6期7卷

作者： Shukla, Praveen Krishna, C. Rama Patil, Nilesh Vishwasrao Natl Inst Tech Teachers Training & Res Dept Comp Sci & Engn Chandigarh India Govt Polytech Dept Comp Engn Aurangabad Maharashtra India

With the rapid proliferation of insecure Internet of Things (IoT) devices, the security of Internet-based applications and networks has become a prominent concern. One of the most significant security threats encountered in IoT environments is a distributed Denial of Service (DDoS) attack. This attack can severely disrupt critical services and prevent smart devices from functioning normally, leading to severe consequences for businesses and individuals. It aims to overwhelm victims' resources, websites, and other services by flooding them with massive attack packets, making them inaccessible to legitimate users. Researchers have developed multiple detection schemes to detect DDoS attacks. As technology advances and other facilitating factors have increased, it is a challenge to identify such powerful attacks in real-time. In this paper, we propose a novel distributed detection scheme for IoT network traffic-based DDoS attacks by deploying it in a Kafka Streams processing framework named Kafka-Shield. The Kafka-Shield comprises two stages: design and deployment. Firstly, the detection scheme is designed on the hadoop cluster employing a highly scalable *** machine learning platform. Secondly, a portable, scalable, and distributed detection scheme is deployed on the Kafka Streams processing framework. To analyze the incoming traffic data and categorize it into nine target classes in real time. Additionally, Kafka-Shield stores each network flow with significant input features and the predicted outcome in the hadoop distributed file system (HDFS). It enables the development of new models or updating current ones. To validate the effectiveness of the Kafka-Shield, we performed critical analysis using various configured attack scenarios. The experimental results affirm Kafka-Shield's remarkable efficiency in detecting DDoS attacks. It has a detection rate of over 99% and can process 0.928 million traces in nearly 3.027 s.

关键词： Apache hadoop Bot-IoT dataset DDoS attacks H2O.ai machine learning platform hadoop distributed file system Internet of Things IoT devices Kafka Streams processing framework

来源：评论

学校读者我要写书评

暂无评论

Efficient Data Exchange Between Typical Data Lake and DWH Corporate systems

Efficient Data Exchange Between Typical Data Lake and DWH Co...

引用

IEEE International Conference on Electrical, Computer, and Energy Technologies (ICECET)

作者： Suleykin, Alexander Bobkova, Anna Panfilov, Peter Chumakov, Ilya Russian Acad Sci VA Trapeznikov Inst Control Sci Doctoral Sch Moscow Russia HSE Univ Grad Sch Business Dept Business Informat Moscow Russia Liebherr Int Deutschland GmbH Digital Mkt & ECommerce Liebherr Appliances Moscow Russia

ISBN: (纸本)9781665442312

In the last five years, many companies around the world have been successfully implemented Apache hadoop as a main Data Lake storage for all data presented in the organization. At the same time, the adoption of other Open-Source technologies has been also increasing for years, such as classical MPP-based systems for Analytical workloads. Thus, the question of efficient and fast data integration between Apache hadoop and other organizational data storage systems is highly important for enterprises, where business and decision makers need the minimum delay of big heterogeneous data exchange between hadoop and other storages. In this paper, we compare different options for loading data from Apache hadoop, representing the Data Lake of organization, into Open-Source MPP Greenplum database with the role of classical data warehouse for analytical workloads, and choose the best one. Also, we identify potential risks of using different data loading methods.

关键词： Big Data Data Lake hadoop distributed file system Greenplum Massively Parallel Processing Data Warehouse

来源：评论

学校读者我要写书评

暂无评论

Statistical analysis of an improved tuning method for optimizing performances of hadoop applications

引用

JOURNAL OF INFORMATION & OPTIMIZATION SCIENCES 2022年第3期43卷 533-541页

作者： Singh, Prabhdeep Prasad, M. S. Guru Taneja, Harsh Pai, H. Aditya Sharonchrista Graph Era Deemed Univ Dept Comp Sci & Engn Dehradun Dehra Dun Uttarakhand India Bharati Vidyapeeths Coll Engn Dept Comp Sci & Engn New Delhi India

hadoop is one among upcoming and most promising technologies for handling immense volume of data, generated from a variety of sources. It supports parallel processing among multiple jobs, and when such jobs are being executed on hadoop, it is of paramount importance to know if hadoop duster resources are being completely utilized or not, which in turn tells us about efficiency in resource utilization for the purpose of effective completion of jobs. It could happen that hadoop duster resources are not being completely utilized, and because of which there arises a need for performance tuning. Fortunately, hadoop framework equips us with several parameters to do the same and fine-tune aforesaid jobs. Performance tuning comprises of four main components: CPU utilization, Memory consumption, Disk I/O and Network traffic. In the proposed work, authors have presented several important relative parameters in relation to aforesaid components, along with how to fine-tune these parameters with appropriate values so as to optimize hadoop execution.

关键词： Big data distributed computing Parallel processing Tuning parameters Jaw virtual machine hadoop Yarn hadoop distributed file system MapReduce

来源：评论

学校读者我要写书评

暂无评论

An IoT based efficient Air pollution prediction system using DLMNN classifier

引用

PHYSICS AND CHEMISTRY OF THE EARTH 2022年第0期128卷

作者： Nemade, Bhushankumar Shah, Deven Mumbai Univ Thakur Coll Engn & Technol Dept Informat Technol Mumbai India Mumbai Univ Shree LR Tiwari Coll Engn Mira Rd Mumbai India

The Internet of Things (IoT) is a global system of "smart devices " which senses and connects with their environment and communicates with users as well as other systems. Air Pollution (AP) is one of the most significant global issues. Prevailing AP systems have low accuracy and require laboratory-based analysis. Therefore, improved prediction systems are needed. To overcome such problems, this paper proposes an IoT based efficient APprediction system utilizing the Deep Learning Modified Neural Network (DLMNN) classifier. Initially, the faulty node detection is done in the sensor nodes using the H-ANFIS algorithm. Here, ANFIS is hybridized with the K-Medoid algorithm. After that, the features are extracted from the sensed data and the unnecessary features are reduced by using the MPCA algorithm. Next, based on the reduced features, the data are balanced by using Entropy-HOA. Then, the balanced sensed data are pre-processed using replacing of missing attributes and HDFS. Next, the pre-processed data are tested with an AP prediction system employing the DLMNN classifier, where the Pity Beetle Algorithm (PBA) is used for weight optimization. Then, the predicted result is stored in the cloud server. Finally, the stored data is visualized. Experimental results have proved that the proposed system gives a better result than the existing systems.

关键词： DLMNN Deep Learning Modified Neural Network H-ANFIS Hybrid-Adaptive Neuro-Fuzzy Inference system MPCA Modified Principal Component Analysis Entropy-HOA Entropy-based Hummingbird Optimization Algorithm PBA Pitty Beetle Algorithm AP Air Pollution IoT Internet of Things ANN Artificial Neural Network AQM Air Quality Monitoring AQP Air Quality Prediction HDFS hadoop distributed file system SC Smart City LB Load Balancing DL Deep Learning

来源：评论

学校读者我要写书评

暂无评论

Maintaining data integrity in cloud systems through version management

引用

INTERNATIONAL JOURNAL OF AD HOC AND UBIQUITOUS COMPUTING 2020年第2期34卷 63-73页

作者： Yeh, Tsozen Wang, Yipin Tu, Yiming Fu Jen Catholic Univ Dept Comp Sci & Informat Engn New Taipei Taiwan

As the era of the big data arrives, the enormous amount of data collected has far exceeded what traditional computer systems can appropriately handle and process. Accordingly, cloud computing has been largely used to facilitate the processing of big data. Often individual data files contain data inserted at different time, which means they have chronological versions of contents since their creation. hadoop is one of the most popular cloud systems used nowadays. Unfortunately, it does not support efficient schemes to conduct version management for files. Previously, we improved hadoop by realising autonomous snapshot and extra duplication for files covered in snapshots. In this paper, we report our efforts to design and implement version management for files in snapshots. With the help of autonomous snapshot and extra file duplication, version management can further maintain data integrity for important files contained in snapshots.

关键词： big data data integrity cloud computing hadoop hadoop distributed file system HDFS

来源：评论

学校读者我要写书评

暂无评论

Deep neural network combined with MapReduce for abnormal data mining and detection in cloud storage

引用

JOURNAL OF AMBIENT INTELLIGENCE AND HUMANIZED COMPUTING 2020年 1-12页

作者： Yin, Chuan Pan, Canlin Zhang, Pengquan Hangzhou Dianzi Univ Sch Elect & Informat Hangzhou 310018 Zhejiang Peoples R China Henan Inst Sci & Technol Sch Informat Engn Xinxiang 453000 Henan Peoples R China Southeast Univ State Key Lab Millimeter Waves Nanjing 210096 Jiangsu Peoples R China

Aiming at the abnormal data behavior such as huge amount of data and easy to be stolen or lost in the process of distributed cloud computing in cloud storage environment, an abnormal data mining and detection algorithm of MapReduce based on hadoop distributed file system (HDFS) and deep neural network is proposed. Firstly, the algorithm analyzes the MAC timestamp characteristics generated by HDFS folder replication, establishes the detection and measurement methods of replication behavior, and ensures that all the patterns that lead to data anomalies, including theft, packet loss and malicious attack, can be detected. Secondly, the algorithm combines deep neural network to design a task partition strategy suitable for arbitrary MapReduce data, and records the input dataset of HDFS hierarchical relationship. Finally, combined with the parallel processing ability of MapReduce, the efficient analysis of massive timestamp data is realized by designing the dataset and algorithm execution scheme suitable for MapReduce task partition. The experimental results show that the algorithm can control the missed detection rate and the number of false detection folders through the segmentation detection strategy. Compared with the existing big data anomaly detection method, the algorithm has higher execution efficiency and good scalability.

关键词： hadoop distributed file system Deep neural network MapReduce Cloud storage Abnormal data mining and detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：