检索结果-内蒙古大学图书馆

Intelligent systems Conference

作者： Zhang, Yijie Wu, Chase Q. Hou, Aiqin New Jersey Inst Technol Newark NJ 07102 USA Northwest Univ Xian 710127 Shaanxi Peoples R China

ISBN: (纸本)9783031663284;9783031663291

hadoop stands as a cornerstone in the realm of big data processing, with its hadoop distributed file system (HDFS) serving as a pivotal layer ensuring fault tolerance and high throughput data storage. Through mechanisms such as block replication and cluster-wide distribution, HDFS facilitates parallel computing in higher layers. However, the inherent heterogeneity within hadoop clusters introduces complexities, particularly concerning the reliability of stored data. The failure of DataNodes within heterogeneous clusters poses a significant risk, potentially leading to data loss and compromising data reliability. Notably, the default block recovery strategy within HDFS overlooks the varying capacities of data nodes and the diverse patterns of data access, rendering it inadequate for heterogeneous environments. To address this gap, we first propose a novel approach for block recovery selection based on dueling double deep Q-networks, augmented with Gaussian Process Regression. We further formulate block recovery placement as an optimization problem in heterogeneous clusters, show its NP-completeness, and design an approximation algorithm that leverages linear programming-based iterative rounding (LPIR-BR), which offers a robust performance guarantee. Extensive experimental results validates the efficacy of LPIR-BR, showcasing its superiority over existing algorithms and affirming the soundness of our theoretical framework.

关键词： Big data Block distribution Approximation algorithm Dueling double deep Q-network hadoop distributed file system Performance bound Reliability

来源：评论

学校读者我要写书评

暂无评论

Blockchain-Based Dashcam Video Management Method for Data Sharing and Integrity in V2V Network

引用

IEEE ACCESS 2022年 10卷 3307-3319页

作者： Na, Dongjun Park, Sejin Keimyung Univ Dept Comp Engn Daegu 42601 South Korea

A dashboard camera (Dashcam) is attached to the front or rear of a vehicle to record images and video. As it is effective in crime prevention and accident management and it can also be used as learning data for Traffic Accident Detection technology for Autonomous Vehicles the use of Dashcam is increasing. However, the video data is stored on a memory card or a cloud server, so there is a high possibility of data loss and forgery. Although it is possible to prevent forgery and falsification using distributed storage based on blockchain technology, the authenticity and privacy of the image data stored in the blockchain cannot be guaranteed. In this study, to solve this problem, we propose a multi-signature-based access control method by grouping and storing video data of multiple vehicles based on GPS (Global Positioning system) data. To ensure the privacy of the video data stored in the blockchain, only users uploading video belonging to the relevant GPS can access nearby Dashcam videos. Through these experimental results, it show that Experimental results proposed method can maintain low latency in large-scale request environment to the privacy and reach data management efficiently in a distributed file system. According to our experiments, the dashcam video data distributed storage latency was fast enough, with an average of 25 ms for uploads and less than 15 ms for downloads. The signature generation time is 10ms, and the verification time required for access control is also less than 100 ms, which is the same even if the number of nodes increases, so scalability is not affected. The blockchain transaction processing took about 2 seconds, but it does not affect the V2V network. Also, client registration time does not affect performance.

关键词： Blockchains Accidents Forgery Access control Servers Global Positioning system Reliability Dashcam blockchain access control vehicle to vehicle multi signature oracle hadoop distributed file system

来源：评论

学校读者我要写书评

暂无评论

MapReduce based intelligent model for intrusion detection using machine learning technique

引用

JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES 2022年第10期34卷 9723-9731页

作者： Asif, Muhammad Abbas, Sagheer Khan, M. A. Fatima, Areej Khan, Muhammad Adnan Lee, Sang-Woong Natl Coll Business Adm & Econ Sch Comp Sci Lahore 54000 Pakistan Riphah Int Univ Riphah Sch Comp Innovat Lahore Campus Lahore 54000 Pakistan Lahore Garrison Univ Dept Comp Sci Lahore 54000 Pakistan Gachon Univ Dept Software Pattern Recognit & Machine Learning Lab Seongnam 13557 South Korea

With the emergence of the Internet of Things (IoT), the computer networks' phenomenal expansion, and enormous relevant applications, data is continuously increasing. In this way, cybersecurity has gained significant importance in protecting networks from different cyber-attacks like Intrusions, Denial-of-Service (DoS), Eavesdropping, Rushing Attack, etc. A traditional Intrusion Detection system (IDS) tangled with the clustering technique plays a vital role in modern security. Still, it has limitations to analyze the vast volumes of data to identify an anomaly intelligently. Machine learning is a technique that may be tangled with the MapReduce-Based Intelligent Model for Intrusion Detection (MR-IMID) to automate intrusion detection intelligently. MR-IMID is proposed to detect intrusions on a network with multiple data classification tasks in this research work. The proposed MR-IMID processes big data sets reliably using commodity hardware. In this proposed research work, multiple network sources are being utilized in Real-time for intrusion detection. In this proposed research, the MR-IMID detects intrusions by predicting unknown test scenarios and stores the data in the database to minimize future inconsistencies. The detection accuracy of the proposed model during training and validation phases is 97.7% and 95.7%, respectively, which is better than previously published approaches. (c) 2021 The Authors. Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://***/licenses/by-nc-nd/4.0/).

关键词： Denial-of-Service Intrusion detection system Cyber-attacks Network traffic hadoop distributed file system

来源：评论

学校读者我要写书评

暂无评论

Small files? problem in hadoop: A systematic literature review

引用

JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES 2022年第10期34卷 8658-8674页

作者： Aggarwal, Raveena Verma, Jyoti Siwach, Manvi JC Bose Univ Sci & Technol Comp Sci Engn Faridabad Haryana India

Apache hadoop is an open-source software library which integrates a wide variety of software tools and utilities to facilitate the distributed batch processing of big data sets. hadoop ecosystem comprises two major components - hadoop distributed file system (HDFS), which is primarily used for storage and MapReduce, which is primarily used for processing of the files. The performance of hadoop holds back when it comes to storage and processing of small files. Small files are essentially the files that are significantly smaller in size when compared to the default block size of HDFS. This is because each small file consumes a block individually leading to excessive memory requirement, access time and processing time. Scaling the memory, allowing access latencies and processing delays beyond a limit is not an option. Henceforth, in this paper, a systematic Literature Review has been performed to provide a comprehensive and exhaustive gestalt of the small files' problem in hadoop. The paper defines a comprehensive taxonomy of hadoop ecosystem and its' existent small files problem. Further, the study also attempts to critically analyze the solutions that have been proposed to overcome this problem. These solutions have been analytically studied to identify the set of parameters that should be considered in impending while proposing solutions pertaining to this problem. (c) 2021 The Authors. Published by Elsevier B.V. on behalf of King Saud University. This is an open access article under the CC BY-NC-ND license (http://***/licenses/by-nc-nd/4.0/).

关键词： hadoop hadoop distributed file system Small files MapReduce file merging

来源：评论

学校读者我要写书评

暂无评论

distributed lossless coding system based on cloud computing in video transcoding for MRI and neuroimaging

引用

JOURNAL OF ENGINEERING-JOE 2022年第11期2022卷 1059-1066页

作者： Li, Ying Shabaz, Mohammad Anhui Polytech Univ Sch Arts Wuhu Anhui Peoples R China Arba Minch Univ Hydraul & Water Resources Engn Arba Minch Ethiopia

A distributed lossless coding system based on cloud computing is proposed to increase the video transcoding capacity of the new media interactive broadcasting project in the research department. distributed transcoding is carried out by using the programming ideas of two hadoop cores: HDFS (hadoop distributed file system) and map reduce. At the same time, the specific process of distributed transcoding is introduced and designed in detail. The experimental results show that the efficiency of the distributed transcoding scheme is greatly improved. The results show that this experiment selects the 32 m with the best segment size as the benchmark, and calculates that the distributed transcoding time is 60.3% higher than the traditional single machine transcoding. This improvement significantly improves the transcoding efficiency, improves the experience effect of video service, and adapts to the trend of the rapid development of new media in the current era. The design of the system architecture not only greatly improves the transcoding efficiency, but also saves the transcoding time.

关键词： MRI distributed transcoding time data handling video service media interactive broadcasting project neurophysiology video transcoding capacity map reduce video coding neuroimaging distributed transcoding scheme distributed databases system architecture design transcoding biomedical MRI cloud computing transcoding efficiency distributed lossless coding system hadoop cores hadoop distributed file system parallel processing system architecture

来源：评论

学校读者我要写书评

暂无评论

Performance Impact of NVMe-Over-TCP on HDFS Workloads 15

Performance Impact of NVMe-Over-TCP on HDFS Workloads

引用

IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC) / 9th International Conference on Big Data Computing, Applications and Technologies (BDCAT)

作者： Sharma, Nikita Li, Ruihao Wu, Qinzhe John, Lizy K. Univ Texas Austin Dept Comp Sci Austin TX 78712 USA Univ Texas Austin Dept Elect & Comp Engn Austin TX 78712 USA

ISBN: (纸本)9781665460873

Storage is one of the important components in datacenters. As the data volume rises and the service scale grows, some workloads like database demand increasing amount of storage. While a single server can only host a limited number of disks, distributed file systems (e.g., hadoop distributed file system referred to as HDFS) enable accessing disks mounted on the other servers in the cluster, satisfying the storage requirements. On the other side, NVMe-over-Fabric protocols (e.g., NVMe-over-TCP) have been released as a solution on the device level to provide access to remote NVMe disks. Therefore, for those applications developed on top of HDFS, there are at least two choices to make use of the storage resources distributed in datacenters. A concern is whether NVMe-over-TCP will hurt the performance. The evaluation in this paper reveals that the performance degradation NVMe-over-TCP caused on HDFS-based workloads is limited, suggesting NVMe-over-TCP a performant and economical solution in datacenter design to support workloads that needs a lot of storage (such as database applications).

关键词： storage datacenter NVMe-over-TCP hadoop distributed file system

来源：评论

学校读者我要写书评

暂无评论

SLA-Aware Multi-Criteria Data Placement in Cloud Storage systems

引用

IEEE ACCESS 2021年 9卷 54369-54388页

作者： Maghsoudloo, Mohammad Rahdari, Arezoo Khoshavi, Navid Golestan Univ Dept Comp Engn Gorgan *** Golestan Iran Florida Polytech Univ Dept Comp Sci Lakeland FL 33805 USA Florida Polytech Univ Dept Elect & Comp Engn Lakeland FL 33805 USA

This paper proposes a multi-criteria data placement mechanism for typical cloud storage systems. The primary goal of this study is to place the clients' files in the storage cluster by taking into account the principles of service level agreements and the levels of user support services. For this purpose, a multi-criteria scoring model is defined based on a trade-off between three criteria that can be crucial from the clients' viewpoint: data access latency, data privacy, and computational performance. The second goal of the proposed mechanism is to improve the fairness of the storage allocation mechanism for various kinds of files compared to the conventional mechanisms. In this case, the cloud service provider gives each data storage node a chance to become the host of an already received file using a probabilistic ranking model. The experimental results demonstrate that the proposed data placement mechanism increases the robustness of cloud storage architecture under different file, network, and storage configurations compared to the existing data placement mechanisms.

关键词： Cloud computing Memory Data privacy Data models file systems Probabilistic logic Measurement Cloud storage systems hadoop distributed file system data placement service level agreement data storage nodes

来源：评论

学校读者我要写书评

暂无评论

An automated learning model for sentiment analysis and data classification of Twitter data using balanced CA-SVM

引用

CONCURRENT ENGINEERING-RESEARCH AND APPLICATIONS 2021年第4期29卷 386-395页

作者： Cyril, C. Pretty Diana Beulah, J. Rene Subramani, Neelakandan Mohan, Prakash Harshavardhan, A. Sivabalaselvamani, D. SRM Inst Sci & Technol Fac Engn & Technol Coll Engn & Technol Dept Comp Sci & Engn Chennai Tamil Nadu India Jeppiaar Inst Technol Dept Informat Technol Chennai Tamil Nadu India Karpagam Coll Engn Data Sci & Analyt Ctr Coimbatore 641032 Tamil Nadu India VNR Vignana Jyothi Inst Engn & Technol Dept Comp Sci & Engn Hyderabad Telangana India Kongu Engn Coll Dept Comp Applicat Erode Tamil Nadu India

The modern society runs over the social media for their most time of every day. The web users spend their most time in social media and they share many details with their friends. Such information obtained from their chat has been used in several applications. The sentiment analysis is the one which has been applied with Twitter data set toward identifying the emotion of any user and based on those different problems can be solved. Primarily, the data as of the Twitter database is preprocessed. In this step, tokenization, stemming, stop word removal, and number removal are done. The proposed automated learning with CA-SVM based sentiment analysis model reads the Twitter data set. After that they have been processed to extract the features which yield set of terms. Using the terms, the tweets are clustered using TGS-K means clustering which measures Euclidean distance according to different features like semantic sentiment score (SSS), gazetteer and symbolic sentiment support (GSSS), and topical sentiment score (TSS). Further, the method classifies the tweets according to support vector machine (CA-SVM) which classifies the tweet according to the support value which is measured based on the above two measures. The attained results are validated utilizing k-fold cross-validation methodology. Then, the classification is performed by utilizing the Balanced CA-SVM (Deep Learning Modified Neural Network). The results are evaluated and compared with the existing works. The Proposed model achieved 92.48 % accuracy and 92.05% sentiment score contrasted with the existing works.

关键词： sentiment analysis big data hadoop distributed file system SVM big data k-means ANN TSS machine learning

来源：评论

学校读者我要写书评

暂无评论

Data Processing on distributed systems Storage Challenges 4th

Data Processing on Distributed Systems Storage Challenges

引用

4th International Conference on Networks, Intelligent systems and Security (NISS)

作者： Eddoujaji, Mohamed Samadi, Hassan Bohorma, Mohamed Abdelmalek Essaadi Univ Natl Sch Appl Sci Tangier Tangier Morocco

ISBN: (纸本)9789811636370;9789811636363

hadoop is an open-source software framework for storing data and launching applications on standard machine clusters. This solution provides massive storage space for all types of data, tremendous processing power, and the ability to support virtually any amount of work. Based on Java, this framework is part of the Apache project, sponsored by Apache Software Foundation [hadoop official site. http://***/]. Thanks to the Map Reduce framework, it can handle huge amounts of data. Rather than having to move the data to a network for processing, MapReduce allows you to directly move the processing software to the data. No one will be able to neglect the results realized by the hadoop technique nor to question its power and its performance to manage important volumetrics of the data of any type: structured, semi-structured, or unstructured [https://***/hadoop]. But it does not shine by its processing speed when datasets are smaller and even smaller. This problem, the "small files" problem has been well-defined by the hadoop community, and also by the researchers, the majority of the proposed solutions and concepts only deal with the performance and the pressure exerted on the NameNode memory;some new approaches have been proposed as the casi-random, grouping of small heterogeneous files and in different formats, this solves the problem of memory because the number of metadata has been considerably reduced;the real challenge companies face is the performance of the hadoop cluster when processing a very large number of small files.

关键词： hadoop distributed file system Improve performance Small files in hadoop

来源：评论

学校读者我要写书评

暂无评论

Monitoring and Mitigating Climate-Induced Natural Disasters with Cloud IoT

引用

JOURNAL OF CLIMATE CHANGE 2024年第1期10卷 61-66页

作者： Taneja, Harsh Verma, Rohan Singh, Prabh Deep Singh, Kiran Deep Graphic Era Dehra Dun India Chitkara Univ Inst Engn & Technol Chandigarh Punjab India

Innovative solutions are necessary for efficient monitoring and mitigation techniques as climate change increases the frequency and severity of natural catastrophes. In this study, we investigate how Cloud Internet of Things (IoT) might help mitigate climate-related calamities. Through the use of cloud computing and sensor networks, Cloud IoT facilitates the capture, processing, and distribution of data in near-real time. This strategy promotes catastrophe planning, response, and recovery by offering early warnings, predictive insights, and simplified communication. The article emphasises the cumulative influence of Cloud IoT components such as sensor networks, data analytics, decision support systems, and remote control in the context of disaster management. Cloud IoT is useful in real-world scenarios, such as the tracking of floods in Bangladesh and the identification of wildfires in California. These cases show how the technology may prevent injuries and preserve property through early warnings and well-coordinated responses. Despite its potential, it faces obstacles including ensuring the safety of data and dealing with issues related to the necessary infrastructure. In conclusion, including Cloud IoT in disaster management provides a cost-effective, scalable, and efficient solution, greatly contributing to constructing resilient communities and creating a sustainable future in the face of climate-induced natural catastrophes.

关键词： Big data distributed computing Parallel processing Tuning parameters Java virtual machine hadoop Yarn hadoop distributed file system MapReduce

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：