检索结果-内蒙古大学图书馆

An Efficient Binary Locally Repairable Code for hadoop distributed file system

IEEE COMMUNICATIONS LETTERS 2014年第8期18卷 1287-1290页

作者： Shahabinejad, Mostafa Khabbazian, Majid Ardakani, Masoud Univ Alberta Dept Elect & Comp Engn Edmonton AB T6G 2V4 Canada

In the hadoop distributed file systems (HDFSs), to lower costly communication traffic for data recovery, the concept of locally repairable codes (LRCs) has been recently proposed. With regard to the immense size of modern energy-hungry HDFS, computational complexity reduction can be attractive. In this letter, to avoid finite field multiplications, which are the major source of complexity, we put forward the idea of designing binary locally repairable codes (BLRCs). More specifically, we design a BLRC with a length of 15, rate of 2/3, and minimum distance of 4, which has the minimum possible locality among its type. We show that our code has lower complexity than most recent non-binary LRC in the literature while meeting other desirable requirements in HDFS such as storage overhead and reliability.

关键词： Erasure coding hadoop distributed file system locally repairable codes

来源：评论

学校读者我要写书评

暂无评论

Performance Evaluation of Read and Write Operations in hadoop distributed file system 6

Performance Evaluation of Read and Write Operations in Hadoo...

引用

6th International Symposium on Parallel Architectures, Algorithms, and Programming (PAAP)

作者： Krishna, T. Lakshmi Siva Rama Ragunathan, T. Battula, Sudheer Kumar Jawaharlal Nehru Inst Adv Studies Hyderabad Andhra Pradesh India ACE Engn Coll Comp Sci & Engn Hyderabad Andhra Pradesh India

ISBN: (纸本)9781479938445

hadoop distributed file system (HDFS) is the core component of Apache hadoop project. In HDFS, the computation is carried out in the nodes where relevant data is stored. hadoop also implemented a parallel computational paradigm named as Map-Reduce. In this paper, we have measured the performance of read and write operations in HDFS by considering small and large files. For performance evaluation, we have used a hadoop cluster with five nodes. The results indicate that HDFS performs well for the files with the size greater than the default block size and performs poorly for the files with the size less than the default block size.

关键词： hadoop distributed file system hadoop distributed file system Map-Reduce

来源：评论

学校读者我要写书评

暂无评论

A Novel Indexing Scheme for Efficient Handling of Small files in hadoop distributed file system

A Novel Indexing Scheme for Efficient Handling of Small File...

引用

3rd International Conference on Computer Communication and Informatics (ICCCI)

作者： Chandrasekar, S. Dakshinamurthy, R. Seshakumar, P. G. Prabavathy, B. Babu, Chitra SSN Coll Engn Dept Comp Sci & Engn Kalavakkam Tamil Nadu India

ISBN: (纸本)9781467329071;9781467329064

hadoop distributed file system (HDFS) is designed for reliable storage and management of very large files. All the files in HDFS are managed by a single server, the NameNode. NameNode stores metadata, in its main memory, for each file stored into HDFS. As a consequence, HDFS suffers a performance penalty with increased number of small files. Storing and managing a large number of small files imposes a heavy burden on the NameNode. The number of files that can be stored into HDFS is constrained by the size of NameNode's main memory. Further, HDFS does not take the correlation among files into account, and it does not provide any prefetching mechanism to improve the I/O performance. In order to improve the efficiency of storing and accessing the small files on HDFS, we propose a solution based on the works of Dong et al., namely Extended hadoop distributed file system (EHDFS). In this approach, a set of correlated files is combined, as identified by the client, into a single large file to reduce the file count. An indexing mechanism has been built to access the individual files from the corresponding combined file. Further, index prefetching is also provided to improve I/O performance and minimize the load on NameNode. The experimental results indicate that EHDFS is able to reduce the metadata footprint on NameNode's main memory by 16% and also improve the efficiency of storing and accessing large number of small files.

关键词： hadoop distributed file system small file extended hdfs file correlation indexing prefetching

来源：评论

学校读者我要写书评

暂无评论

Securing big healthcare data using attribute and honey-based encryption in cloud environment

引用

JOURNAL OF SUPERCOMPUTING 2025年第1期81卷 1-28页

作者： Kapil, Gayatri Kumar, Neeraj Mourya, Ashish Kumar Kumar, Vijay GNIOT Inst Profess Studies Greater Noida Uttar Pradesh India Dr BR Ambedkar Natl Inst Technol Dept Informat Technol Jalandhar Punjab India ABESIT Dept Informat Technol Ghaziabad Uttar Pradesh India

The recent advancements on the Internet of clinical things, intelligent agents, data frameworks, and cloud administrations has prompted a digitalized change in the health industry. The healthcare data has been processed using several software-based techniques over the years. The cloud environment is used to reduce the time and space complexity. The size of available healthcare dataset is increased using different augmentation techniques. However, hospitals and other healthcare organizations have been worried about privacy and security of patients' data during transmission, storage, usage, and movement of information. During security breaches, several factors such as users, customers, partners, associations, and organizations are affected. This paper proposes an encryption framework that secures big data in healthcare on cloud technology. A ciphertext-policy attribute-based honey encryption (CP-ABHE) algorithm has been used to encode each sensitive public healthcare data document to enable the mechanism of versatile access control and fine-grained security for medical records. In the former stage, the security is enhanced with password protection, while in latter stage, honey words are produced to prevent unauthorized access by attackers.

关键词： Attribute-based Honey Encryption Ciphertext policy Cloud Server hadoop distributed file system Healthcare

来源：评论

学校读者我要写书评

暂无评论

A novel three-factor authentication and optimal mapreduce frameworks for secure medical big data transmission over the cloud with shaxecc

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2024年第26期83卷 68363-68391页

作者： Rajeshkumar, K. Dhanasekaran, S. Vasudevan, V. Kalasalingam Acad Res & Educ Krishnankoil 626126 India Koneru Lakshmaiah Educ Fdn Green Fileds Vaddeswaram 522302 Andhra Pradesh India

Big Data (BD) is a concept that deals with enormous amounts of data storage, processing, and analysis. With the exponential advancement in the evolution of cloud computing domains in healthcare (HC), the security and confidentiality of medical records have evolved into a primary consideration for HC services and applications. There needs to be more than the present-day cryptosystems to address these troubles. Therefore, this paper introduces a novel Three-Factor Authentication (3FA) and optimal Map-Reduce (MR) framework for secure BD transmission over the cloud with Secure Hashing Authentication XOR-ed Elliptical Curve Cryptography (SHAXECC). The authentication procedure is initially carried out with the SHA-512 algorithm, which protects the network from unauthorized access. Next, data deduplication is done using the SHA-512 algorithm to eliminate duplicate files. After that, an optimal MR design is introduced to handle a large amount of BD. In an optimal MR, the mapper uses the Modified Fuzzy C-means (MFCM) clustering approach to initially form the BD clusters. Then, the reducer uses the Levy Flight and Scoring Mutation-based Chimp Optimization Algorithm (LSCOA) to form final BD clusters. Finally, the SHAXECC is used to transmit the data securely. Experiments are performed to compare the superiority of the proposed technique with the existing techniques in terms of some performance measures. The proposed approach outperformed other existing models concerning clustering and security measures. So, the proposed model is the best for data protection and privacy in cloud-enabled HC data.

关键词： Medical big data Big data security Trusted center Map reduce Data deduplication hadoop distributed file system Fuzzy C-Means Chimp optimization algorithm

来源：评论

学校读者我要写书评

暂无评论

SXAD: Shapely eXplainable AI-Based Anomaly Detection Using Log Data

引用

IEEE ACCESS 2024年 12卷 95659-95672页

作者： Alam, Kashif Kifayat, Kashif Sampedro, Gabriel Avelino Karovic Jr, Vincent Naeem, Tariq Air Univ Fac Comp & AI Dept Comp Sci Islamabad 44000 Pakistan Air Univ Fac Comp & AI Dept Cyber Secur Islamabad 44000 Pakistan De La Salle Coll St Benilde Sch Management & Informat Technol Manila 1004 Philippines Comenius Univ Fac Management Dept Informat Management & Business Syst Bratislava 82005 Slovakia

Artificial Intelligence (AI) has made tremendous progress in anomaly detection. However, AI models work as a black-box, making it challenging to provide reasoning behind their judgments in a Log Anomaly Detection (LAD). To the rescue, Explainable Artificial Intelligence (XAI) improves system log analysis. It follows a white-box model for transparency, understandability, trustworthiness, and dependability of Machine Learning (ML) and Deep Learning (DL) Models. In addition, Shapely Additive Explanation (SHAP), added to system dynamics, makes informed judgments and adoptable proactive methods to optimize system functionality and reliability. Therefore, this paper proposed the Shapely eXplainable Anomaly Detection (SXAD) framework to identify different events (features) that impact the models' interpretability, trustworthiness, and explainability. The framework utilizes the Kernel SHAP approach, which is based on Shapley values principle, providing an innovative approach to event selection and identifying specific events causing abnormal behavior. This study addresses the LAD by transforming it from a black-box model into a white-box one, leveraging XAI to make it transparent, interpretable, explainable, and dependable. It utilizes benchmark data from the hadoop distributed file system (HDFS), organized using a Drain parser, and employs several ML models, such as Decision Tree (DT), Random Forest (RF), and Gradient Boosting (GB). These models achieve impressive accuracy rates of 99.99%, 99.85%, and 99.99%, respectively. Our contribution are novel because no earlier work has been done in the area of Log Anomaly Detection (LAD) with integration of XAI-SHAP.

关键词： Explainable artificial intelligence Shapley additive explanation hadoop distributed file system machine learning Explainable artificial intelligence Shapley additive explanation hadoop distributed file system machine learning

来源：评论

学校读者我要写书评

暂无评论

Dynamic Replication Policy on HDFS Based on Machine Learning Clustering

引用

IEEE ACCESS 2023年 11卷 18551-18559页

作者： Ahmed, Motaz A. Khafagy, Mohamed H. Shaheen, Masoud E. Kaseb, Mostafa R. Fayoum Univ Fac Comp & Artificial Intelligence Dept Comp Sci Al Fayyum 63514 Egypt

Data growth in recent years has been swift, leading to the emergence of big data science. distributed file systems (DFS) are commonly used to handle big data, like Google file system (GFS), hadoop distributed file system (HDFS), and others. The DFS should provide the availability of data and reliability of the system in case of failure. The DFS replicates the files in different locations to provide availability and reliability. These replications consume storage space and other resources. The importance of these files differs depending on how frequently they are used in the system. So some of these files do not deserve to replicate many times because it is unimportant in the system. This paper introduces a Dynamic Replication Policy using Machine Learning Clustering (DRPMLC) on HDFS, which uses Machine Learning to cluster the files into different groups and apply other replication policies to each group to reduce the storage consumption, improve the read and write operations time and keep the availability and reliability of HDFS as a High-Performance distributed Computing (HPDC).

关键词： Feature extraction file systems Machine learning Databases Big Data distributed computing Support vector machines Replicability Availability big data clustering hadoop distributed file system high-performance distributed computing machine learning reliability replication policy

来源：评论

学校读者我要写书评

暂无评论

Efficient and secure medical big data management system using optimal map-reduce framework and deep learning

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2023年第16期83卷 47111-47138页

作者： Rajeshkumar, K. Dhanasekaran, S. Vasudevan, V. Kalasalingam Acad Res & Educ Dept Comp Sci & Engn Krishnankoil Tamil Nadu India Kalasalingam Acad Res & Educ Dept Informat Technol Krishnankoil Tamil Nadu India

Big Data (BD) and cloud computing (CC) are the two widely used technologies and focus of study in several industries. Amongst all, healthcare sources generate a tremendous amount of data daily. Traditional processing techniques cannot handle this data because they are huge. Furthermore, being large, this data is also dynamic and diverse. Large data sets are stored, processed, and analyzed under BD. However, as the volume of data increased, third parties could hack it easily. The data are regularly stored in the cloud in an encrypted form to protect the data from intruders. This paper proposes a secure and efficient medical BD management and classification scheme using optimal Map Reduce (MR) and deep learning framework in a cloud environment. The system comprises '4' phases: authentication of patients, BD management in the cloud, secure data transfer and BD classification. First, the patient who wants to upload the files to the cloud and access the resources from the cloud is registered with the TC. It generates hash value using the Whirlpool Hashing (WH) algorithm, and the user credentials are stored in Blockchain (BC) to protect the network from unauthorized access. Once the authentication is successful, the patient can access any data in the cloud. Before uploading the file to the cloud, the preprocessing is carried out using missing values imputation, numerical conversion, and normalization, which improves the data quality. It is fed into the MR framework using Kernelized K-Means (KKM) clustering and Enhanced Butterfly Optimization Algorithm (EBOA) for managing the data efficiently. Then the map-reduced data is encrypted using Whirlpool Hashing-based Enhanced Rivest Shamir Adelman (WHERSA) to upload into the cloud securely. Finally, the classification of BD is done using Enhanced Ant Colony Weight Optimization based Deep Belief Network (EACWODBN) for disease prediction. The experimental outcomes reveal that the proposed system outperformed state-of-art methods w

关键词： Cloud Computing Big data hadoop distributed file system Blockchain Map Reduce Electronic Health Records Encryption and Decryption

来源：评论

学校读者我要写书评

暂无评论

A Heuristic Approach to Improve the Data Processing in Big Data using Enhanced Salp Swarm Algorithm (ESSA) and MK-means Algorithm

引用

JOURNAL OF INTELLIGENT & FUZZY systemS 2023年第2期45卷 2625-2640页

作者： Sundarakumar, M. R. Nayagi, D. Salangai Vinodhini, V. VinayagaPriya, S. Marimuthu, M. Basheer, Shajahan Santhakumar, D. Renoald, A. Johny Galgotias Univ Sch Comp Sci & Engn Greater Noida Uttar Pradesh India New Horizon Coll Engn Dept CSE Bengaluru India Sona Coll Technol Dept CSE Salem India St Josephs Coll Engn Dept ECE Chennai Tamil Nadu India Jain Univ Dept Comp Sci & Engn Kanakapura India CK Coll Engn & Technol Dept CSE Cuddalore Tamil Nadu India Erode Sengunthar Engn Coll Dept EEE Erode Tamil Nadu India

Improving data processing in big data is a delicate procedure in our current digital era due to the massive amounts of data created by humans and machines in daily life. Handling this data, creating a repository for storage, and retrieving photos from internet platforms is a difficult issue for businesses and industries. Currently, clusters have been constructed for many types of data, such as text, documents, audio, and video files, but the extraction time and accuracy during data processing remain stressful. hadoop distributed file system (HDFS) is a system that provides a large storage area in big data for managing large datasets, although the accuracy level is not as high as desired. Furthermore, query optimization was used to produce low latency and high throughput outcomes. To address these concerns, this study proposes a novel technique for query optimization termed the Enhanced Salp Swarm Algorithm (ESSA) in conjunction with the Modified K-Means Algorithm (MKM) for cluster construction. The process is separated into two stages: data collection and organization, followed by data extraction from the repository. Finally, numerous experiments with assessments were carried out, and the outcomes were compared. This strategy provides a more efficient method for enhancing data processing speed in a big data environment while maintaining an accuracy level of 98% while processing large amounts of data.

关键词： hadoop distributed file system latency throughput query optimization hash algorithms clustering

来源：评论

学校读者我要写书评

暂无评论

Microblog Emotion Analysis Using Improved DBN Under Spark Platform

引用

INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGIES AND systemS APPROACH 2023年第2期16卷 1-16页

作者： Chang, Wanjun Li, Yangbo Du, Qidong Henan Inst Technol Zhengzhou Peoples R China Guangzhou Railway Polytechn Guangzhou Peoples R China

In order to solve the problems that traditional single-machine methods find it difficult to complete the task of emotion classification quickly, and the time efficiency and scalability are not high;a microblog emotion analysis method using improved deep belief network (DBN) under Spark platform is proposed. First, the hadoop distributed file system is used to realize the distributed storage of text data, and the preprocessed data and emotion dictionary are converted into word vector representation based on the continuous bag-of-words model. Then, an improved DBN model is constructed by combining the adaptive learning method of DBN with the active learning method, and it is applied to the learning analysis of text word vectors. Finally, the data parallel optimization of the improved DBN model is realized, based on Spark platform to accurately and quickly obtain the emotion types of microblog texts. The experimental analysis of the proposed method based on the microblog text data set shows that the classification accuracy is more than 94%.

关键词： Big Data Analysis Data Parallel Optimization Deep Learning hadoop distributed file system Improved DBN Model Microblog Emotion Analysis Spark Platform

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：