检索结果-内蒙古大学图书馆

Cellular automata-based MapReduce design: Migrating a big data processing model from Industry 4.0 to Industry 5.0

e-Prime - Advances in Electrical Engineering, Electronics and Energy 2024年 8卷

作者： Mitra, Arnab Department of Information Technology Techno International New Town West Bengal Kolkata 700156 India

A successful deployment of Industry 5.0 is significantly dependent on the synergetic integration of several advanced technologies such as big data processing, Artificial Intelligence (AI) integration, and several effective digitization techniques that emphasize the uses of Robotics, Internet of Things (IoTs), Cloud Computing, etc. with active participations from human workers. Several researchers have explored the importance of big data processing in view of Industry 4.0 as it facilitated enhanced production at any smart manufacturing line by ensuring efficient process management, typically involving big data processing. Researchers presented several MapReduce-based data processing models at smart manufacturing lines to facilitate big data processing. Among several others, the Elementary Cellular Automata (ECAs)-based MapReduce model was introduced as an energy-efficient low-cost model for big data processing in view of the Industry 4.0 scenario. In the present research, an investigation is further proposed to explore the true potential (if any) for the ECAs-based MapReduce model with reference to available several MapReduce models, to migrate an existing big data processing model from Industry 4.0 into the future i.e., Industry 5.0. Investigation results as achieved from the comparison among several MapReduce models and further examinations about the inherent quality of shuffle in those MapReduce blocks, explore the inherent advantages of ECAs-based MapReduce model towards its choice for big data processing in Industry 5.0 scenario. © 2024 The Author(s)

关键词： big data processing Cyber-physical systems (CPS) Elementary cellular automata (ECAs) Green computing Industry 4.0 Industry 5.0 MapReduce model Sustainability

来源：评论

学校读者我要写书评

暂无评论

Fault tolerance in big data storage and processing systems: A review on challenges and solutions

引用

AIN SHAMS ENGINEERING JOURNAL 2022年第2期13卷 101538-101538页

作者： Saadoon, Muntadher Ab Hamid, Siti Hafizah Sofian, Hazrina Altarturi, Hamza H. M. Azizul, Zati Hakim Nasuha, Nur Univ Malaya Fac Comp Sci & Informat Technol Dept Software Engn Kuala Lumpur 50603 Malaysia

big data systems are sufficiently stable to store and process a massive volume of rapidly changing data. However, big data systems are composed of large-scale hardware resources that make their subspecies easily fail. Fault tolerance is the main property of such systems because it maintains availability, reliability, and constant performance during faults. Achieving an efficient fault tolerance solution in a big data system is challenging because fault tolerance must meet some constraints related to the system performance and resource consumption. This study aims to provide a consistent understanding of fault tolerance in big data systems and highlights common challenges that hinder the improvement in fault tolerance efficiency. The fault tolerance solutions applied by previous studies intended to address the identified challenges are reviewed. The paper also presents a perceptive discussion of the findings derived from previous studies and proposes a list of future directions to address the fault tolerance challenges. (C) 2021 THE AUTHORS. Published by Elsevier BV on behalf of Faculty of Engineering, Ain Shams University.

关键词： Fault tolerance Fault detection Fault recovery big data storage big data processing

来源：评论

学校读者我要写书评

暂无评论

Optimizing Apache Spark MLlib: Predictive Performance of Large-Scale Models for big data Analytics

引用

ALGORITHMS 2025年第2期18卷 74-74页

作者： Theodorakopoulos, Leonidas Karras, Aristeidis Krimpas, George A. Univ Patras Dept Management Sci & Technol Patras 26334 Greece Univ Patras Comp Engn & Informat Dept Patras 26504 Greece

In this study, we analyze the performance of the machine learning operators in Apache Spark MLlib for K-Means, Random Forest Regression, and Word2Vec. We used a multi-node Spark cluster along with collected detailed execution metrics computed from the data of diverse datasets and parameter settings. The data were used to train predictive models that had up to 98% accuracy in forecasting performance. By building actionable predictive models, our research provides a unique treatment for key hyperparameter tuning, scalability, and real-time resource allocation challenges. Specifically, the practical value of traditional models in optimizing Apache Spark MLlib workflows was shown, achieving up to 30% resource savings and a 25% reduction in processing time. These models enable system optimization, reduce the amount of computational overheads, and boost the overall performance of big data applications. Ultimately, this work not only closes significant gaps in predictive performance modeling, but also paves the way for real-time analytics over a distributed environment.

关键词： Apache Spark MLlib big data processing big data analytics performance prediction machine learning resource optimization feature engineering emerging technologies decision-making

来源：评论

学校读者我要写书评

暂无评论

Supporting Efficient Family Joins for big data Tables via Multiple Freedom Family Index

引用

IEEE ACCESS 2025年 13卷 21707-21722页

作者： Zhu, Qiang Zhu, Chao Univ Michigan Dearborn Dept Comp & Informat Sci Dearborn MI 48128 USA

The Hadoop/MapReduce framework has been widely utilized for processing big data. To overcome the limitations of existing work and meet the growing requirements of querying big data, this paper introduces novel join operations, called family joins, for HBase tables using their column families as join keys. Family joins possess the closure property that is demanded by many big data applications. This work explores four types of family joins according to different types of freedom in prefix matching for join comparisons. Two approaches to processing such family joins are discussed. The first is the direct method, which is inspired by the straightforward nested-loop strategy. The second is an index-based method, which utilizes a special index for HBase tables. Detailed definitions, practical applications, and processing strategies and algorithms for family joins are provided. Experimental results demonstrate that the index-based join method is quite promising in efficiently processing family joins.

关键词： big data Indexes Cellular phones Landline Query processing Programming Algebra Time series analysis Reviews Open source software database HBase big data processing family join query optimization index

来源：评论

学校读者我要写书评

暂无评论

Text reuse in large historical corpora: insights from the optimization of a data science system

引用

INTERNATIONAL JOURNAL OF data SCIENCE AND ANALYTICS 2025年 1-13页

作者： Mahadevan, Ananth Mathioudakis, Michael Makela, Eetu Tolonen, Mikko Univ Helsinki Dept Comp Sci Helsinki Finland Univ Helsinki Dept Digital Humanities Helsinki Finland

Text reuse is of fundamental importance in humanities research, as near-verbatim pieces of text in different documents provide invaluable information about the historical spread, evolution of ideas and composition of cultural artifacts. Traditionally, scholars have studied text reuse at a very small scale, for example, when comparing the writings of two philosophers;however, modern digitized corpora spanning entire centuries promise to revolutionize humanities research through the detection of previously unobserved large-scale patterns. This paper presents insights from ReceptionReader, a system for large-scale text reuse analysis over almost all known 18th-century books, articles, and newspapers. The system implements a data management pipeline for billions of text reuse instances and supports analysis tasks based on database queries (e.g., retrieving the most reused quotes from queried documents). The paper describes the principled and extensive evaluations across different normalization levels, query execution engines, and queries of interest that led to an optimized system-and offers insights from the observed trade-offs and how they were resolved to fit specific requirements. In summary, the paper explains how, for our system, (1) the row-store engine (MariaDB Aria) with denormalized relations emerged as the optimal choice for front-end interfaces, while (2) big data processing (Apache Spark) proved irreplaceable for data preprocessing.

关键词： data science system optimization big data processing Historical corpora Text reuse

来源：评论

学校读者我要写书评

暂无评论

big data Analytics Using Graph Signal processing

引用

Computers, Materials & Continua 2023年第1期74卷 489-502页

作者： Farhan Amin Omar M.Barukab Gyu Sang Choi Department of Information and Communication Engineering Yeungnam UniversityGyeongsan38541Korea Faculty of Computing and Information Technology-Rabigh King Abdulaziz UniversityJeddahSaudi Arabia

The networks are fundamental to our modern world and they appear throughout science and *** to a massive amount of data presents a unique opportunity to the researcher’s *** networks grow in size the complexity increases and our ability to analyze them using the current state of the art is at severe risk of failing to keep ***,this paper initiates a discussion on graph signal processing for large-scale data *** first provide a comprehensive overview of core ideas in Graph signal processing(GSP)and their connection to conventional digital signal processing(DSP).We then summarize recent developments in developing basic GSP tools,including methods for graph filtering or graph learning,graph signal,graph Fourier transform(GFT),spectrum,graph frequency,*** filtering is a basic task that allows for isolating the contribution of individual frequencies and therefore enables the removal of *** then consider a graph filter as a model that helps to extend the application of GSP methods to large *** show the suitability and the effeteness,we first created a noisy graph signal and then applied it to the *** several rounds of simulation *** see that the filtered signal appears to be smoother and is closer to the original noise-free distance-based *** using this example application,we thoroughly demonstrated that graph filtration is efficient for big data analytics.

关键词： big data data science big data processing graph signal processing social networks

来源：评论

学校读者我要写书评

暂无评论

An Efficient and Scalable Framework for processing Remotely Sensed big data in Cloud Computing Environments

引用

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 2019年第7期57卷 4294-4308页

作者： Sun, Jin Zhang, Yi Wu, Zebin Zhu, Yaoqin Yin, Xianliang Ding, Zhongzheng Wei, Zhihui Plaza, Javier Plaza, Antonio Nanjing Univ Sci & Technol Sch Comp Sci & Engn Nanjing 210094 Jiangsu Peoples R China Nanjing Robot Res Inst Co Ltd Nanjing 211135 Jiangsu Peoples R China Univ Extremadura Hyperspectral Comp Lab Dept Technol Comp & Commun Caceres 10071 Spain

The large amount of data produced by satellites and airborne remote sensing instruments has posed important challenges to efficient and scalable processing of remotely sensed data in the context of various applications. In this paper, we propose a new big data framework for processing massive amounts of remote sensing images on cloud computing platforms. In addition to taking advantage of the parallel processing abilities of cloud computing to cope with large-scale remote sensing data, this framework incorporates task scheduling strategy to further exploit the parallelism during the distributed processing stage. Using a computation- and data-intensive pan-sharpening method as a study case, the proposed approach starts by profiling a remote sensing application and characterizing it into a directed acyclic graph (DAG). With the obtained DAG representing the application, we further develop an optimization framework that incorporates the distributed computing mechanism and task scheduling strategy to minimize the total execution time. By determining an optimized solution of task partitioning and task assignments, high utilization of cloud computing resources and accordingly a significant speedup can be achieved for remote sensing data processing. Experimental results demonstrate that the proposed framework achieves promising results in terms of execution time as compared with the traditional (serial) processing approach. Our results also show that the proposed approach is scalable with regard to the increasing scale of remote sensing data.

关键词： big data processing cloud computing remote sensing task scheduling

来源：评论

学校读者我要写书评

暂无评论

Trustworthy processing of Healthcare big data in Hybrid Clouds

引用

IEEE CLOUD COMPUTING 2015年第2期2卷 78-84页

作者： Nepal, Surya Ranjan, Rajiv Choo, Kim-Kwang Raymond CSIRO Canberra ACT Australia Univ S Australia Sch Informat Technol & Math Sci Adelaide SA 5001 Australia

As we delve deeper into the "Digital Age," we're witnessing an explosive growth in the volume, velocity, variety, veracity, and value (the 5Vs) of data produced over the Internet. According to recent Cisco(1) and IBM2 reports, we now generate 2.5 quintillion bytes of data per day, and this is set to explode to 40 yottabyes by 2020(3)-that is, 5,200 gigabytes for every person on the earth. As noted in previous "Blue Skies" columns, data generated by Internet of Things (IoT) devices and sensors are part of the big data landscape. 4,5 IoT comprises billions of Internet-connected devices (ICDs) or "things," each of which can sense, communicate, compute, and potentially actuate, and can have intelligence, multimodal interfaces, physical/virtual identities, and attributes. ICDs can be mobile devices, sensors, medical imaging devices, individual archives, social networks, smart cameras, body sensors, automobile cosimulations, or software logs. In a nutshell, a large volume of veracity data is generated at high velocity from a variety of sources.

关键词： Cloud Computing big data Biomedical Imaging Medical Services Cryptography data Privacy Privacy Cloud Computing big data Healthcare big data processing Internet Connected Devices

来源：评论

学校读者我要写书评

暂无评论

A Multi-Dimensional Trust Model for processing big data Over Competing Clouds

引用

IEEE ACCESS 2018年 6卷 39989-40007页

作者： El Kassabi, Hadeel T. Serhani, Mohamed Adel Dssouli, Rachida Benatallah, Boualem Concordia Univ Concordia Inst Informat Syst Engn Montreal PQ H4B 1R6 Canada United Arab Emirates Univ Coll Informat Technol Al Ain 15551 U Arab Emirates Univ New South Wales Sch Comp Sci & Engn Kensington NSW 2052 Australia

Cloud computing has emerged as a powerful paradigm for delivering data-intensive services over the Internet. Cloud computing has enabled the implementation and success of big data, a recent phenomenon handling huge data being generated from different sources. Competing clouds have made it challenging to select a cloud provider that guarantees quality of cloud service (QoCS). Also, cloud providers' claims of guaranteeing QoCS are exaggerated for marketing purposes;hence, they cannot often be trusted. Therefore, a comprehensive trust model is necessary to evaluate the QoCS prior to making any selection decision. In this paper, we propose a multi-dimensional trust model for big data workflow processing over different clouds. It evaluates the trustworthiness of cloud providers based on: the most up-to-date cloud resource capabilities, the reputation evidence measured by neighboring users, and a recorded personal history of experiences with the cloud provider. The ultimate goal is to ensure an efficient selection of trustworthiness cloud provider who eventually will guarantee high QoCS and fulfills key big data workflow requirements. Various experiments were conducted to validate our proposed model. The results show that our model captures the different components of trust, ensures high QoCS, and effectively adapts to the dynamic nature of the cloud.

关键词： big data big data processing cloud computing cloud selection trust model quality of cloud services service evaluation community

来源：评论

学校读者我要写书评

暂无评论

High-efficient energy saving processing of big data of communication under mobile cloud computing

引用

International Journal of Modeling, Simulation, and Scientific Computing 2019年第4期10卷 96-106页

作者： Yazhen Liu Pengfei Fan Jiyang Zhu Liping Wen Xiongfei Fan Information Communication Operation and Maintenance Center Information and Communication Branch State Grid Inner Mongolia Eastern Electric Power Co. Ltd.HohhotInner Mongolia Autonomous Region 010020P.R.China Hohhot Power Supply Bureau Inner Mongolia Power(Group)Co.Ltd.HohhotInner Mongolia Autonomous Region 010020P.R.China Training Center Inner Mongolia Power(Group)Co.Ltd.HohhotInner Mongolia Autonomous Region 010020P.R.China

From 21st century,it is hard for traditional storage and algorithm to provide service with high quality because of big data of communication which grows ***,cloud computing technology with relatively low cost of hardware facilities is ***,to guarantee the quality of service in the situation of the rapid growth of data volume,the energy consumption cost of cloud computing begins to exceed the hardware *** order to solve the problems mentioned above,this study briefly introduced the virtual machine and its energy consumption model in the mobile cloud environment,introduced the basic principle of the virtual machine migration strategy based on the artificial bee colony algorithm and then simulated the performance of processing strategy to big data of communication based on artificial bee colony algorithm in mobile cloud computing environment by CloudSim3.0 software,which was compared with the performance of two algorithms,resource management(RM)and genetic algorithm(GA).The results showed that the power consumption of the migration strategy based on the artificial bee colony algorithm was lower than the other two strategies,and there were fewer failed virtual machines under the same number of requests,which meant that the service quality was higher.

关键词： Mobile cloud computing big data processing artificial bee colony algorithm energy saving

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：