检索结果-内蒙古大学图书馆

MapReduce-Based Improved Random Forest Model for Massive Educational data processing and Classification

MOBILE NETWORKS & APPLICATIONS 2021年第1期26卷 191-199页

作者： Xu, Wei Hoang, Vinh Truong Xian Univ Finance & Econ Business Sch Xian 710100 Shaanxi Peoples R China Ho Chi Minh City Open Univ Fac Comp Sci Ho Chi Minh City 70000 Vietnam

This paper takes education data mining as the research theme, mine the existing massive education big data, compares the analysis methods of existing data models, and proposes an improved random forest reference model. The information gain of various features is calculated by introducing the feature weighting system, and the evaluation index is used to improve the existing data analysis. The simulation results show that the improved model is highly efficient as compared to the existing models for classification. In order to resolve the performance bottleneck of a single node in multiple data classification tasks in the era of big data, a classification and prediction model of graduates' large-scale employment data, based on distributed improved RF algorithm, is proposed. The MapReduce distributed computing framework is used to complete the serial writing and deserialization loading of the training model between the local disk and the distributed file system, and realizing the distributed expansion of the large-scale data classification model based on the improved RF model.

关键词： Machine learning data classification model big data processing MapReduce

来源：评论

学校读者我要写书评

暂无评论

An efficient architecture for processing real-time traffic data streams using apache flink

引用

MULTIMEDIA TOOLS AND APPLICATIONS 2023年第13期83卷 37369-37385页

作者： Deepthi, B. Gnana Rani, K. Sandhya Krishna, P. Venkata Saritha, V. Sri Padmavati Mahila Visvavidyalayam Dept Comp Sci & Engn Tirupati India

big data technologies emerging day by day and are making drastic changes in various real-world applications. Traditional data mining tools adequate to process volumes of data but from past decades the rapid growth in data becomes difficult for processing. Due to continuous flow of data, data streams require additional computational processing than the traditional one. big data stream processing considers different features of the data streams heterogeneity, scalability, fault tolerance and query optimization. Efficient implementation of these features in real-world applications using big data analytics is a challenging job during data storage, processing, and analysis phases. Therefore, the proposed model FRTSPS is a generic architecture which is influenced by popular big data processing Lambda architecture, based on distributed computing platform. The architecture using open-source platform Apache Flink for doing data processing. Flink is a popular platform for processing historical and stream data flows at once parallelly. Its stateful streaming can obtain more scalability and flexibility along with high throughput and low latency than the remaining stream processing programming models.

关键词： big data big data processing Stream Computing Apache Flink

来源：评论

学校读者我要写书评

暂无评论

Profit Maximization of big data Jobs in Cloud Using Stochastic Optimization

引用

IEEE TRANSACTIONS ON CLOUD COMPUTING 2021年第4期9卷 1563-1574页

作者： Nabavinejad, Seyed Morteza Goudarzi, Maziar Inst Res Fundamental Sci IPM Sch Comp Sci Tehran Iran Sharif Univ Technol Dept Comp Engn Tehran Iran

Reserved instances offered by cloud providers make it possible to reserve resources and computing capacity for a specific period of time. One should pay for all the hours of that time interval;in exchange, the hourly rate is significantly lower than on-demand instances. Reserved Instances can significantly reduce the monetary cost of resources needed to process big data applications in cloud. However, purchases of these instances are non-refundable, and hence, one should be able to estimate the required resources prior to purchase to avoid over-payment. It becomes important especially when the results obtained by big data job has monetary value, such as business intelligence applications. But, estimating the resource demand of big data processing jobs is hard because of numerous factors that affect them such as data locality, data skew, stragglers, internal settings of big data processing framework, interference among instances, instances availability, etc. To maximize the profit of processing such big data jobs in cloud considering fluctuating nature of their resource demand, as well as reserved instances limitations, we propose Reserved Instances Stochastic Allocation (RISA) approach. Using historical traces of resource demand of big data jobs submitted by user, RISA leverages stochastic optimization to determine the amount of resources needed to be reserved for that user to maximize the profit. Our evaluation using real-world traces shows that RISA can increase the net profit by up to 10x, compared to previous approaches. RISA can also find solutions as close as 2 percent to the best possible solution.

关键词： Cloud computing big data processing reserved instances stochastic optimization profit maximization

来源：评论

学校读者我要写书评

暂无评论

Adia: Achieving High Link Utilization with Coflow-Aware Scheduling in data Center Networks

引用

IEEE TRANSACTIONS ON CLOUD COMPUTING 2019年第2期7卷 431-441页

作者： Jiang, Jingjie Ma, Shiyao Li, Bo Li, Baochun Hong Kong Univ Sci & Technol Dept Comp Sci & Engn Clear Water Bay Hong Kong Peoples R China Univ Toronto Dept Elect & Comp Engn Toronto ON M5S Canada

Link utilization has received extensive attention since data centers become the most pervasive platform for data-parallel applications. A specific job of such applications involves communication among multiple machines. The recently proposed coflow abstraction depicts such communication through a group of parallel flows, and captures application performance through corresponding communication requirements. Existing techniques to improve link utilization, however, either restrict themselves to achieving work conservation, or merely focus on flow-level metrics and ignore coflow-level performance. In this paper, we address the coflow-aware scheduling problem with the objective of maximizing link utilization. Through theoretic analyses, we formulate the coflow-aware scheduling problem as a NP-hard open shop scheduling problem with heterogeneous concurrency. We design Adia, a hierarchical scheduling framework to conduct both inter- and intra- link scheduling. The design of Adia leverages priority-based scheduling while guarantees work-conserving and starvation-free bandwidth allocation at the same time. We also prove Adia's algorithm is two-approximate in terms of link utilization. Extensive simulation results on ns3 further show that Adia outperforms both per-flow mechanisms coflow schemes in terms of link utilization, and achieves similar coflow performance in comparison with the stateof-art coflow scheduling schemes.

关键词： datacenter networks parallel computing big data processing resource utilization

来源：评论

学校读者我要写书评

暂无评论

A big data inspired chaotic solution for fuzzy feedback linearization model in cyber-physical systems

引用

AD HOC NETWORKS 2015年 35卷 97-104页

作者： Liu, Lei Zhao, Shulin Yu, Zhilou Dai, Hongjun Shandong Univ Dept Comp Sci & Technol Jinan 250101 Peoples R China Inspur Inc Ctr Technol Jinan Peoples R China

Cyber-Physical System (CPS) that integrates computational and physical capabilities has emerged as a promising topic. It interacts with physical world and humans through ad hoc communications. In contemporary, with the fast development of CPS theory and applications, CPS generates a large volume of data, which may lead its control into a chaotic status. Hence, there is a pressing demand to solve the chaotic status CPS control. To this end, the chaotic time series prediction algorithm is employed to resolve the chaotic status featured by a fuzzy feedback linearization model. Modeling the CPS under big data without taking chaotic features into account may lead to unexpected results. This is because chaotic CPS is dramatically sensitive to small disturbances or minor changes of initiators. This paper developed a CPS model in light of fuzzy feedback linearization. Further, the chaotic time prediction algorithm is applied to solve the chaotic control problem in CPS. The developed algorithm takes both tracking control problem and synchronization control problem into account. The numerical results suggest that the developed method is feasible and efficient in tracking control and synchronization of two chaotic CPS. (C) 2015 Elsevier B.V. All rights reserved.

关键词： big data processing Cyber-physical system Fuzzy feedback linearization Chaotic time series prediction

来源：评论

学校读者我要写书评

暂无评论

Scalable Tensor-Train-Based Tensor Computations for Cyber-Physical-Social big data

引用

IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS 2020年第4期7卷 873-885页

作者： Liu, Huazhong Yang, Laurence T. Ding, Jihong Guo, Yimu Xie, Xia Wang, Zhi-Jie Huazhong Univ Sci & Technol Sch Comp Sci & Technol Wuhan 430074 Peoples R China Jiujiang Univ Sch Informat Sci & Technol Jiujiang 332005 Peoples R China St Francis Xavier Univ Dept Comp Sci Antigonish NS B2G 2W5 Canada Zhejiang Univ Technol Sch Educ Sci & Technol Hangzhou 310014 Peoples R China Tencent Inc Dept Social Network Applicat Shenzhen 518052 Peoples R China Chongqing Univ Coll Comp Sci Chongqing 400044 Peoples R China

Tensor-based big data analysis approaches are effectively exploited to handle multisource and heterogeneous cyber-physical-social big data generated from diverse spaces. However, the curse of dimensionality seriously restricts their widespread exploitation, especially under edge/fog computing environments. To alleviate the dilemma, we attempt to present a set of tensor-train (TT)-based tensor operations with their scalable computations and then propose a novel TT-based big data processing framework under edge/fog computing environments. Specifically, in this article, we first summarize and present a set of TT-based tensor operations by converting the original high-order tensor operation to a series of low-order (second- or third-order) TTcore-based operations. Then, we propose a two-layer scalable TT-based computation architecture, including inter-TTcore and intra-TTcore scalable models. Afterward, according to various scalable models, a series of scalable TT-based tensor computations (STT-TCs) with their complexity analysis are proposed in detail. Finally, we propose a novel TT-based big data processing framework to adapt to edge/fog computing environments. We conduct extensive experiments based on both random data sets and real-world ubiquitous bus traffic data sets. Experimental results demonstrate that the proposed STT-TCs can significantly improve computation efficiency and are suitable for edge/fog computing environments.

关键词： Tensile stress big data Computational modeling Adaptation models Computer science Computer architecture data analysis big data processing cyber-physical-social system (CPSS) distributed/parallel computing edge/fog computing scalable tensor computations tensor-train (TT)-based tensor operations

来源：评论

学校读者我要写书评

暂无评论

In-memory big data analytics under space constraints using dynamic programming

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2018年 83卷 219-227页

作者： Gai, Keke Qiu, Meikang Liu, Meiqin Xiong, Zenggang Beijing Inst Technol Sch Comp Sci & Technol Beijing 100081 Peoples R China Shenzhen Univ Coll Comp Sci & Software Engn Shenzhen 518060 Peoples R China Hubei Engn Univ Sch Comp Sci & Informat Technol Xiaogan 43200 Hubei Peoples R China Zhejiang Univ Coll Elect Engn Hangzhou 310027 ZJ Peoples R China

The emergence of persistent memories has powered the data processing with the in-memory environment and in-memory data analytics have become an advance of high-performance data processing. Recent explorations of using in-memory technologies address the improvement of the memory performance from re-designing file systems. Most current approaches mitigate data exchanges between buffers and disks by migrating workload to memories. However, this type of solutions will be encountering the restriction of the memory size with the rapid growth of the application volume. This paper focuses on the issue caused by the large amount of data processing within in-memory systems and proposes a novel approach that is designed to dynamically determine whether the data processing should be accomplished in the memory. The proposed approach is called Smart In-Memory data Analytics Manager (SIM-DAM) model, which utilizes a dynamic working manner of the file system, as well as fully uses hardware mappings. The experimental results obtained from our laboratory evaluations represent that the throughputs of SIM-DAM can achieve a high-level performance with different input data sizes without the constraints of the memories' spaces. (C) 2018 Elsevier B.V. All rights reserved.

关键词： In-memory data analytics Dynamic programming Heterogeneous computing big data processing On-chip memory architecture

来源：评论

学校读者我要写书评

暂无评论

On the analysis of big data indexing execution strategies

引用

JOURNAL OF INTELLIGENT & FUZZY SYSTEMS 2017年第5期32卷 3259-3271页

作者： Siddiqa, Aisha Karim, Ahmad Saba, Tanzila Chang, Victor Univ Malaya Dept Comp Syst & Technol Kuala Lumpur 50603 Malaysia Bahauddin Zakariya Univ Dept Informat Technol Multan Pakistan Prince Sultan Univ Coll Comp & Informat Sci Riyadh Saudi Arabia Xian Jiaotong Liverpool Univ IBSS Suzhou Peoples R China

Efficient response to search queries is very crucial for data analysts to obtain timely results from big data spanned over heterogeneous machines. Currently, a number of big-data processing frameworks are available in which search operations are performed in distributed and parallel manner. However, implementation of indexing mechanism results in noticeable reduction of overall query processing time. There is an urge to assess the feasibility and impact of indexing towards query execution performance. This paper investigates the performance of state-of-the-art clustered indexing approaches over Hadoop framework which is de facto standard for big data processing. Moreover, this study leverages a comparative analysis of nonclustered indexing overhead in terms of time and space taken by indexing process for varying volume data sets with increasing Index Hit Ratio. Furthermore, the experiments evaluate performance of search operations in terms of data access and retrieval time for queries that use indexes. We then validated the obtained results using Petri net mathematical modeling. We used multiple data sets in our experiments to manifest the impact of growing volume of data on indexing and data search and retrieval performance. The results and highlighted challenges favorably lead researchers towards improved implication of indexing mechanism in perspective of data retrieval from big data. Additionally, this study advocates selection of a non-clustered indexing solution so that optimized search performance over big data is obtained.

关键词： big data indexing big data processing data retrieval

来源：评论

学校读者我要写书评

暂无评论

big data-Based Smart Health Monitoring System: Using Deep Ensemble Learning

引用

IEEE ACCESS 2023年 11卷 114880-114903页

作者： Abidi, Mustufa Haider Umer, Usama Mian, Syed Hammad Al-Ahmari, Abdulrahman King Saud Univ Adv Mfg Inst Raytheon Chair Syst Engn Riyadh 11421 Saudi Arabia

Human life has become smarter by utilizing big data, telecommunication technologies, and wearable sensors over pervasive computing to give better healthcare services. big data is built with the possibility to improve the healthcare industry. big data makes the interconnection between patients, wearable sensors, healthcare caregivers, and providers through the utilization of Information and Communication Technology (ICT) and software. Most of the economic challenges in developing countries are caused by the healthcare sector, which occurs predominantly due to the increasing population requiring more quality of care concerning older people. Older people need great attention and care as they lead with irreparable damages when a minor accident or insignificant disease occurs. Therefore, the necessity of implementing new technologies and tools has arisen to support senior citizens regarding their healthcare. Various advancements in wireless technology, miniaturization, computing power, and processing made diverse healthcare innovations that led to developing the connected medical devices. Hence, this proposal develops a new healthcare monitoring system for tracking the activities of elderly people, where the Hadoop MapReduce technique for parallel processing the large-sized data. The data collected as mentioned in the available datasets is performed by using the numerous wearable sensors fixed on the "subject's left ankle, right arm, and chest" that are transformed to the cloud platform and also to the data analytics layer according to the Internet of Medical Things (IoMT) devices. The given input undergoes data splitting to produce tiny chunks. These small chunks of the input files are then considered as Map tasks. Here, in the map phase, the features are optimally selected by the Hybrid Dingo Coyote Optimization (HDCO). The combiner phase classifies the physical activities using the developed Deep Ensemble Learning (DEL) consisting of classifiers such as "Extreme Learn

关键词： Smart health monitoring system eHealth health care 40 big data processing deep ensemble learning extreme learning machine convolutional neural network deep belief network

来源：评论

学校读者我要写书评

暂无评论

A New data Layout Scheme for Energy-Efficient MapReduce processing Tasks

引用

JOURNAL OF GRID COMPUTING 2018年第2期16卷 285-298页

作者： Tran, Xuan T. Tien Van Do Rotter, Csaba Hwang, Dosam Ton Duc Thang Univ Fac Informat Technol Div Knowledge & Syst Engn ICT Ho Chi Minh City Vietnam Budapest Univ Technol & Econ Dept Networked Syst & Serv Magyar Tudosok Korutja 2 H-117 Budapest Hungary Nokia Bell Labs Bokay Janos U 36-42 H-1082 Budapest Hungary Yeungnam Univ Dept Comp Engn Gyongsan South Korea

Yet Another Resource Negotiator (YARN) is a framework to manage and allocate resource requests from applications that process big data stored in HDFS. However, dynamic power management methods are not efficient when YARN manage applications to process big data stored in the default data layout of HDFS. In this paper, we propose a new data layout scheme that can be implemented for HDFS. A comparison between our proposal and the existing HDFS data layout scheme shows that the new data layout algorithm significantly reduces the energy consumption at the slight expense of the mean response time of jobs.

关键词： Hadoop Layout big data processing data locality Dynamic power management

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：