检索结果-内蒙古大学图书馆

Distributed fuzzy c-means algorithms for big sensor data based on cloud computing

INTERNATIONAL JOURNAL OF sensor NETWORKS 2015年第1-2期18卷 32-39页

作者： Zhang, Qingchen Chen, Zhikui Leng, Yonglin Dalian Univ Technol Sch Software Technol Dalian 116620 Peoples R China Bohai Univ Coll Informat Sci & Technol Jinzhou 121000 Peoples R China

Fuzzy c-means (FCM) clustering has emerged as one important technique for pattern recognition, image processing and data analysis. Owing to the huge amount of data and computational complexity, FCM is difficult to cluster big sensor data in real time. The paper proposes three distributed FCM algorithms, namely distributed FCM algorithm based on MapReduce (DFCM), distributed online FCM algorithm based on sampled affinity propagation (AP) clustering (DOFCM) and distributed kernel FCM algorithm based on MapReduce (DKFCM). The proposed algorithms use cloud computing technology to improve the efficiency for clustering big sensor data. Experiments prove the superiority of the proposed methods through the clustering accuracy and clustering speed.

关键词： sensor networks big sensor data fuzzy c-means clustering cloud computing affinity propagation

来源：评论

学校读者我要写书评

暂无评论

Support high-order tensor data description for outlier detection in high-dimensional big sensor data

引用

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE 2018年 81卷 177-187页

作者： Deng, Xiaowu Jiang, Peng Peng, Xiaoning Mi, Chunqiao Hangzhou Dianzi Univ Coll Automat Hangzhou 310018 Zhejiang Peoples R China Huaihua Univ Sch Comp Sci & Engn Huaihua 418000 Peoples R China Hunan Prov Key Lab Ecol Agr Intelligent Control T Huaihua 418000 Peoples R China

The various high-dimensional sensor data can be collected by wireless sensor networks, video monitoring systems and multimedia sensor networks, while High-dimensional sensor data is inherently large-scale because each sensor node has spatial attributes and may also be associated with large amounts of measurement data evolving over time. Detecting outlier in high-dimensional big sensor data is a challenging task. Most of existing outlier detection methods is based on vector representation. However, high-dimensional sensor data is naturally described by tensor representations. The vector-based methods can lead to destroy original structural information and correlation for high-dimensional sensors data, result in the problem of curse of dimensionality, and some outliers cannot be detected. To solve this problem, support high-order tensor data description (STDD) and kernel support high-order tensor data description (KSTDD) are proposed to detect outliers for tensor data. STDD and KSTDD extend support vector data description from vector space to tensor space. KSTDD maintains the structural information of data, avoids the problem caused by the vectorization of tensor data, and improves the performance of outlier detection. Experiments on four sensor datasets show that the proposed method is superior to the traditional vectorized data analysis method. (C) 2017 Elsevier B.V. All rights reserved.

关键词： big sensor data High-dimensional data Outlier detection CP factorization KSTDD

来源：评论

学校读者我要写书评

暂无评论

An Intelligent Outlier Detection Method With One Class Support Tucker Machine and Genetic Algorithm Toward big sensor data in Internet of Things

引用

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS 2019年第6期66卷 4672-4683页

作者： Deng, Xiaowu Jiang, Peng Peng, Xiaoning Mi, Chunqiao Hangzhou Dianzi Univ Coll Automat Hangzhou 310018 Zhejiang Peoples R China Huaihua Univ Sch Comp Sci & Engn Huaihua 418008 Peoples R China Hunan Prov Key Lab Ecol Agr Intelligent Control T Huaihua 418008 Peoples R China

Various types of sensor data can be collected by the Internet of Things (IoT). Each sensor node has spatial attributes and may also be associated with a large number of measurement data that evolve over time;therefore, these high-dimensional sensor data are inherently large scale. Detecting outliers in large-scale IoT sensor data is a challenging task. Most existing anomaly detection methods are based on a vector representation. However, large-scale IoT sensor data have characteristics that make tensor methods more efficient for extracting information. The vector-based methods can destroy original structural information and correlation within large-scale sensor data, resulting in the problem of the "curse of dimensionality," and some outliers hence cannot be detected. In this paper, we propose a one-class support Tucker machine (OCSTuM) and an OCSTuM based on tensor Tucker factorization and a genetic algorithm called GA-OCSTuM. These methods extend one-class support vector machines to tensor space. OCSTuM and GA-OCSTuM are unsupervised anomaly detection approaches for big sensor data. They retain the structural information of data while improving the accuracy and efficiency of anomaly detection. The experimental evaluations on real data sets demonstrate that our proposed method improves the accuracy and efficiency of anomaly detection while retaining the intrinsic structure of big sensor data.

关键词： Anomaly detection big sensor data feature selection genetic algorithm (GA) Internet of Things (IoT) one-class support Tucker machine (OCSTuM) parameter setting

来源：评论

学校读者我要写书评

暂无评论

BSDP: big sensor data Preprocessing in Multi-Source Fusion Positioning System Using Compressive Sensing

引用

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY 2019年第9期68卷 8866-8880页

作者： Zhao, Wanlong Han, Shuai Meng, Weixiao Sun, Dajun Hu, Rose Qingyang Harbin Inst Technol Commun Res Ctr Harbin 150080 Heilongjiang Peoples R China Harbin Engn Univ Coll Underwater Acoust Engn Harbin 150001 Heilongjiang Peoples R China Utah State Univ Dept Elect & Comp Engn Logan UT 84322 USA

A multi-source fusion positioning system (MFPS) adopts a data fusion technique for hybrid positioning. This technique fully utilizes several types of positioning sources in contrast to the conventional positioning methods, which use a single positioning source. sensor data collection and processing are essential components in MFPSs. A framework for a practical center-server-based MFPS is proposed in this paper. A big sensor data preprocessing (BSDP) scheme, which is composed of data extraction (DE-BSDP), data gathering (DG-BSDP), and data transmission (DT-BSDP), is further proposed under this framework. DE-BSDP attempts to remove useless positioning data from fusion sources. The big sensor data are further compressed in DG-BSDP, in which a compressive sensing technique is adopted to realize data compression before data transmission. After the data gathering phase, the compressed data are transmitted to the fusion center for reconstruction in DT-BSDP. The proposed BSDP method can reduce the data collection amount significantly and improves the data transmission efficiency with a slight reduction on positioning accuracy. Experiments and simulations verify the effectiveness of the proposed BSDP scheme.

关键词： big sensor data data preprocessing multi-source fusion positioning compressive sensing

来源：评论

学校读者我要写书评

暂无评论

A Weighted Fuzzy c-Means Clustering Algorithm for Incomplete big sensor data 11th

A Weighted Fuzzy c-Means Clustering Algorithm for Incomplete...

引用

11th China Wireless sensor Network Conference (CWSN)

作者： Li, Peng Chen, Zhikui Hu, Yueming Leng, Yonglin Li, Qiucen Dalian Univ Technol Sch Software Technol Dalian 116620 Peoples R China South China Agr Univ Coll Nat Resources & Environm Guangzhou 510642 Guangdong Peoples R China

ISBN: (纸本)9789811081231;9789811081224

sensor data processing plays an important role on the development of the wireless sensor networks in the big data era. Owning to the existence of a large number of incomplete data in wireless sensor networks, fuzzy c-means clustering algorithm (FCM) finds it difficult to produce an appropriate cluster result. The paper proposes a distributed weighted fuzzy c-means algorithm based on incomplete data imputation for big sensor data (DWFCM). DWFCM improves Affinity Propagation (AP) clustering algorithm by designing a new similarity metrics for imputing incomplete sensor data, and then proposes a weighted FCM (wFCM) by assigning a lower weighted value to the incomplete data object for improving the cluster accuracy. Finally, we validate the proposed weighted FCM algorithm on the dataset collected from the smart WSN lab. Experiments demonstrate that the weighted FCM algorithm could fill the missing values very accurately and improve the clustering results effectively.

关键词： Wireless sensor network big sensor data Fuzzy c-means algorithm

来源：评论

学校读者我要写书评

暂无评论

A big sensor data Offloading Scheme in Rail Networks 89

A Big Sensor Data Offloading Scheme in Rail Networks

引用

89th IEEE Vehicular Technology Conference (VTC Spring)

作者： Saki, Mahdi Abolhasan, Mehran Lipman, Justin Univ Technol Sydney Sch Elect & Data Engn Sydney NSW Australia

ISBN: (纸本)9781728112176

In this paper, we propose an offloading scheme to transfer massive stored sensor data from rolling stock to railway data centers. We apply a delayed offloading strategy for non-critical stored data assuming that the critical data has been already separated through an appropriate edge processing task and has been sent via a real-time communication such as cellular networks. We propose train stations as potential and feasible spots for data offloading via available wireless local area networks (WLAN) such as existing WiFi network at stations. Thus, stations will not only be the places of passenger exchange but also data exchange. We develop an analytical model customized for the proposed offloading strategy in rail applications. Then we validate the performance of our model through simulation in various scenarios in Omnet. The simulation results shows an accuracy of %98.67 for the proposed analytical model with reference to the simulation results in Omnetpp. Additionally, by using our proposed scheme, we can theoretically offload up to 5.43 GB per each stopping station.

关键词： big sensor data delayed offloading IEEE 802.11

来源：评论

学校读者我要写书评

暂无评论

Reconstruction of big sensor data 2

Reconstruction of Big Sensor Data

引用

2nd IEEE International Conference on Computer and Communications (ICCC)

作者： Shao, Yongshuai Chen, Zhe Li, Fangfang Fu, Chong Northeastern Univ Sch Comp Sci & Engn Shenyang Liaoning Peoples R China

ISBN: (纸本)9781467390262

With ubiquitous sensors continuously monitoring and collecting large amounts of information, there is no doubt that this is an era of big data. One of the important source for scientific big data is the datasets collected by Internet of things (IoT). For an IoT application to analyze big sensor data, it is necessary that the data are clean and loss less. However, data loss in IoT is common due to unreliable wireless link or hardware failure in the nodes. To reconstruct the big sensor data, this paper first presents an algorithm based on matrix completion method. Then for multi-attributes sensor data, a tensor-based method is provided to estimate missing values. Moreover, an effective solution is proposed using the alternating direction method of multipliers. Finally, the experiments with real-word sensor data show the effectiveness of the proposed methods.

关键词： big sensor data data loss matrix completion tensor

来源：评论

学校读者我要写书评

暂无评论

A Framework for Real-Time Information Derivation from big sensor data 17

A Framework for Real-Time Information Derivation from Big Se...

引用

2015 IEEE 17th International Conference on High Performance Computing and Communications (HPCC)

作者： Chen, Liehuo Kang, Kyoung-Don SUNY Binghamton Dept Comp Sci Binghamton NY 13902 USA

ISBN: (纸本)9781479989379

In data-intensive real-time applications, e.g., transportation management and location-based services, the amount of sensor data is exploding. In these applications, it is desirable to extract value-added information, e.g., fast driving routes, from sensor data streams in real-time rather than overloading users with massive raw data. However, achieving the objective is challenging due to the data volume and complex data analysis tasks with stringent timing constraints. Most existing big data management systems, e.g., Hadoop, are not directly applicable to real-time sensor data analytics, since they are timing agnostic and focus on batch processing of previously stored data that are potentially outdated and subject to I/O overheads. To address the problem, we design a new real-time big data management framework, which supports a non-preemptive periodic task model for continuous in-memory sensor data analysis and a schedulability test based on the EDF (Earliest Deadline First) algorithm to derive information from current sensor data in real-time by extending the map-reduce model originated in functional programming. As a proof-of-concept case study, a prototype system is implemented. In the performance evaluation, it is empirically shown that all deadlines can be met for the tested sensor data analysis benchmarks.

关键词： big sensor data Real-Time Information

来源：评论

学校读者我要写书评

暂无评论

A Framework for Real-Time Information Derivation from big sensor data

A Framework for Real-Time Information Derivation from Big Se...

引用

IEEE International Symposium on Cyberspace Safety and Security

作者： Liehuo Chen Kyoung-Don Kang Dept. of Comput. Sci. State Univ. of New York at Binghamton Binghamton NY USA

ISBN: (纸本)9781479989386

关键词： big data batch processing (computers) data analysis functional programming parallel processing performance evaluation EDF algorithm I/O overheads batch processing complex data analysis tasks continuous in-memory sensor data analysis data volume data-intensive real-time applications earliest deadline first algorithm nonpreemptive periodic task model real-time big data management framework real-time information derivation real-time sensor big data analytics schedulability test stringent timing constraints timing agnostic value-added information extraction Analytical models big data data analysis data models Mobile radio mobility management Real-time systems Timing big sensor data Real-Time Information

来源：评论

学校读者我要写书评

暂无评论

Time-Efficient Advent for Diagnosing Flaws in Hadoop on the big sensor Type data 1

引用

International Conference on Computing, Analytics and Networking (ICCAN)

作者： Jagdishchandra, Mehta Jaldhi Upadhyay, Bhargavi R. Amrita Univ Amrita Vishwa Vidyapeetham Amrita Sch Engn Dept Comp Sci & Engn Bengaluru India

ISBN: (数字)9789811078712

ISBN: (纸本)9789811078712;9789811078705

Hadoop is a MapReduce-based distributed processing framework used in the area of big data analytics in every organizations. big sensor data is difficult to manage with the traditional data management tools. Thus, Hadoop challenges to manage it in high scalable amount in a time-efficient manner. In this paper, for fast detection of flaws in big sensor data sets, a different type of approach in diagnosing flaws with the time efficiency is used. Due to the wireless transfer of data across the nodes in a wireless sensor networks, there can be loss of data which will result in wrong interpretation of data at the nodes. The proposed approach of this paper is to form a group of sensors as a cluster. If any sensor detects violations, then the energy of that sensor has to be compared with the other sensors. The sensor having the highest energy will become the cluster head, and it will send the sensed data to the data center. The data center then diagnoses the flaw with respect to the sensed data in the big sensor data.

关键词： HDFS Hadoop MapReduce big sensor data

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：