检索结果-内蒙古大学图书馆

An edge streaming data processing framework for autonomous driving

CONNECTION SCIENCE 2021年第2期33卷 173-200页

作者： Zhao, Hang Yao, LinBin Zeng, ZhiXin Li, DongHua Xie, JinLiang Zhu, WeiLing Tang, Jie South China Univ Technol Sch Comp Sci & Engn Guangzhou Peoples R China

In recent years, with the rapid development of sensing technology and the Internet of Things (IoT), sensors play increasingly important roles in traffic control, medical monitoring, industrial production and etc. They generated high volume of data in a streaming way that often need to be processed in real time. Therefore, streaming data computing technology plays an indispensable role in the real-time processing of sensor data in high throughput but low latency. However, there are two problems in deploying streaming data process ability in cloud computing data centre. Firstly, massive sensor nodes simultaneously upload data to the remote cloud computing data centre, which requires a large number of bandwidth resources supports. The existing network infrastructure cannot provide enough bandwidth at a reasonable price. Secondly, due to the geographical distribution characteristics of the cloud computing data centre, there will inevitably be large transmission delay during the process of data transmission. Such end-to-end delay is intolerable to mobile applications especially for those latency sensitive tasks. In view of the above problems, this paper proposes an autonomous driving oriented edge streaming data processing framework, which migrates the computing and storage capability from the remote cloud data centre to the edge data centre. It focuses on the change of vehicle flow in a specific geographical area, and uses the computing power sunk to edge node to process the massive streaming data generated by autonomous vehicles nearby. The proposed framework is implemented on top of Spark streaming, which builds up a gray model based traffic flow monitor, a traffic prediction orientated prediction layer and a fuzzy control based Batch Interval dynamic adjustment layer for Spark streaming. It could forecast the variation of sensors data arrive rate, make streaming Batch Interval adjustment in advance and implement real-time streaming process by edge. Therefore, it can

关键词： Edge computing streaming data processing spark streaming autonomous driving

来源：评论

学校读者我要写书评

暂无评论

Developing a streaming data processing workflow for querying space-time activities from geotagged tweets

引用

COMPUTERS ENVIRONMENT AND URBAN SYSTEMS 2016年 59卷 256-268页

作者： Wachowicz, Monica Arteaga, M. Dolores Cha, Sangwhan Bourgeois, Yves Univ New Brunswick People Mot Lab Fredericton NB E3B 5A3 Canada Univ New Brunswick Urban & Community Studies Inst Fredericton NB E3B 5A3 Canada

The critical dimensions in describing space-time activities are "what", "where", "when", and "who", which are frequently applied to collect data about basic functions people per... 详细信息

The critical dimensions in describing space-time activities are "what", "where", "when", and "who", which are frequently applied to collect data about basic functions people perform in space in the course of a day. Collecting data about these dimensions using activity-based surveys has presented researchers with a number of technical and social limitations, ranging from the restricted period of time participants have to record their activities to the level of accuracy with which participants complete a survey. This paper, proposes a new streaming data processing workflow for querying space-time activities (STA) as a by-product of microblogging communication. It allows exploring a large volume of geotagged tweets to discover STA patterns of daily life in a systematic manner. A sequence of tasks have been implemented using different cloud-based computing resources for handling over one million of daily geotagged tweets from Canada for a period of six months. The STA patterns have revealed activity choices that might be attributable to personal motivations for communicating an activity in social networks. (C) 2015 Elsevier Ltd. All rights reserved.

关键词： Space-time activity Twitter streaming data processing Cloud computing

来源：评论

学校读者我要写书评

暂无评论

Feature-based Dividing Symbolic Time Series Representation for streaming data processing 9

Feature-based Dividing Symbolic Time Series Representation f...

引用

9th International Conference on Information Technology in Medicine and Education (ITME)

作者： Zhan, Peng Hu, Yupeng Zhang, Qi Zheng, Jiecai Li, Xueqing Shandong Univ Sch Software Jinan Shandong Peoples R China Shandong Sprot Univ Sch Sport Commun & Informat Technol Jinan Shandong Peoples R China

ISBN: (纸本)9781538677438

Along with the continuous development of information technology, massive numbers of detecting instruments or systems in various fields are continuously producing plenty number of streaming time series data. In recent years, many representation approaches for time series has been proposed with the main objective of dimensionality reduction to support various data mining algorithms in the domain of time series data processing. Symbolic Aggregate approXimation(SAX) is a major symbolic representation and dimensionality reduction algorithm which has been widely used in many application scenarios of time series data mining, such as motifs discovery, outlier detection, etc. In this paper, we propose a symbolic representation method of streaming time series based on VTPdiving with sliding window and a similarity measurement algorithm for the proposed representation method which lower bounding the Euclidean distance on the original data. Extensive experiments on different kinds of typical time series datasets have been conducted to demonstrate the advantages of our proposed method.

关键词： Symbolic Aggregate Approximation Time Series Dimensionality Reduction streaming data processing

来源：评论

学校读者我要写书评

暂无评论

Detecting Anomalous Users via streaming data processing in Smart Grid

Detecting Anomalous Users via Streaming Data Processing in S...

引用

作者： Yu Kang Xinting Wang Xiu Cao Yangfan Zhou Zhichao Lai Yuhao Li Xuqi Zhang Wei Geng School of Computer Science Fudan University China Shanghai Jiao Tong University School of Computer Science Fudan University Shanghai Inspection &Maintenance of Power Company South Kent School

Anomalous user detection is an important concern in smart grid. Conduct the anomaly detection in real time with streaming data processing technology is a hot research field for smart grid maintenance. We first study the characteristics of power measurement data. Based on our findings, we adopt Clustream, an outlier detection model, for finding anomalous users. Combined with grid based DBSCAN algorithm, we achieve a two-phase flow processing architecture including offline and online phases for the goal of detecting anomalous user. The implementation details of our system are introduced. The experimental results show that our system can achieve better detection results.

关键词： Smart grid operation anomaly detection streaming data processing energy theft

来源：评论

学校读者我要写书评

暂无评论

Applying spark based machine learning model on streaming big data for health status prediction

引用

COMPUTERS & ELECTRICAL ENGINEERING 2018年 65卷 393-399页

作者： Nair, Lekha R. Shetty, Sujala D. Shetty, Siddhanth D. BITS Pilani Dept Comp Sci Dubai CampusPOB 345055 Dubai U Arab Emirates

Machine learning is one of the driving forces of science and commerce, but the proliferation of Big data demands paradigm shifts from traditional methods in the application of machine learning techniques on this voluminous data having varying velocity. With the availability of large health care datasets and progressions in machine learning techniques, computers are now well equipped in diagnosing many health issues. This work aims at developing a real time remote health status prediction system built around open source Big data processing engine, the Apache Spark, deployed in the cloud which focus on applying machine learning model on streaming Big data. In this scalable system, the user tweets his health attributes and the application receives the same in real time, extracts the attributes and applies machine learning model to predict user's health status which is then directly messaged to him/her instantly for taking appropriate action. (C) 2017 Elsevier Ltd. All rights reserved.

关键词： Big data machine learning streaming data processing Tweet processing Apache spark Health informatics

来源：评论

学校读者我要写书评

暂无评论

Big data streaming Analytics for QoE Monitoring in Mobile Networks: A Practical Approach

Big Data Streaming Analytics for QoE Monitoring in Mobile Ne...

引用

IEEE International Conference on Big data (Big data)

作者： Rueda, Diego F. Vergara, Dahyr Reniz, David Everis NTT Data Co Telecom & Business Analyt Bogota Colombia

ISBN: (纸本)9781538650356

Traditionally, Mobile Network Operators (MNOs) use a set of Key Performance Indicators (KPIs) to measure the quality offered to their customers. However, these KPIs do not reflect the quality perceived by the customers because they are high-level and network-based metrics. Instead, Quality of Experience (QoE) monitoring of the most common mobile applications can help MNOs to determine when and where customer experience is degraded. In this paper, a customized tool based on Big data streaming is proposed to solve the needs of customer experience monitoring in a real-life MNO and to overcome the challenges of processing a large amount of data collected in 3G and 4G mobile networks. Moreover, real-life case studies of value creation through Big data Analytics for telecommunication industry are also defined. Results show that the streaming data processing enables new opportunities for the MNO to take actions focused on customer experience improvement in near real-time.

关键词： Big data Analytics customer experience management mobile networks quality of experience streaming data processing

来源：评论

学校读者我要写书评

暂无评论

streaming data Priority Scheduling Framework for Autonomous Driving by Edge 45

Streaming Data Priority Scheduling Framework for Autonomous ...

引用

45th Annual International IEEE-Computer-Society Computers, Software, and Applications Conference (COMPSAC)

作者： Yao, Lingbing Zhao, Hang Tang, Jie Liu, Shaoshan Gaudiot, Jean-Luc South China Univ Technol Sch Comp Sci & Engn Guangzhou Peoples R China PerceptIn Fremont CA USA Univ Calif Irvine Dept Elect Engn & Comp Sci Irvine CA USA

ISBN: (纸本)9781665424639

In recent years, intelligent vehicles like autonomous vehicles generate a huge amount of sensing data continuously. The computations on those data streams are far beyond the processing capacity of on-board computing. To deal with the streaming data process in real-time, the deployment of streaming data processing system by edge turns to the first choice in terms of performance. However, the existing frameworks cannot satisfy the complicated demands from autonomous driving tasks and lack the ability in supporting the task priority scheduling. In this paper, we propose a streaming data priority scheduling framework for autonomous driving by edge on Spark streaming and make an implementation on Spark 2.3.0. The proposed framework can identify the priorities among different data processing tasks and implement the task scheduling based on non-preemptive priority queuing theory. To meet differentiated service level requirements, the proposed non-preemptive priority queuing scheduling mechanism considers the priority category of tasks, the distance between vehicles and edge nodes, and the priority weight of vehicles. Experiments show that this mechanism can effectively identify the priority information of different tasks from different vehicles and reduce the end-to-end latency of high-priority tasks by up to 46% than low-priority tasks.

关键词： Autonomous driving streaming data processing Priority scheduling Spark streaming

来源：评论

学校读者我要写书评

暂无评论

Continuous Histogram Computing on streaming data Approach

Continuous Histogram Computing on Streaming Data Approach

引用

2024 Conference of Young Researchers in Electrical and Electronic Engineering, ElCon 2024

作者： Azarevich, Artem D. Efremov, Mikhail A. St. Petersburg Electrotechnical University 'LETI' St. Petersburg Russia St. Petersburg Electrotechnical University 'LETI' Mathematical Support and Computer Applications Department St. Petersburg Russia

ISBN: (纸本)9798350360646

In federative distributed learning, when combining trained models into one, information about the distribution of training samples is used. However, in some cases (for example, when training with reinforcement), it is impossible to obtain information about the distribution using standard methods due to the large volume of data or their generation during training. This paper describes the method of computing a histogram for the case of a random value with a previously unknown min and max values in one pass through the streaming data. © 2024 IEEE.

关键词： cumulative distribution function histogram streaming data processing

来源：评论

学校读者我要写书评

暂无评论

Multistage identification method for real-time abnormal events of streaming data

引用

INTERNATIONAL JOURNAL OF DISTRIBUTED SENSOR NETWORKS 2019年第12期15卷

作者： Luo, Hao Sun, Kexin Wang, Junlu Liu, Chengfeng Ding, Linlin Song, Baoyan Liaoning Univ Sch Informat Shenyang 110036 Liaoning Peoples R China

With the development of streaming data processing technology, real-time event monitoring and querying has become a hot issue in this field. In this article, an investigation based on coal mine disaster events is carried out, and a new anti-aliasing model for abnormal events is proposed, as well as a multistage identification method. Coal mine micro-seismic signal is of great importance in the investigation of vibration characteristic, attenuation law, and disaster assessment of coal mine disasters. However, as affected by factors like geological structure and energy losses, the micro-seismic signals of the same kind of disasters may produce data drift in the time domain transmission, such as weak or enhanced signals, which affects the accuracy of the identification of abnormal events ("the coal mine disaster events"). The current mine disaster event monitoring method is a lagged identification, which is based on monitoring a series of sensors with a 10-s-long data waveform as the monitoring unit. The identification method proposed in this article first takes advantages of the dynamic time warping algorithm, which is widely applied in the field of audio recognition, to build an anti-aliasing model and identifies whether the perceived data are disaster signal based on the similarity fitting between them and the template waveform of historical disaster data, and second, since the real-time monitoring data are continuous streaming data, it is necessary to identify the start point of the disaster waveform before the identification of the disaster signal. Therefore, this article proposes a strategy based on a variable sliding window to align two waveforms, locating the start point of perceptual disaster wave and template wave by gradually sliding the perceptual window, which can guarantee the accuracy of the matching. Finally, this article proposes a multistage identification mechanism based on the sliding window matching strategy and the characteristics of the waveforms

关键词： streaming data processing micro-seismic signals real-time event monitoring and query data drift dynamic time warping anti-aliasing model multistage identification

来源：评论

学校读者我要写书评

暂无评论

Optique: Zooming in on Big data

引用

COMPUTER 2015年第3期48卷 60-67页

作者： Giese, Martin Soylu, Ahmet Vega-Gorgojo, Guillermo Waaler, Arild Haase, Peter Jimenez-Ruiz, Ernesto Lanti, Davide Rezk, Martin Xiao, Guohui Oezcep, Oezguer Rosati, Riccardo Univ Oslo Dept Informat N-0316 Oslo Norway Univ Oxford Dept Comp Sci Oxford OX1 2JD England Free Univ Bozen Bolzano Fac Comp Sci Bolzano Italy Univ Lubeck Inst Informat Syst Lubeck Germany Univ Roma La Sapienza Dept Comp Control & Management Engn Rome Italy

Optique overcomes problems in current ontology-based data access systems pertaining to installation overhead, usability, scalability, and scope by integrating a useroriented query interface, semi-automated managing me... 详细信息

关键词： Big data ontologies (artificial intelligence) query processing user interfaces Optique big data installation overhead ontology-based data access systems query rewriting techniques semiautomated managing methods streaming data processing temporal data processing user-oriented query interface Big data database languages database management Ontologies Visualization Optique big data distributed databases graphical user interfaces ontology

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：