Simultaneous evaluating a batch of iterative graph queries on a distributed system enables amortization of high communication and computation costs across multiple queries. As demonstrated by our prior work on MultiLy...
详细信息
ISBN:
(数字)9781728162515
ISBN:
(纸本)9781728162522
Simultaneous evaluating a batch of iterative graph queries on a distributed system enables amortization of high communication and computation costs across multiple queries. As demonstrated by our prior work on MultiLyra [Bigdata'19], batched graph query processing can deliver significant speedups and scale up to batch sizes of hundreds of *** this paper, we greatly expand the applicable scenarios for batching by developing BEAD, a system that supports Batching in the presence of Evolving Analytics Demands. First, BEAD allows the graph data set to evolve (grow) over time, more vertices (e.g., users) and edges (e.g., interactions) are added. In addition, as the graph data set evolves, BEAD also allows the user to add more queries of interests to the query batch to accommodate new user demands. The key to the superior efficiency offered by BEAD lies in a series of incremental evaluation techniques that leverage the results of prior request to "fast-foward" the evaluation of the current *** performed experiments comparing batching in BEAD with batching in MultiLyra for multiple input graphs and algorithms. Experiments demonstrate that BEAD's batched evaluation of 256 queries, following graph changes that add up to 100K edges to a billion edge Twitter graph and also query changes of up to 32 new queries, outperforms MultiLyra's batched evaluation by factors of up to 26.16 × and 5.66 × respectively.
PVplr is an R package designed with the intent to offer a variety of options and side-by-side comparisons when modeling outdoor PV time-series data for performance loss rate (PLR) analysis. Built as a part of the IEA-...
详细信息
ISBN:
(数字)9781728161150
ISBN:
(纸本)9781728161167
PVplr is an R package designed with the intent to offer a variety of options and side-by-side comparisons when modeling outdoor PV time-series data for performance loss rate (PLR) analysis. Built as a part of the IEA-PVPS task 13 study on the determination and uncertainty of PLR calculation, the package is designed to not to run a single method but to compare multiple commonly used methods on the same systems to determine system stability and identify biases in the analysis. The workflow is designed as a pipeline with steps of data cleaning, weather correction, time-series processing, and PLR determination, with multiple options at each step replicating commonly used methods or specific process created by other groups. Non-linear PLR evaluation capabilities have been added using piecewise linear modeling. Implementation of this pipeline in prior work has shown when 40 unique assumed linear PLR values are calculated for individual systems results can show significant variance depending on the steps chosen in dataprocessing and modeling, highlighting the biases that can be induced by differences in analysis processes. Future updates to the package are planned that add more options to each step of the pipeline and allow python and R integration so python packages such as RdTools and PVlib-python can be used in conjunction with PVplr.
Vehicle detection and classification plays a vital role within the space of the traffic management system. There's an outsized area for the development during this system as associated with accuracy and exactness....
详细信息
ISBN:
(数字)9781728141084
ISBN:
(纸本)9781728141091
Vehicle detection and classification plays a vital role within the space of the traffic management system. There's an outsized area for the development during this system as associated with accuracy and exactness. Because of increasing traffic within the advanced occasions, it's basic to arrange a framework winning to keep up a record of vehicles going through a path or a street. Spontaneous identification of vehicle data has been broadly used in the vehicle identification and classification system. Applications of the system developed are often useful in the traffic signal controller, vehicle lane departure warning system. The techniques goal is to provide appropriate data about traveling vehicles with the exact count. The convolutional neural network technique models based on YOLOv3 is used. The input is given in the form of video and pre-processing is finished and also the output is gained i.e. the count of vehicles, classification of vehicles supported its sort and total variety of vehicle motion at a specific time.
Today, drought has become part of the identity as well as the fate of many countries. In fact, drought is considered among the most damaging natural disasters. The severe consequences resulting from drought affect the...
详细信息
ISBN:
(数字)9781728175133
ISBN:
(纸本)9781728175140
Today, drought has become part of the identity as well as the fate of many countries. In fact, drought is considered among the most damaging natural disasters. The severe consequences resulting from drought affect the nature and society at different levels. Proper and efficient management is not possible without accurate prediction of drought and the identification of its various aspects. Thus, the existence of a considerable body of literature on drought monitoring. However, significant growth of remote sensing databases as will an increased amount of available data related to drought have been detected. Therefore, a more adequate approach should be developed. During the past decades, data Mining (DM) methods have been introduced for drought monitoring. According to the best of our knowledge, a review of drought monitoring using remote sensing data and DM methods is lacking. Thereby, the purpose of this paper is to review and discuss the applications of DM methods. This paper consolidates the finding of drought monitoring, models, tasks, and methodologies.
Aiming at the low efficiency of the Big data query of the internet of Things, the research based on the smart urban management platform, and on the premise of retaining the original platform data integration framework...
详细信息
ISBN:
(数字)9781728177380
ISBN:
(纸本)9781728177397
Aiming at the low efficiency of the Big data query of the internet of Things, the research based on the smart urban management platform, and on the premise of retaining the original platform data integration framework. The query model and process are redesigned and a new data improved query model is proposed by introducing hash coding, distributed index and data dictionary. Compared with the one by one comparison query method, the speed of the improved query model is increased by 10 6 times when processing millions of data, which significantly improves the query efficiency of big data. The research results can help improve the dataprocessing and analysis ability of Chongqing Liangjiang Smart Urban Management Platform in China.
Social media platforms such as Twitter are increasingly used to collect data of all kinds. During natural disasters, users may post text and image data on social media platforms to report information about infrastruct...
详细信息
ISBN:
(数字)9781728169972
ISBN:
(纸本)9781728169989
Social media platforms such as Twitter are increasingly used to collect data of all kinds. During natural disasters, users may post text and image data on social media platforms to report information about infrastructure damage, injured people, cautions and warnings. Effective processing and analysing tweets in real time can help city organisations gain situational awareness of the affected citizens and take timely operations. With the advances in deep learning techniques, recent studies have significantly improved the performance in classifying crisis-related tweets. However, deep learning models are vulnerable to adversarial examples, which may be imperceptible to the human, but can lead to model's misclassification. To process multi-modal data as well as improve the robustness of deep learning models, we propose a multi-modal adversarial training method for crisis-related tweets classification in this paper. The evaluation results clearly demonstrate the advantages of the proposed model in improving the robustness of tweet classification.
The autonomous underwater vehicle (AUV) aided mobile data collection is an effective method for reducing the energy consumption of the underwater acoustic (UWA) sensor networks. In this paper, we propose an AUV-aided ...
详细信息
ISBN:
(数字)9781728172026
ISBN:
(纸本)9781728172033
The autonomous underwater vehicle (AUV) aided mobile data collection is an effective method for reducing the energy consumption of the underwater acoustic (UWA) sensor networks. In this paper, we propose an AUV-aided path-planning scheme using cooperative transmission mechanism for a medium-scale UWA sensor network. In the proposed scheme, we analyze not only the energy consumption, but also the task duration and path-planning cost comprehensively for practical applications. We analyze four different path-planning schemes in terms of energy consumption of UWA sensor nodes and travel cost of AUV. The simulation results show that the lawn mower path-planning scheme has lower energy consumption of UWA sensor networks. But the circle path-planning scheme has lower working time and path energy consumption of AUV. Therefore, in view of different needs, we should make a comprehensive selection.
The hardware sample of multi-core data-flow recurrent architecture has been developed and tested on the digital signal processing domain. An analysis of the iterative algorithms execution results made it possible to p...
详细信息
ISBN:
(数字)9781728157610
ISBN:
(纸本)9781728157627
The hardware sample of multi-core data-flow recurrent architecture has been developed and tested on the digital signal processing domain. An analysis of the iterative algorithms execution results made it possible to propose a number of mechanisms to improve one of the components of the architecture - the Iterator. A significant problem in architecture programming is a high program redundancy produced by a significant number of special operands that are designed to control its internal resources. The Iterator component is designed to solve this issue, but its capabilities were not enough. The article presents the development results of the Iterator component. A description of the developed mechanisms to control the Iterator is provided. We demonstrate the results of the Iterator improvements using an example of the Viterbi algorithm for searching at hidden Markov models. The developed tools made it possible to nearly halve the volume of special operands and optimize the software implementation of the algorithm.
Electricity load forecasting is a prevalent research topic in recent years. In this study, we predict the electricity consumption using only previous power data (i.e., without using weather information or other featur...
详细信息
ISBN:
(数字)9781728169262
ISBN:
(纸本)9781728169279
Electricity load forecasting is a prevalent research topic in recent years. In this study, we predict the electricity consumption using only previous power data (i.e., without using weather information or other features). We survey existing univariate methods such as MLP-based, CNN-based, XGBoost-based, RF-based, and EN3-bestK. However, these existing methods do not perform well due to that the range of power values varies a lot. Therefore, we present an electricity consumption forecast system called Dynamic Weight Ensemble Model (DWEM). There are three stages in the proposed DWEM. First of all, we provide three types of data serialization in data preprocessing. Second, we train four types of models (i.e., MLP-based, CNN-based, XGBoost-based, and RF-based) for building the ensemble model later. Finally, we combine the four types of models into an ensemble model, using the proposed Two-Phase Ensemble. In the two-phase ensemble, the first phase is to ensemble the models trained using the same algorithm but different serializations, and the second phase is to ensemble the models from different algorithms. The two-phase ensemble method is designed to dynamically adjust weights based on the previous performance of the corresponding models. Moreover, we notice that properly handling missing values is an important factor in system performance. Therefore, we present a statistical method to estimate the missing values. We compare DWEM with various state-of-the-art methods. Comparison of DWEM and the state-of-the-art ensemble method, the results show that DWEM is on average about 46.95% and 44.47% better than EN3-bestK on the MAPE and MAE indicators, respectively.
The entire world is facing the Covid19 pandemic. This pandemic has various consequences on the political, cultural, economical and social life of the community. Lockdown has affected the psychological impact on societ...
详细信息
ISBN:
(数字)9781728159706
ISBN:
(纸本)9781728159713
The entire world is facing the Covid19 pandemic. This pandemic has various consequences on the political, cultural, economical and social life of the community. Lockdown has affected the psychological impact on society. This is reflected in various social media sites. In such a phase social media analytics for twitter data can be useful for understanding public opinion. In this paper, we have applied the Latent Dirichlet Allocation Algorithm as a topic modeling algorithm. Topic modeling finds the main theme that pervades the large data set. Twitter media is considered as the most popular microblogging platform, hence data during this pandemic is extracted from twitter. Natural language processing Techniques applied as preprocessing and then topic modeling applied which has given satisfactory results in terms of perplexity as a performance measure. Topic extracted gives an idea of the impact of Covid19 on society through their opinion on twitter. This can be helpful for making future policies by policymakers.
暂无评论