检索结果-内蒙古大学图书馆

作者： Singh, Shailendra University of Ottawa

学位级别：master

The big data framework applied to smart meters offers an exception platform for data-driven forecasting and decision making to achieve sustainable energy efficiency. Buying-in consumer confidence through respecting occupants'' energy consumption behavior and preferences towards improved participation in various energy programs is imperative but difficult to obtain. The key elements for understanding and predicting household energy consumption are activities occupants perform, appliances and the times that appliances are used, and inter-appliance dependencies. This information can be extracted from the context rich big data from smart meters, although this is challenging because: (1) it is not trivial to mine complex interdependencies between appliances from multiple concurrent data streams; (2) it is difficult to derive accurate relationships between interval based events, where multiple appliance usage persist; (3) continuous generation of the energy consumption data can trigger changes in appliance associations with time and appliances. To overcome these challenges, we propose an unsupervised progressive incremental data mining technique using frequent pattern mining (appliance-appliance associations) and cluster analysis (appliance-time associations) coupled with a Bayesian network based prediction model. The proposed technique addresses the need to analyze temporal energy consumption patterns at the appliance level, which directly reflect consumers'' behaviors and provide a basis for generalizing household energy models. Extensive experiments were performed on the model with real-world datasets and strong associations were discovered. The accuracy of the proposed model for predicting multiple appliances usage outperformed support vector machine during every stage while attaining accuracy of 81.65\%, 85.90\%, 89.58\% for 25\%, 50\% and 75\% of the training dataset size respectively. Moreover, accuracy results of 81.89\%, 75.88\%, 79.23\%, 74.74\%, and 72.81\% wer

关键词： Smart Grid Smart Meter Behavioral Analytics data-Driven Approach Energy Consumption Patterns Frequent Pattern Correlation Pattern Cluster Analysis Online data mining Incremental data mining distributed data mining Association Rules Appliance Usage Prediction Energy Consumption Prediction Prediction Visualization

来源：评论

学校读者我要写书评

暂无评论

Transient power quality assessment based on big data analysis

Transient power quality assessment based on big data analysi...

引用

China International Conference on Electricity Distribution (CICED)

作者： Huang Zhiwei Gao Tian Zhang Huaying Han Xu Cao Junwei Hu Ziheng Yao Senjing Zhu Zhengguo Shenzhen Power Supply Co Ltd Shenzhen Peoples R China Tsinghua Univ Beijing Shi Peoples R China

ISBN: (纸本)9781479941254

A transient power quality assessment method is proposed in this paper, using Naive Bayes classification method which is based on big data processing architecture. In this architecture, data sources will be extended to the aspects of power grid monitoring data, the power customer data and the public data, and the assessment severity will be classified into the normal state, the abnormal state, the critical state, and the failed state, according to the Naive Bayes classification results. Based on the data type of transient power quality assessment, big data processing architecture used in this paper can be able to process distributed data and streaming data, so that it can ensure not only updates classifier rules regularly, but also the real-time condition assessment. In the classifier training phase, we use the massive historical data as the distributed learning object, and generate assessment rules periodically. In the state assessment phase, each assessment node will update the assessment rules generated by training phase, generate real-time evaluation of samples from stream processing framework, and evaluate the power quality state according to the current rule. On this basis, this paper designs a Naive Bayes classification method based on MapReduce processing, and realizes the map and reduce process method to compute the priori probability and the conditional probability in distributed way. Experiments show that the transient power quality evaluation method based on the big data analysis presented in this paper is feasible, and achieve good results both in classification accuracy and processing speed.

关键词： Big data MapReduce distributed data mining Naive bayes classification

来源：评论

学校读者我要写书评

暂无评论

Secure distributed Computation of Anonymized Views of Shared databases

引用

ACM TRANSACTIONS ON dataBASE SYSTEMS 2012年第2期37卷 11-11页

作者： Tassa, Tamir Gudes, Ehud Open Univ Israel Dept Math & Comp Sci Raanana Israel Ben Gurion Univ Negev Dept Comp Sci IL-84105 Beer Sheva Israel

We consider the problem of computing efficient anonymizations of partitioned databases. Given a database that is partitioned between several sites, either horizontally or vertically, we devise secure distributed algorithms that allow the different sites to obtain a k-anonymized and l-diverse view of the union of their databases, without disclosing sensitive information. Our algorithms are based on the sequential algorithm [Goldberger and Tassa 2010] that offers anonymizations with utility that is significantly better than other anonymization algorithms, and in particular those that were implemented so far in the distributed setting. Our algorithms can apply to different generalization techniques and utility measures and to any number of sites. While previous distributed algorithms depend on costly cryptographic primitives, the cryptographic assumptions of our solution are surprisingly minimal.

关键词： Algorithms Design Experimentation Security k-anonymization privacy-preserving data mining secure multiparty computation distributed data mining

来源：评论

学校读者我要写书评

暂无评论

Improving prediction for medical institution with limited patient data: Leveraging hospital-specific data based on multicenter collaborative research network

引用

ARTIFICIAL INTELLIGENCE IN MEDICINE 2021年 113卷 102024-102024页

作者： Li, Jin Tian, Yu Li, Runze Zhou, Tianshu Li, Jun Ding, Kefeng Li, Jingsong Zhejiang Univ Coll Biomed Engn & Instrument Sci Engn Res Ctr EMR & Intelligent Expert Syst Minist Educ 38 Zheda Rd Hangzhou 310027 Zhejiang Peoples R China Zhejiang Univ Affiliated Hosp 2 Sch Med Dept Surg Oncol Hangzhou Peoples R China Res Ctr Healthcare Data Sci Zhejiang Lab Hangzhou Peoples R China

Background and objective: Clinical decision support assisted by prediction models usually faces the challenges of limited clinical data and a lack of labels when the model is developed with data from a single medical institution. Accordingly, research on multicenter clinical collaborative networks, which can provide external medical data, has received increasing attention. With the increasing availability of machine learning techniques such as transfer learning, leveraging large-scale patient data from multiple hospitals to build data-driven predictive models with clinical application potential provides an alternative solution to address the problem of limited patient data. Methods: A multicenter hybrid semi-supervised transfer learning model (MHSTL) is proposed in this study on the basis of unified common data model to ensure multicenter data standardized representation. Then the hospital-specific features, along with the co-occurrence features across domains, are aligned through a representation learning architecture that is built based on deep neural networks and the newly proposed neural decision forest model. In this process, limited patient data from the target hospital, both labeled and unlabeled, are incorporated during the feature adaptation process, thereby contributing to better model performance. Without patient-level data sharing, the proposed model learning strategy which overcomes feature misalignment and distribution divergence, enables the multi-source transfer learning process in the case of insufficient and unlabeled patient data at target hospital. Results: The effectiveness of the proposed transfer learning model was evaluated on a collaborative research network of colorectal cancer patients in the US and China. The results demonstrate that the proposed model can achieve much better performance for predicting target risk with limited resources on patient data than baseline models . Better discrimination and calibration ability are also observed

关键词： Transfer learning data-limited settings distributed data mining Model generalizability Clinical decision support systems Prognosis prediction

来源：评论

学校读者我要写书评

暂无评论

Privacy Preservation for Global Cyclic Associations in distributed databases

引用

Procedia Technology 2012年 6卷 962-969页

作者： Nirali Nanavati Devesh Jinwala First affiliation Address City and Postcode Country Second affiliation Address City and Postcode Country

There have been immense advancements in collaborative data analysis. Hence the desire to keep one's own sensitive data secret is an important challenge of preserving privacy while mining in a distributed setup. There have been indeed numerous attempts to propose privacy preserving data mining algorithms. However, in order to extract the associations with respect to the time segment, the cyclic nature of the association rules must be examined. To the best of our knowledge and explorations, the current privacy preserving algorithms, do not consider the cyclic nature of the association rules. Hence, we here propose techniques with illustrative examples that help to privately decipher cyclic association rules that are common to all the participating parties. The techniques proposed by us for a multi-party scenario are based on homomorphism and Shamir's secret sharing.

关键词： Privacy Cyclic Association Rules distributed data mining

来源：评论

学校读者我要写书评

暂无评论

A METHODOLOGY FOR SCALING UP CLASSIFICATION ALGORITHMS

引用

International Journal on Artificial Intelligence Tools 2004年第3期13卷 623-639页

作者： BASILIS BOUTSINAS GEORGE PRASSAS GERASIMOS ANTZOULATOS Dept. of Business Administration University of Patras Artificial Intelligence Research Center (UPAIRC) University of Patras 26500 Rio Patras Greece UPAIRC 26500 Rio Patras Greece

Contrary to much of the research in machine learning where there is a concentration on problems with relatively small volume of data, one of the main challenges of the today's data mining systems is their ability to handle data that is substantially larger than available main memory on a single processor. In this paper, we present a distributed technique that consists in combining the partial results of different classifiers supplied, in parallel, with different subsets of data. It is, actually, a two-phase process. At first, a number of classifiers are trained, each with a different subset of the data. Then, the trained classifiers are used in the construction of a new training data set which has exactly the same format as, but is substantially smaller than, the initial one. The new data set is used to train the final classifier through an iterative process, which is guided by a threshold concerning the size of the data set and the achieved increase in accuracy. We present extensive empirical tests, which demonstrate that the proposed technique significantly reduces time complexity, usually at the expense of a lower accuracy, compared to a single classifier supplied with all the data.

关键词： data mining classification distributed data mining meta-learning

来源：评论

学校读者我要写书评

暂无评论

A Reduced Network Traffic Method for IoT data Clustering

引用

ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM data 2021年第1期15卷 13-13页

作者： De Azevedo, Ricardo Machado, Gabriel Resende Goldschmidt, Ronaldo Ribeiro Choren, Ricardo Mil Inst Engn Praca Gen Tiburcio 80 Rio De Janeiro RJ Brazil

Internet of Things (IoT) systems usually involve interconnected, low processing capacity, and low memory sensor nodes (devices) that collect data in several sorts of applications that interconnect people and things. In this scenario, mining tasks, such as clustering, have been commonly deployed to detect behavioral patterns from the collected data. The centralized clustering of IoT data demands high network traffic to transmit the data from the devices to a central node, where a clustering algorithm must be applied. This approach does not scale as the number of devices increases, and the amount of data grows. However, distributing the clustering process through the devices may not be a feasible approach as well, since the devices are usually simple and may not have the ability to execute complex procedures. This work proposes a centralized IoT data clustering method that demands reduced network traffic and low processing power in the devices. The proposed method uses a data grid to summarize the information at the devices before transmitting it to the central node, reducing network traffic. After the data transfer, the proposed method applies a clustering algorithm that was developed to process data in the summarized representation. Tests with seven datasets provided experimental evidence that the proposed method reduces network traffic and produces results comparable to the ones generated by DBSCAN and HDBSCAN, two robust centralized clustering algorithms.

关键词： data traffic reduction data summarization Internet of Things distributed data mining

来源：评论

学校读者我要写书评

暂无评论

Performance Evaluation of distributed Association Rule mining Algorithms

引用

Procedia Computer Science 2016年 79卷 127-134页

作者： Vinaya Sawant Ketan Shah Assistant Professor IT Department D. J. Sanghvi College of Engineering Mumbai India Professor IT Department MPSTME Mumbai India

Association Rule mining (ARM) is a popular and well researched method for discovering interesting relations between variables in large databases. It is intended to identify strong rules discovered in databases using different measures of interestingness. Most ARM algorithms focus on a sequential or centralized environment where no external communication is required. distributed ARM algorithms (DARM), aim to generate rules from different data sets spread over various geographical sites; hence, they require external communications throughout the entire process. DARM algorithm efficiency is highly dependent on data distribution. The Classical algorithms used in DARM are Count Distribution Algorithm (CDA), Fast distributed mining (FDM) Algorithm and Optimized distributed Association mining (ODAM) Algorithm. This paper presents the implementation details and experimental results of above mentioned algorithms. The paper also highlights the issues of message exchange size in a distributed environment of current DARM algorithms that can affect the communication costs in a distributed environment.

关键词： Association Rule mining distributed data mining

来源：评论

学校读者我要写书评

暂无评论

Synthesizing High-Utility Patterns from Different data Sources

引用

data 2018年第3期3卷 32-32页

作者： Muley, Abhinav Gudadhe, Manish St Vincent Pallotti Coll Engn & Technol Dept Comp Engn Nagpur 441108 Maharashtra India

In large organizations, it is often required to collect data from the different geographic branches spread over different locations. Extensive amounts of data may be gathered at the centralized location in order to generate interesting patterns via mono-mining the amassed database. However, it is feasible to mine the useful patterns at the data source itself and forward only these patterns to the centralized company, rather than the entire original database. These patterns also exist in huge numbers, and different sources calculate different utility values for each pattern. This paper proposes a weighted model for aggregating the high-utility patterns from different data sources. The procedure of pattern selection was also proposed to efficiently extract high-utility patterns in our weighted model by discarding low-utility patterns. Meanwhile, the synthesizing model yielded high-utility patterns, unlike association rule mining, in which frequent itemsets are generated by considering each item with equal utility, which is not true in real life applications such as sales transactions. Extensive experiments performed on the datasets with varied characteristics show that the proposed algorithm will be effective for mining very sparse and sparse databases with a huge number of transactions. Our proposed model also outperforms various state-of-the-art distributed models of mining in terms of running time.

关键词： data integration data mining high-utility patterns knowledge discovery weighted model multi-database mining distributed data mining

来源：评论

学校读者我要写书评

暂无评论

New Approach for distributed Clustering

New Approach for Distributed Clustering

引用

2011 IEEE International Conference on Spatial data mining and Geographical Knowledge Services(ICSDM 2011)

作者： Souhila Ghanem Tahar Kechadi A.Kamel Tari Department of Computer Science University of Bjaia Targa OuzemourBejaiaAlgeria School of Computer Science and Informatics University College Dublin BelfieldDublin 04Ireland

Nowadays the data collections are huge and in most cases do not reside in a centralised *** latter complicates the task of traditional data mining techniques,as datasets are distributed and often *** this paper we propose a distributed approach based on the aggregation of models produced *** datasets will be processed locally on each node to produce clusters from local data then,we construct global clusters *** aim of this approach is to minimise the communications,maximise the parallelism and load balance the work among different nodes of the system,and reduce the overhead due to extra processing while executing the hierarchical *** technique is evaluated and compared to the sequential version using benchmark datasets and the results are very promising.

关键词： data mining distributed data mining Clustering OPTICS

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：