检索结果-内蒙古大学图书馆

IEEE North-West-Russia-Section Young Researchers in Electrical and Electronic Engineering Conference (ElConRusNW)

作者： Kholod, Ivan Borisenko, Konstantin St Petersburg Electrotech Univ LETI Fac Comp Sci & Technol St Petersburg Russia

ISBN: (纸本)9781509004454

the article describes the mapping of the algorithm decomposed into functional blocks on a distributed execution environment. In addition, it describes the architecture and implementation of service to perform data mining algorithms in that environment. As an example, it describes the implementation and experiments with classification algorithm - 1R.

关键词： distributed data mining cloud computing data mining cloud

来源：评论

学校读者我要写书评

暂无评论

CrossFIM: a spark-based hybrid frequent itemset mining algorithm for large datasets

引用

CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS 2025年第4期28卷 1-23页

作者： Raj, Shashi Ramesh, Dharavath Gantela, Prabhakar Bakhtiyarpur Coll Engn Dept Comp Sci & Engn Patna 803212 Bihar India Indian Sch Mines Indian Inst Technol Dept Comp Sci & Engn Dhanbad 826004 Jharkhand India Univ Econ & Human Sci Departmet Comp Sci Warsaw Poland Swarna Bharathi Inst Sci & Technol Dept Comp Sci & Engn Khammam 507001 Telangana India

Frequent Itemset mining (FIM) is the fundamental technique for discovering interesting patterns from transactional datasets. Typical algorithmic solutions for extracting such patterns are inefficient since they lead to an exponential increase in computational complexity with input data size. Due to the rapid growth in data accumulation, FIM demands high computational resources. Numerous adaptations of the fundamental FIM algorithms have been presented in a distributed setting to accommodate the ever-growing data needs. In continuation of the episode, several MapReduce-based FIM techniques have recently been demonstrated to alleviate the speed and scalability constraints. However, the existing MapReduce-based alternatives continue to underperform because of the iterative approach of algorithms, large intermediate data produced, workload skewness, and high disk I/O and network communication overhead. This paper proposes CrossFIM, a scalable Spark-based FIM algorithm combining Apriori and Eclat-based variants with partitioning techniques. The Apriori variant is used for the first few iterations until frequent itemsets start decreasing. After that, the Eclat variant is used with the partitioning technique. When used in conjunction with the Eclat variant, the partitioning approach makes it possible to efficiently handle large transaction sets (or diffsets) in memory and compute their intersection effectively. It uses the advantages of both horizontal and vertical data formats to boost performance. It regulates the number of key-value pairs shuffled amongst cluster nodes throughout iterations. It was demonstrated that CrossFIM significantly outperforms other state-of-the-art methods in efficiency and scalability through comprehensive trials using benchmark datasets.

关键词： distributed data mining Frequent itemset mining Apache spark Large datasets

来源：评论

学校读者我要写书评

暂无评论

Learning latent variable models from distributed and abstracted data

引用

INFORMATION SCIENCES 2011年第14期181卷 2964-2988页

作者： Zhang, Xiaofeng Cheung, William K. Li, C. H. Harbin Inst Technol Sch Comp Sciecne & Technol Shenzhen Grad Sch Kowloon Tong Hong Kong Peoples R China Hong Kong Baptist Univ Dept Comp Sci Kowloon Tong Hong Kong Peoples R China

Discovering global knowledge from distributed data sources is challenging, where the important issues include the ever-increasing data volume at the highly distributed sources and the general concern on data privacy. Properly abstracting the distributed data with a compact representation which can retain sufficient local details for global knowledge discovery in principle can address both the scalability and the data privacy challenges. This calls for the need to develop formal methodologies to support knowledge discovery on abstracted data. In this paper, we propose to abstract distributed data as Gaussian mixture models and learn a family of generative models from the abstracted data using a modified EM algorithm. To demonstrate the effectiveness of the proposed approach, we applied it to learn (a) data cluster models and (b) data manifold models, and evaluated their performance using both synthetic and benchmark data sets with promising results in terms of both effectiveness and scalability. Also, we have demonstrated that the proposed approach is robust against heterogeneous data distributions over the distributed sources. (C) 2011 Published by Elsevier Inc.

关键词： distributed data mining data abstraction Model-based methods Gaussian mixture model Generative topographic mapping

来源：评论

学校读者我要写书评

暂无评论

Fast distributed outlier detection in mixed-attribute data sets

引用

data mining AND KNOWLEDGE DISCOVERY 2006年第2-3期12卷 203-228页

作者： Otey, ME Ghoting, A Parthasarathy, S Ohio State Univ Dept Comp Sci & Engn Columbus OH 43210 USA

Efficiently detecting outliers or anomalies is an important problem in many areas of science, medicine and information technology. Applications range from data cleaning to clinical diagnosis, from detecting anomalous defects in materials to fraud and intrusion detection. Over the past decade, researchers in data mining and statistics have addressed the problem of outlier detection using both parametric and non-parametric approaches in a centralized setting. However, there are still several challenges that must be addressed. First, most approaches to date have focused on detecting outliers in a continuous attribute space. However, almost all real-world data sets contain a mixture of categorical and continuous attributes. Categorical attributes are typically ignored or incorrectly modeled by existing approaches, resulting in a significant loss of information. Second, there have not been any general-purpose distributed outlier detection algorithms. Most distributed detection algorithms are designed with a specific domain (e.g. sensor networks) in mind. Third, the data sets being analyzed may be streaming or otherwise dynamic in nature. Such data sets are prone to concept drift, and models of the data must be dynamic as well. To address these challenges, we present a tunable algorithm for distributed outlier detection in dynamic mixed-attribute data sets.

关键词： outlier detection anomaly detection distributed data mining mining dynamic data mixedattribute data sets data streams

来源：评论

学校读者我要写书评

暂无评论

Social choice in distributed classification tasks: Dealing with vertically partitioned data

引用

INFORMATION SCIENCES 2016年 332卷 56-71页

作者： Recarnonde-Mendoza, Mariana Bazzan, Ana L. C. Univ Fed Rio Grande do Sul Inst Informat PPGC Porto Alegre RS Brazil

In many situations, a centralized, conventional classification task can not be performed because the data is not available in a central facility. In such cases, we are dealing with distributed data mining problems, in which local models must be individually built and later combined into a consensus, global model. In this paper, we are particularly interested in distributed classification tasks with vertically partitioned data, i.e., when features are distributed among several sources. This restriction implies a challenging scenario given that the development of ail accurate model usually requires access to all the features that are relevant for classification. To deal with such a situation, we propose an agent-based classification system, in which the preference orderings of each agent regarding the probability of an instance to belong to the target class are aggregated by means of social choice functions. We employ this method to classify microRNA target genes, an important bioinformatics problem, showing that the predictions derived from the social choice tend to outperform local models in this application. This performance gain is accompanied by other interesting advantages: the aggregation methods herein proposed are extremely simple, do not require transfer of large volumes of data, do not assume an offline training process or parameters setup, and preserve data privacy. (C) 2015 Elsevier Inc. All rights reserved.

关键词： distributed data mining Vertical partition Heterogeneous sources Social choice theory

来源：评论

学校读者我要写书评

暂无评论

A distributed data clustering algorithm in P2P networks

引用

APPLIED SOFT COMPUTING 2017年 51卷 147-167页

作者： Azimi, Rasool Sajedi, Hedieh Ghayekhloo, Mohadeseh Islamic Azad Univ Qazvin Branch Young Researchers & Elite Club Qazvin Iran Univ Tehran Coll Sci Sch Math Stat & Comp Dept Comp Sci Tehran Iran

Clustering is one of the important data mining issues, especially for large and distributed data analysis. distributed computing environments such as Peer-to-Peer (P2P) networks involve separated/scattered data sources, distributed among the peers. According to unpredictable growth and dynamic nature of P2P networks, data of peers are constantly changing. Due to the high volume of computing and communications and privacy concerns, processing of these types of data should be applied in a distributed way and without central management. Today, most applications of P2P systems focus on unstructured P2P systems. In unstructured P2P networks, spreading gossip is a simple and efficient method of communication, which can adapt to dynamic conditions in these networks. Recently, some algorithms with different pros and cons have been proposed for data clustering in P2P networks. In this paper, by combining a novel method for extracting the representative data, a gossip-based protocol and a new centralized clustering method, a Gossip Based distributed Clustering algorithm for P2P networks called GBDC-P2P is proposed. The GBDC-P2P algorithm is suitable for data clustering in unstructured P2P networks and it adapts to the dynamic conditions of these networks. In the GBDC-P2P algorithm, peers perform data clustering operation with a distributed approach only through communications with their neighbours. The GBDC-P2P does not need to rely on a central server and it performs asynchronously. Evaluation results demonstrate the superior performance of the GBDC-P2P algorithm. Also, a comparative analysis with other well-established methods illustrates the efficiency of the proposed method. (C) 2016 Elsevier B. V. All rights reserved.

关键词： distributed data mining data clustering Gossiping Overlay peer-to-Peer networka

来源：评论

学校读者我要写书评

暂无评论

Induction of multiclass multifeature split decision trees from distributed data

引用

PATTERN RECOGNITION 2009年第9期42卷 1786-1794页

作者： Ouyang, Jie Patel, Nilesh Sethi, Ishwar Oakland Univ Dept Comp Sci & Engn Intelligent Informat Engn Lab Rochester MI 48309 USA

The decision tree-based classification is a popular approach for pattern recognition and data mining. Most decision tree induction methods assume training data being present at one central location. Given the growth in distributed databases at geographically dispersed locations, the methods for decision tree induction in distributed settings are gaining importance. This paper describes one such method that generates compact trees using multifeature splits in place of single feature split decision trees generated by most existing methods for distributed data. Our method is based on Fisher's linear discriminant function, and is capable of dealing with multiple classes in the data. For homogeneously distributed data, the decision trees produced by our method are identical to decision trees generated using Fisher's linear discriminant function with centrally stored data. For heterogeneously distributed data, a certain approximation is involved with a small change in performance with respect to the tree generated with centrally stored data. Experimental results for several well-known datasets are presented and compared with decision trees generated using Fisher's linear discriminant function with centrally stored data. (C) 2009 Elsevier Ltd. All rights reserved.

关键词： distributed data mining Decision tree Fisher linear discriminant

来源：评论

学校读者我要写书评

暂无评论

Evolutionary k-means for distributed data sets

引用

NEUROCOMPUTING 2014年第1期127卷 30-42页

作者： Naldi, M. C. Campello, R. J. G. B. Univ Fed Vicosa BR-38810000 Rio Paranaiba MG Brazil Univ Sao Paulo Inst Math & Comp Sci BR-13560970 Sao Carlos SP Brazil

One of the challenges for clustering resides in dealing with data distributed in separated repositories, because most clustering techniques require the data to be centralized. One of them, k-means, has been elected as one of the most influential data mining algorithms for being simple, scalable and easily modifiable to a variety of contexts and application domains. Although distributed versions of k-means have been proposed, the algorithm is still sensitive to the selection of the initial cluster prototypes and requires the number of clusters to be specified in advance. In this paper, we propose the use of evolutionary algorithms to overcome the k-means limitations and, at the same time, to deal with distributed data. Two different distribution approaches are adopted: the first obtains a final model identical to the centralized version of the clustering algorithm;the second generates and selects clusters for each distributed data subset and combines them afterwards. The algorithms are compared experimentally from two perspectives: the theoretical one, through asymptotic complexity analyses;and the experimental one, through a comparative evaluation of results obtained from a collection of experiments and statistical tests. The obtained results indicate which variant is more adequate for each application scenario. (C) 2013 Elsevier B.V. All rights reserved.

关键词： distributed clustering Evolutionary k-means distributed data mining

来源：评论

学校读者我要写书评

暂无评论

A Review of distributed data Models for Learning 12th

A Review of Distributed Data Models for Learning

引用

12th International Conference on Hybrid Artificial Intelligent Systems (HAIS)

作者： Angel Rodriguez, Miguel Fernandez, Alberto Peregrin, Antonio Herrera, Francisco Univ Huelva Dept Informat Technol Huelva Spain Univ Granada Dept Comp Sci & Artificial Intelligence Granada Spain

ISBN: (纸本)9783319596501;9783319596495

This paper deals with aspects of data distribution for machine learning tasks, considering the advantages as well as the drawbacks that are frequently associated with data partitioning and its different models. This study, from the point of view of the distributed data, reviews some of the algorithms that have been used to treat each case, although it is not a review of learning or computation algorithms. Finally, this report looks into the issues that new data partitioning based models such as MapReduce have brought to distributed learning.

关键词： distributed data mining data partitioning MapReduce

来源：评论

学校读者我要写书评

暂无评论

Communication efficient construction of decision trees over heterogeneously distributed data

Communication efficient construction of decision trees over ...

引用

4th IEEE International Conference on data mining

作者： Giannella, C Liu, K Olsen, T Kargupta, H Univ Maryland Baltimore Cty Dept Comp Sci & Elect Engn Baltimore MD 21250 USA

ISBN: (纸本)0769521428

We present an algorithm designed to efficiently construct a decision tree over heterogeneously distributed data without centralizing. We compare our algorithm against a standard centralized decision tree implementation in terms of accuracy as well as the communication complexity. Our experimental results show that by using only 20% of the communication cost necessary to centralize the data we can achieve trees with accuracy at least 80% of the trees produced by the centralized version.

关键词： decision trees distributed data mining random projection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：