检索结果-内蒙古大学图书馆

A scalable algorithm for Constructing Frequent Pattern Tree

INTERNATIONAL JOURNAL OF INTELLIGENT INFORMATION TECHNOLOGIES 2014年第1期10卷 42-56页

作者： Abdullah, Zailani Herawan, Tutut Noraziah, A. Deris, Mustafa Univ Malaysia Dept Comp Sci Terengganu Malaysia Univ Ahmad Dahlan Dept Math Educ Yogyakarta Indonesia Univ Malaysia Comp Syst & Software Engn Kuantan Malaysia Univ Tun Hussein Onn Malaysia Comp Sci & Informat Technol Johor Baharu Malaysia

Frequent Pattern Tree (FP-Tree) is a compact data structure of representing frequent itemsets. The construction of FP-Tree is very important prior to frequent patterns mining. However, there have been too limited efforts specifically focused on constructing FP-Tree data structure beyond from its original database. In typical FP-Tree construction, besides the prior knowledge on support threshold, it also requires two database scans;first to build and sort the frequent patterns and second to build its prefix paths. Thus, twice database scanning is a key and major limitation in completing the construction of FP-Tree. Therefore, this paper suggests scalable Trie Transformation Technique algorithm (T3A) to convert our predefined tree data structure, Disorder Support Trie Itemset (DOSTrieIT) into FP-Tree. Experiment results through two UCI benchmark datasets show that the proposed T3A generates FP-Tree up to 3 magnitudes faster than that the benchmarked FP-Growth.

关键词： Association Rules Data Mining Frequent patterns Frequent Pattern Tree (FP-Tree) scalable algorithm

来源：评论

学校读者我要写书评

暂无评论

HPC Based scalable Logarithmic Kernelized Fuzzy Clustering algorithms for Handling Big Data 29th

HPC Based Scalable Logarithmic Kernelized Fuzzy Clustering A...

引用

29th International Conference on Neural Information Processing

作者： Jha, Preeti Tiwari, Aruna Bhari, Neha Ratnaparkhe, Milind Patel, Om Prakash Salonil, Sawarkar Sreeharsh, Namani Indian Inst Technol Indore Indore Madhya Pradesh India Mahindra Univ Hyderabad India ICAR Indian Inst Soybean Res Indore Madhya Pradesh India

ISBN: (纸本)9789819916412;9789819916429

With the advent of big data in the new millennium, the previous scalable clustering methods were no longer able to match the accuracy and efficiency requirements of big clustering. In light of this, we propose a fuzzy-based scalable incremental kernelized clustering algorithm for Big Data. First, we present the details of scalable kernelized fuzzy clustering techniques for Big Data that are based on the Radial Basis Function (RBF). Next, we define the membership degree and the cluster center for the logarithmic kernel function. For the purpose of managing Big Data, the Logarithmic Kernelized scalable Random Sampling with Iterative Optimization Fuzzy c-Means (LKSRSIO-FCM) clustering algorithm has been developed on Apache Spark. These kernel functions translate the input data space non-linearly onto a high-dimensional feature space, so, these kernelized clustering approaches have developed in order to deal with non-linearly separable problems. Hence, our aim is to design and implement the logarithmic kernelized incremental fuzzy clustering algorithms on Apache Spark, which, as a result of its in-memory cluster computing methodology, is able to effectively perform the clustering of Big Data. Extensive experiments on a variety of datasets derived from the real world demonstrate that the proposed LKSRSIO-FCM has superior performance to the scalable kernelized fuzzy clustering algorithms based on the RBF kernel in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score, respectively. The results were obtained by comparing the LKSRSIO-FCM to the scalable kernelized fuzzy clustering algorithms.

关键词： Fuzzy Clustering Logarithmic kernel scalable algorithm Big Data

来源：评论

学校读者我要写书评

暂无评论

Convergence guarantee for the sparse monotone single index model

引用

ELECTRONIC JOURNAL OF STATISTICS 2022年第2期16卷 4449-4496页

作者： Dai, Ran Song, Hyebin Barber, Rina Foygel Raskutti, Garvesh Univ Nebraska Med Ctr Dept Biostat Omaha NE 68198 USA Penn State Univ Dept Stat State Coll PA USA Univ Chicago Dept Stat Chicago IL USA Univ Wisconsin Dept Stat Madison WI USA

We consider a high-dimensional monotone single index model (hdSIM), which is a semiparametric extension of a high-dimensional gen-eralize linear model (hdGLM), where the link function is unknown, but constrained with monotone non-decreasing shape. We develop a scalable projection-based iterative approach, the "Sparse Orthogonal Descent Single -Index Model" (SOD-SIM), which alternates between sparse-thresholded or-thogonalized "gradient-like" steps and isotonic regression steps to recover the coefficient vector. Our main contribution is that we provide finite sam-ple estimation bounds for both the coefficient vector and the link function in high-dimensional settings under very mild assumptions on the design matrix X, the error term E, and their dependence. The convergence rate for the link function matches the low-dimensional isotonic regression minimax rate up to some poly-log terms (n-1/3). The convergence rate for the coeffi-cients is also n-1/3 up to some poly-log terms. This method can be applied to many real data problems, including GLMs with mis-specified link, clas-sification with mislabeled data, and classification with positive-unlabeled (PU) data. We study the performance of this method via both numerical studies and also an application on a PU data example.

关键词： Single-index model isotonic regression scalable algorithm high-dimensional

来源：评论

学校读者我要写书评

暂无评论

Efficiency of algorithms for Computing Influence and Information Spreading on Social Networks

引用

algorithmS 2022年第8期15卷 262-262页

作者： Kuikka, Vesa Aalto, Henrik Ijas, Matias Kaski, Kimmo K. Finnish Def Res Agcy Tykkikentantie 1POB 10 Riihimaki 11311 Finland Eficode Oy Helsinki 00100 Finland Aalto Univ Sch Sci Dept Comp Sci POB 15500 Aalto 00076 Finland Alan Turing Inst 96 Euston RdKings Cross London NW1 2DB England

Modelling interactions on complex networks needs efficient algorithms for describing processes on a detailed level in the network structure. This kind of modelling enables more realistic applications of spreading processes, network metrics, and analyses of communities. However, different real-world processes may impose requirements for implementations and their efficiency. We discuss different transmission and spreading processes and their interrelations. Two pseudo-algorithms are presented, one for the complex contagion spreading mechanism using non-self-avoiding paths in the modelling, and one for simple contagion processes using self-avoiding paths in the modelling. The first algorithm is an efficient implementation that can be used for describing social interaction in a social network structure. The second algorithm is a less efficient implementation for describing specific forms of information transmission and epidemic spreading.

关键词： social networks influence spreading information spreading epidemic spreading scalable algorithm pseudo-algorithm modelling social networks social media networks centrality measures community detection

来源：评论

学校读者我要写书评

暂无评论

High-Performance Computing based scalable Online Fuzzy Clustering algorithms for Big Data

High-Performance Computing based Scalable Online Fuzzy Clust...

引用

IEEE Symposium Series on Computational Intelligence (IEEE SSCI)

作者： Jha, Preeti Tiwari, Aruna Bharill, Neha Ratnaparkhe, Milind Patel, Om Prakash Pulakitha, Rapolu Chauhan, Aditi Indian Inst Technol Indore Comp Sci & Engn Indore India Mahindra Univ Comp Sci & Engn Hyderabad India ICAR Indian Inst Soybean Res Biotechnol Indore India

ISBN: (纸本)9781665487689

With the rise of big data trends so quickly, realtime stream data processing has become very important. Stream data is a type of big, fast, and unreliable dataset that cannot be handled well by traditional algorithms. Designing the algorithm that can efficiently process streaming data is a challenging task. This paper shows how important is to make a real-time clustering algorithm for data streams with high concept drift and an algorithm that can adapt to different dimensions. We propose scalable Random Sampling Online Optimization Weighted Fuzzy c-Means (SRSOO-WFCM) algorithms for handling Big Data in a High-Performance Computing (HPC) environment using an Apache Spark cluster. To compare SRSOO-WFCM with the traditional Online Fuzzy c-Means (OFCM) algorithm, we made a scalable version of OFCM named SOFCM. The proposed SRSOO-WFCM and SOFCM are incremental algorithms that involve making one sequential run through the data subsets. We employ both loadable and very large datasets to perform extensive experiments that facilitate comparing the proposed SRSOO-WFCM and SOFCM algorithms. The proposed SRSOOWFCM algorithm performs better than the SOFCM in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score, respectively.

关键词： scalable algorithm Online Fuzzy Clustering Incremental Streaming algorithm Apache Spark HPC Big Data

来源：评论

学校读者我要写书评

暂无评论

Scaling Up Mass-Based Clustering 22

Scaling Up Mass-Based Clustering

引用

31st ACM International Conference on Information and Knowledge Management (CIKM)

作者： Ahlawat, Nidhi Awekar, Amit Indian Inst Technol Guwahati Gauhati Assam India

ISBN: (纸本)9781450392365

This paper addresses the problem of scaling up the mass-based clustering paradigm to handle large datasets. The existing algorithm MBScan computes and stores all pairwise distances, resulting in quadratic time and space complexity. However, we observe that mass-based clustering requires information about only a tiny fraction of all possible data point pairs. We propose three optimizations to MBScan for quickly finding such pairs and computing their distances. We empirically evaluate our work on ten real-world and synthetic datasets. Our experiments show that our approach results in fast and memory-efficient clustering with no loss in the quality of clusters.

关键词： scalable algorithm data dependent dissimilarity mass based clustering

来源：评论

学校读者我要写书评

暂无评论

scalable Pilot Assignment Scheme for Cell-Free Large-Scale Distributed MIMO With Massive Access

引用

IEEE ACCESS 2021年 9卷 122107-122112页

作者： Li, Jiamin Wu, Zhenggang Zhu, Pengcheng Wang, Dongming You, Xiaohu Southeast Univ Natl Mobile Commun Res Lab Nanjing 210096 Peoples R China

With the explosive growth of mobile communication technology, the number of access terminals has increased dramatically, which will make the pilot contamination caused by pilot reuse extremely worse due to limited pilot resources. It is difficult to apply a traditional pilot assignment algorithm to serve the numerous access terminals simultaneously in real time with low complexity, so there is necessity to design a scalable pilot assignment scheme in the case of massive access. In this paper, we propose a scalable deep learning-based pilot assignment algorithm to maximize the sum spectral efficiency (SE) of cell-free large-scale distributed multiple-input multiple-output (MIMO) systems with massive access. The mapping between user locations and pilot assignment schemes is learned by a deep neural network (DNN). The training samples of the DNN are generated by a min-max algorithm, which minimizes the maximum interference to alleviate pilot contamination. The output of the pretrained DNN is used as the initial value of the min-max algorithm to achieve better pilot assignment schemes and reduce the algorithm complexity. The simulation results show that the proposed algorithm has better convergence with massive access and achieves a higher sum SE in near real time.

关键词： Contamination Resource management Interference Complexity theory Channel estimation Scalability Uplink scalable algorithm deep neural network (DNN) large-scale distributed MIMO pilot assignment min-max algorithm

来源：评论

学校读者我要写书评

暂无评论

A scalable APPROACH TO COMPUTE DELAY MARGIN OF A CLASS OF NEUTRAL-TYPE TIME DELAY SYSTEMS

引用

SIAM JOURNAL ON CONTROL AND OPTIMIZATION 2021年第2期59卷 805-824页

作者： Ramirez, Adrian Breda, Dimitri Sipahi, Rifat IPICYT Catedras CONACYT Div Control & Dynam Syst San Luis Potosi 78216 San Luis Potosi Mexico Univ Udine Dept Math Comp Sci & Phys CDLab Computat Dynam Lab I-33100 Udine Italy Northeastern Univ Dept Mech & Ind Engn Boston MA 02115 USA

An approach is presented to approximate the delay margin of a class of linear time-invariant neutral-type time delay systems. The approach is based on frequency sweeping tools and factorization of the corresponding characteristic function of the system, without any assumptions on the structure of system matrices. Different from published work, it enables the approximation of the delay margin on a scalable framework by means of a suitable numerical search of roots crossing the imaginary axis, hence it allows to substantially reduce computation times through parallel processing. Effectiveness of the approach is demonstrated on various problems.

关键词： time-delay systems neutral systems delay margin scalable algorithm

来源：评论

学校读者我要写书评

暂无评论

Communication-avoiding kernel ridge regression on parallel and distributed systems

引用

CCF TRANSACTIONS ON HIGH PERFORMANCE COMPUTING 2021年第3期3卷 252-270页

作者： You, Yang Huang, Jingyue Hsieh, Cho-Jui Vuduc, Richard Demmel, James Natl Univ Singapore Dept Comp Sci Singapore Singapore Fudan Univ Sch Data Sci Shanghai Peoples R China Univ Calif Los Angeles Dept Comp Sci Los Angeles CA 90024 USA Georgia Inst Technol Coll Comp Atlanta GA 30332 USA Univ Calif Berkeley Div Comp Sci Berkeley CA 94720 USA

Kernel ridge regression (KRR) is a fundamental method in machine learning. Given an n-by-d data matrix as input, a traditional implementation requires Theta(n(2)) memory to form an n-by-n kernel matrix and Theta(n(3)) flops to compute the final model. These time and storage costs prohibit KRR from scaling up to large datasets. For example, even on a relatively small dataset (a 520k-by-90 input requiring 357 MB), KRR requires 2 TB memory just to store the kernel matrix. The reason is that n usually is much larger than d for real-world applications. On the other hand, weak scaling becomes a problem: if we keep d and n/p fixed as p grows (p is the number of machines), the memory needed grows as Theta(p) per processor and the flops as Theta(p(2)) per processor. In the perfect weak scaling situation, both the memory needed and the flops grow as Theta(1) per processor (i.e. memory and flops are constant). The traditional Distributed KRR implementation (DKRR) only achieved 0.32% weak scaling efficiency from 96 to 1536 processors. We propose two new methods to address these problems: the balanced KRR (BKRR) and K-means KRR (KKRR). These methods consider alternative ways to partition the input dataset into pdifferent parts, generating pdifferent models, and then selecting the best model among them. Compared to a conventional implementation, KKRR2 (optimized version of KKRR) improves the weak scaling efficiency from 0.32 to 38% and achieves a 591x speedup for getting the same accuracy by using the same data and the same hardware (1536 processors). BKRR2 (optimized version of BKRR) achieves a higher accuracy than the current fastest method using less training time for a variety of datasets. For the applications requiring only approximate solutions, BKRR2 improves the weak scaling efficiency to 92% and achieves 3505x speedup (theoretical speedup: 4096x). The source code of this paper is available at https://***/similar to youyang/***.

关键词： Distributed machine learning Knights landing scalable algorithm

来源：评论

学校读者我要写书评

暂无评论

scalable Fuzzy Clustering-based Regression to Predict the Isoelectric Points of the Plant Protein Sequences using Apache Spark

Scalable Fuzzy Clustering-based Regression to Predict the Is...

引用

IEEE CIS International Conference on Fuzzy Systems (FUZZ-IEEE)

作者： Choudhary, Ajay Jha, Preeti Tiwari, Aruna Bharill, Neha Ratnaparkhe, Milind Indian Inst Technol Indore Comp Sci & Engn Indore India Mahindra Univ Ecole Cent Sch Engn Comp Sci & Engn Hyderabad India ICAR Indian Inst Soybean Res Indore India

ISBN: (纸本)9781665444071

Learning in non-stationary environments require modern tools and algorithms to quickly adapt to the new pattern because concept drift can change the underlying distribution. So, the existing assumption that the data is independent and identically distributed may be invalid in data stream scenarios. Given the massive volume of high-speed data streams and the concept drift, traditional machine learning algorithms must be self-adapting. One of the difficulties in handling regression tasks is the complexities of equations for the regression models when combined with drift handling techniques. The high dimensional protein data is a major challenge for bioinformatics researchers to analyse the dynamics of the sequences. This paper proposes a scalable Fuzzy Clustering induced Regression (SFC-R) algorithm to predict the isoelectric point of the plant protein sequences using Apache Spark clusters. The SFC-R algorithm uses the input features extracted from the plant protein sequences and validates performance in terms of mean squared error (MAE) and root-mean-square error (RMSE). Experiments on plant protein datasets are carried out to validate the high accuracy and robustness of our approach.

关键词： Fuzzy Clustering scalable algorithm Apache Spark Regression

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：