检索结果-内蒙古大学图书馆

A scalable and flexible basket analysis system for big transaction data in Spark

INFORMATION PROCESSING & MANAGEMENT 2024年第2期61卷

作者： Sun, Xudong Ngueilbaye, Alladoumbaye Luo, Kaijing Cai, Yongda Wu, Dingming Huang, Joshua Zhexue Shenzhen Univ Natl Engn Lab Big Data Syst Comp Technol Shenzhen 518060 Peoples R China Shenzhen Univ Big Data Inst Coll Comp Sci & Software Engn Shenzhen 518060 Peoples R China Guangdong Lab Artificial Intelligence & Digital Ec Shenzhen 518107 Peoples R China

Basket analysis is a prevailing technique to help retailers uncover patterns and associations of sold products in customer shopping transactions. However, as the size of transaction databases grows, the traditional basket analysis techniques and systems become less effective because of two issues in the applications of the big data age: data scalability and flexibility to adapt different application tasks. This paper proposes a scalable distributed frequent itemset mining (ScaDistFIM) algorithm for basket analysis on big transaction data to solve these two problems. ScaDistFIM is performed in two stages. The first stage uses the FP-Growth algorithm to compute the local frequent itemsets from each random subset of the distributed transaction dataset, and all random subsets are computed in parallel. The second stage uses an approximation method to aggregate all local frequent itemsets to the final approximate set of frequent itemsets where the support values of the frequent itemsets are estimated. We further elaborate on implementing the ScaDistFIM algorithm and a flexible basket analysis system using Spark SQL queries to demonstrate the system's flexibility in real applications. The experiment results on synthetic and real-world transaction datasets demonstrate that compared to the Spark FP-Growth algorithm, the ScaDistFIM algorithm can achieve time savings of at least 90% while ensuring nearly 100% accuracy. Hence, the ScaDistFIM algorithm exhibits superior scalability. On dataset GenD with 1 billion records, the ScaDistFIM algorithm requires only 360 s to achieve 100% precision and recall. In contrast, due to memory limitations, Spark FP-Growth cannot complete the computation task.

关键词： Big transaction data Frequent itemset mining parallel and distributed computing Business basket analysis Basket analysis systems

来源：评论

学校读者我要写书评

暂无评论

Big Data Processing in the ATLAS Experiment: Use Cases and Experience

引用

Procedia Computer Science 2015年 66卷 609-618页

作者： Mikhail Borodin Kaushik De Jose Garcia Navarro Dmitry Golubkov Alexei Klimentov Tadashi Maeno David South Alexandre Vaniachine Big Data Laboratory National Research Centre “Kurchatov Institute” Moscow Russia National Research Nuclear University “MEPhI” Moscow Russia University of Texas Arlington TX U.S.A Instituto de Fisica Corpuscular Universidad de Valencia Spain Institute for High Energy Physics Protvino Russia Brookhaven National Laboratory NY U.S.A Deutsches Elektronen-Synchrotron Hamburg Germany Argonne National Laboratory IL U.S.A

The physics goals of the next Large Hadron Collider run include high precision tests of the Standard Model and searches for new physics. These goals require detailed comparison of data with computational models simulating the expected data behavior. To highlight the role which modeling and simulation plays in future scientific discovery, we report on use cases and experience with a unified system built to process both real and simulated data of growing volume and variety.

关键词： Big Data Grid-based Simulation and computing parallel and distributed computing Large Scale Scientific Instruments

来源：评论

学校读者我要写书评

暂无评论

GPGPU-Aided Ensemble Empirical-Mode Decomposition for EEG Analysis During Anesthesia

引用

IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE 2010年第6期14卷 1417-1427页

作者： Chen, Dan Li, Duan Xiong, Muzhou Bao, Hong Li, Xiaoli Yanshan Univ Inst Elect Engn Qinhuangdao 066004 Peoples R China Yanshan Univ Inst Informat Sci & Engn Qinhuangdao 066004 Peoples R China China Univ Geosci Sch Comp Sci Wuhan 430074 Peoples R China Univ Sci & Technol Beijing Sch Informat Engn Beijing 100083 Peoples R China Univ Birmingham Sch Comp Sci Birmingham B15 2TT W Midlands England

Ensemble empirical-mode decomposition (EEMD) is a novel adaptive time-frequency analysis method, which is particularly suitable for extracting useful information from noisy non-linear or nonstationary data. Unfortunately, since the EEMD is highly compute-intensive, the method does not apply in real-time applications on top of commercial-off-the-shelf computers. Aiming at this problem, a parallelized EEMD method has been developed using general-purpose computing on the graphics processing unit (GPGPU), namely, G-EEMD. A spectral entropy facilitated by G-EEMD was, therefore, proposed to analyze the EEG data for estimating the depth of anesthesia (DoA) in a real-time manner. In terms of EEG data analysis, G-EEMD has dramatically improved the run-time performance by more than 140 times compared to the original serial EEMD implementation. G-EEMD also performs far better than another parallelized implementation of EEMD bases on conventional CPU-based distributed computing technology despite the latter utilizes 16 high-end computing nodes for the same computing task. Furthermore, the results obtained from a pharmacokinetics/pharmacodynamic (PK/PD) model analysis indicate that the EEMD method is slightly more effective than its precedent alternative method (EMD) in estimating DoA, the coefficient of determination R(2) by EEMD is significantly higher than that by EMD(p < 0.05, paired t-test) and the prediction probability P(k) by EEMD is also slighter higher than that by EMD (p < 0.2, paired t-test).

关键词： Depth of anesthesia (DoA) electroencephalogram (EEG) ensemble empirical-mode decomposition (EEMD) general-purpose computing on the graphics processing unit (GPGPU) parallel and distributed computing spectral entropy

来源：评论

学校读者我要写书评

暂无评论

Real science at the petascale

引用

PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES 2009年第1897期367卷 2557-2571页

作者： Saksena, Radhika S. Boghosian, Bruce Fazendeiro, Luis Kenway, Owain A. Manos, Steven Mazzeo, Marco D. Sadiq, S. Kashif Suter, James L. Wright, David Coveney, Peter V. UCL Dept Chem Ctr Computat Sci London WC1H 0AJ England Tufts Univ Dept Math Medford MA 02155 USA

We describe computational science research that uses petascale resources to achieve scientific results at unprecedented scales and resolution. The applications span a wide range of domains, from investigation of fundamental problems in turbulence through computational materials science research to biomedical applications at the forefront of HIV/AIDS research and cerebrovascular haemodynamics. This work was mainly performed on the US TeraGrid 'petascale' resource, Ranger, at Texas Advanced computing Center, in the first half of 2008 when it was the largest computing system in the world available for open scientific research. We have sought to use this petascale supercomputer optimally across application domains and scales, exploiting the excellent parallel scaling performance found on up to at least 32 768 cores for certain of our codes in the so-called 'capability computing' category as well as high-throughput intermediate-scale jobs for ensemble simulations in the 32-512 core range. Furthermore, this activity provides evidence that conventional parallel programming with MPI should be successful at the petascale in the short to medium term. We also report on the parallel performance of some of our codes on up to 65 636 cores on the IBM Blue Gene/P system at the Argonne Leadership computing Facility, which has recently been named the fastest supercomputer in the world for open science.

关键词： petascale computing parallel and distributed computing lattice-Boltzmann methods molecular dynamics material simulation computational biomedicine

来源：评论

学校读者我要写书评

暂无评论

EMINENT: Embarrassingly parallel Mutation Testing

引用

Procedia Computer Science 2016年 80卷 63-73页

作者： Pablo C. Cañizares Mercedes G. Merayo Alberto Núñez Universidad Complutense de Madrid Madrid Spain

During the last decade, the fast evolution in communication networks has facilitated the development of complex applications that manage vast amounts of data, like Big Data applications. Unfortunately, the high complexity of these applications hampers the testing process. Moreover, generating adequate test suites to properly check these applications is a challenging task due to the elevated number of potential test cases. Mutation testing is a valuable technique to measure the quality of the selected test suite that can be used to overcome this difficulty. However, one of the main drawbacks of mutation testing lies on the high computational cost associated to this process. In this paper we propose a dynamic distributed algorithm focused on HPC systems, called EMINENT, which has been designed to face the performance problems in mutation testing techniques. EMINENT alleviates the computational cost associated with this technique since it exploits parallelism in cluster systems to reduce the final execution time. In addition, several experiments have been carried out on three applications in order to analyse the scalability and performance of EMINENT. The results show that EMINENT provides an increase in the speed-up in most scenarios.

关键词： Mutation testing Scientific computing parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

A parallel Algorithm for a Physiological Non-linear Model of the Cochlea

引用

Procedia Computer Science 2013年 18卷 682-691页

作者： Doron Sabo Shlomo Weiss Miriam Furst School of Electrical Engineering Tel Aviv University Tel Aviv 69978 Israel

We present a parallel algorithm that solves a time-domain non-linear mathematical model of the cochlea. The previously known serial solution of the cochlear model is recursive in the longitudinal dimension and iterative in the time dimension. These two characteristics of the serial solution limit parallelism and prevent efficient computations on a massively parallel processor. We introduce a novel parallel algorithm that successfully overcomes the challenges posed by the cochlear model. We present performance results of a parallel implementation of the algorithm that shortens the computation time by a typical factor of 160 – 180, which makes the proposed algorithm of practical value for applications such as clinical audiological diagnosis.

关键词： Computational Methods Complex Systems Computational Applications parallel and distributed computing Advanced computing Architectures GPU

来源：评论

学校读者我要写书评

暂无评论

A vector model for routing queries in web search engines

引用

Procedia Computer Science 2010年第1期1卷 457-464页

作者： Mauricio S. Oyarzun Senen Gonzalez Marcelo Mendoza Flavio Ferrarotti Max Chacon Mauricio Marin Universidad de Santiago de Chile Universidad de Chile Yahoo! Research Latin America Santiago de Chile

This paper proposes a method for reducing the number of search nodes involved in the solution of queries arriving to a Web search engine. The method is applied by the query receptionist machine during situations of sudden peaks in query trafic to reduce the load on the search nodes. The experimental evaluation based on actual traces from users of a major search engine, shows that the proposed method outperforms alternative strategies. This is more evident for systems composed of a large number of search nodes which indicates that the method is also more scalable than the alternative strategies.

关键词： parallel and distributed computing Information retrieval Web search engines

来源：评论

学校读者我要写书评

暂无评论

A Pattern Recognition Approach for Speculative Firing Prediction in distributed Saturation State-Space Generation

引用

ELECTRONIC NOTES IN THEORETICAL COMPUTER SCIENCE 2006年第2期135卷 65-80页

作者： Chung, Ming-Ying Ciardo, Gianfranco Univ Calif Riverside Riverside CA 92521 USA

The saturation strategy for symbolic state-space generation is particularly effective for globallyasynchronous locally-synchronous systems. A distributed version of saturation, Saturation NOW, uses the overall memory available on a network of workstations to effectively spread the memory load, but its execution is essentially sequential. To achieve true parallelism, we explore a speculative firing prediction, where idle workstations work on predicted future event firing requests. A naive approach where all possible firings may be explored a priori, given enough idle time, can result in excessive memory requirements. Thus, we introduce a history-based approach for firing prediction that recognizes firing patterns and explores only firings conforming to these patterns. Experiments show that our heuristic improves the runtime and has a small memory overhead. The saturation strategy for symbolic state-space generation is particularly effective for globallyasynchronous locally-synchronous systems. A distributed version of saturation, SaturationNOW, uses the overall memory available on a network of workstations to effectively spread the memory load, but its execution is essentially sequential. To achieve true parallelism, we explore a speculative firing prediction, where idle workstations work on predicted future event firing requests. A naive approach where all possible firings may be explored a priori, given enough idle time, can result in excessive memory requirements. Thus, we introduce a history-based approach for firing prediction that recognizes firing patterns and explores only firings conforming to these patterns. Experiments show that our heuristic improves the runtime and has a small memory overhead.

关键词： state-space generation decision diagrams distributed systems parallel and distributed computing speculative computing pattern recognition

来源：评论

学校读者我要写书评

暂无评论

An Optical Interconnection Structure Based on the Dual of a Hypercube

引用

Informatica (Ljubljana) 1998年第4期22卷 499-508页

作者： Li, Yueming Zheng, S.Q. Wu, Jie Department of Computer Science Louisiana State University Baton Rouge LA 70803 United States Dept. of Comp. Sci. and Engineering Florida Atlantic University Boca Raton FL 33431 United States

A new class of interconnection networks, the hypernetworks, have been proposed recently. Hypernetworks are characterized by hypergraphs. Compared with point-to-point networks, they allow for increased resource-sharing and communication bandwidth utilization, and they are especially suitable for optical interconnects. One way to derive a hypernetwork is by finding the dual of a point-to-point network. Hypercube Qn, where n is the dimension, is a very popular point-to-point network. In this article, we consider using the dual Q*n of hypercube of Qn as an interconnection network. We investigate the properties of Q*n, and present a set of fundamental data communication algorithms for Q*n. Our results indicate that hypernetwork Q*n is a useful and promising interconnection structure for high-performance parallel and distributed computing systems.

关键词： Hypercube Hypergraph Hypernetwork Interconnection network Optical bus Optical interconnection parallel and distributed computing

来源：评论

学校读者我要写书评

暂无评论

Spatial Management of distributed Social Systems

引用

Journal of Computer Science Research 2020年第3期2卷 1-5页

作者： Peter Simon Sapaty Institute of Mathematical Machines and Systems National Academy of SciencesGlushkova Ave 4203187Kiev Ukraine

The paper describes the use of invented,developed,and tested in different countries of the high-level spatial grasp model and technology capable of solving important problems in large social systems,which may be represented as dynamic,self-evolving and distributed social *** approach allows us to find important solutions on a holistic level by spatial navigation and parallel pattern matching of social networks with active self-propagating scenarios represented in a special recursive *** approach effectively hides inside the distributed and networked language implementation traditional system management routines,often providing hundreds of times shorter and simpler high-level solution *** paper highlights the demands to efficient simulation of social systems,briefs the technology used,and provides some programming examples for solutions of practical problems.

关键词： Social systems Social networks parallel and distributed computing Spatial Grasp Technology Spatial Grasp Language Holistic solutions

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：