检索结果-内蒙古大学图书馆

An experimental study of existing tools for outlier detection and cleaning in trajectories

GEOINFORMATICA 2025年第1期29卷 31-51页

作者： Duarte, Mariana M. Garcez Sakr, Mahmoud Univ Libre Bruxelles Brussels Belgium Ain Shams Univ Cairo Egypt

Outlier detection and cleaning are essential steps in data preprocessing to ensure the integrity and validity of data analyses. This paper focuses on outlier points within individual trajectories, i.e., points that deviate significantly inside a single trajectory. We experiment with ten open-source libraries to comprehensively evaluate available tools, comparing their efficiency and accuracy in identifying and cleaning outliers. This experiment considers the libraries as they are offered to end users, with real-world applicability. We compare existing outlier detection libraries, introduce a method for establishing ground-truth, and aim to guide users in choosing the most appropriate tool for their specific outlier detection needs. Furthermore, we survey the state-of-the-art algorithms for outlier detection and classify them into five types: Statistic-based methods, Sliding window algorithms, Clustering-based methods, Graph-based methods, and Heuristic-based methods. Our research provides insights into these libraries' performance and contributes to developing data preprocessing and outlier detection methodologies.

关键词： Outlier detection Trajectory cleaning Trajectory preprocessing Experimental study programming libraries Outlier detection algorithms.

来源：评论

学校读者我要写书评

暂无评论

A high-level abstraction of shared accesses

引用

ACM TRANSACTIONS ON COMPUTER SYSTEMS 2000年第1期18卷 1-36页

作者： Keleher, PJ Univ Maryland Dept Comp Sci College Pk MD 20742 USA

We describe the design and use of the tape mechanism, a new high-level abstraction of accesses to shared data for software DSMs. Tapes consolidate and generalize a number of recent protocol optimizations, including update-based locks and record-replay barriers. Tapes are usually created by "recording" shared accesses. The resulting recordings can be used to anticipate future accesses by tailoring data movement to application semantics. Tapes-based mechanisms are layered on top of existing shared-memory protocols, and are largely independent of the underlying memory model. Tapes can also be used to emulate the data-movement semantics of several update-based protocol implementations, without altering the underlying protocol implementation. We have used tapes to create the Tapeworm synchronization library. Tapeworm implements sophisticated record-replay mechanisms across barriers, augments locks with data-movement semantics, and allows the use of producer-consumer segments, which move entire modified segments when any portion of the segment is accessed. We show that Tapeworm eliminates 85% of remote misses, reduces message traffic by 63%, and improves performance by an average of 29% for our application suite.

关键词： performance reliability DSM programming libraries shared memory update protocols

来源：评论

学校读者我要写书评

暂无评论

The NumPy Array: A Structure for Efficient Numerical Computation

引用

COMPUTING IN SCIENCE & ENGINEERING 2011年第2期13卷 22-30页

作者： van der Walt, Stefan Colbert, S. Chris Varoquaux, Gael Univ Stellenbosch ZA-7600 Stellenbosch South Africa Enthought Inc Austin TX USA INRIA Le Chesnay France

In the Python world, NumPy arrays are the standard representation for numerical data and enable efficient implementation of numerical computations in a high-level language. As this effort shows, NumPy performance can be improved through three techniques: vectorizing calculations, avoiding copying data in memory, and minimizing operation counts.

关键词： Arrays Computational efficiency Finite element methods NumPy Numerical analysis Performance evaluation Python Python programming language Resource management Vector quantization data structures high level language high level languages mathematics computing numerical analysis numerical computation numerical computations numerical data numpy array programming libraries scientific programming

来源：评论

学校读者我要写书评

暂无评论

SciPipe-Turning Scientific Workflows into Computer Programs

引用

COMPUTING IN SCIENCE & ENGINEERING 2019年第3期21卷 109-113页

作者： Lampa, Samuel Dahlo, Martin Alvarsson, Jonathan Spjuth, Ola Uppsala Univ Uppsala Sweden Uppsala Univ Dept Pharmaceut Biosci Bioinformat Uppsala Sweden Uppsala Univ Dept Pharmaceut Biosci Field Pharmaceut Bioinformat Uppsala Sweden Uppsala Univ Dept Pharmaceut Biosci Uppsala Sweden

Scientific workflows are becoming increasingly popular as a way to automate complex scientific computations consisting of multiple programs. One of the main motivations behind this development is increased robustness and reproducibility of computational analyses. Chaining together multiple programs using plain scripts, as is often the first step in automating a pipeline, can easily become fragile and error prone due to the manual management of file paths and program invocations. Also, plain scripts are not optimal if for some reason you have to cancel a run and try to restart it from any partially finished steps. It can be hard to know which output files are properly finished and which are truncated from the cancelled run. Last but not least, plain scripts do not by default save an execution trace of what was run, such that the full procedure used to create a specific output file can be clearly presented. These are all aspects that scientific workflows are designed to help with.

关键词： Application Program Interfaces Scientific Information Systems Software libraries Workflow Management Software Sci Pipe Scientific Workflows Computer Programs Computer Aided Science Data Pipelines Workflow Tools programming libraries Flow Based programming Principles Programmatic API Reproducibility programming Languages programming Pharmaceuticals Machine Learning Pipelines Task Analysis Bioinformatics Machine Learning

来源：评论

学校读者我要写书评

暂无评论

programming the Future: the Essential Role of System Topology Awareness in Heterogeneous Disaggregated Environments 24

Programming the Future: the Essential Role of System Topolog...

引用

10th International Symposium on Memory Systems

作者： Branchini, Beatrice Lavore, Ian Di Dio Castellana, Vito Giovanni Santambrogio, Marco Politecn Milan Milan Italy Pacific Northwest Natl Lab Richland WA USA

ISBN: (纸本)9798400710919

Heterogeneous disaggregated systems represent a promising solution to deliver the performance required by next-generation High-Performance Computing (HPC) workloads. Nevertheless, their heterogeneity represents a significant challenge in the application development process, making it urgent to identify solutions to program these systems productively and efficiently without sacrificing performance. In this paper, we explore the potential of system topology information in addressing such a challenge. We delve into the foundations of system topology and how this information could be exploited in high-level programming libraries across all layers of the software infrastructures, from runtime systems to user-facing APIs. We propose K-Nearest Neighbors (KNN) as a simple case study, which we implement with a distributed programming library prototyped to exploit topology awareness. We demonstrate the solution's effectiveness on a commodity cluster with GPU-equipped nodes and illustrate how it will apply to next-generation disaggregated hardware.

关键词： Disaggregated Memory High-Performance Computing Distributed Systems Heterogeneous Computing GPU programming libraries

来源：评论

学校读者我要写书评

暂无评论

The Paragraph: Design and Implementation of the STAPL Parallel Task Graph

The Paragraph: Design and Implementation of the STAPL Parall...

引用

作者： Thomas, Nathan 1977- Texas A&M University

Parallel programming is becoming mainstream due to the increased availability of multiprocessor and multicore architectures and the need to solve larger and more complex problems. Languages and tools available for the development of parallel applications are often difficult to learn and use. The Standard Template Adaptive Parallel Library (STAPL) is being developed to help programmers address these difficulties. STAPL is a parallel C++ library with functionality similar to STL, the ISO adopted C++ Standard Template Library. STAPL provides a collection of parallel pContainers for data storage and pViews that provide uniform data access operations by abstracting away the details of the pContainer data distribution. Generic pAlgorithms are written in terms of PARAGRAPHs, high level task graphs expressed as a composition of common parallel patterns. These task graphs define a set of operations on pViews as well as any ordering (i.e., dependences) on these operations that must be enforced by STAPL for a valid execution. The subject of this dissertation is the PARAGRAPH Executor, a framework that manages the runtime instantiation and execution of STAPL PARAGRAPHS. We address several challenges present when using a task graph program representation and discuss a novel approach to dependence specification which allows task graph creation and execution to proceed concurrently. This overlapping increases scalability and reduces the resources required by the PARAGRAPH Executor. We also describe the interface for task specification as well as optimizations that address issues such as data locality. We evaluate the performance of the PARAGRAPH Executor on several parallel machines including massively parallel Cray XT4 and Cray XE6 systems and an IBM Power5 cluster. Using tests including generic parallel algorithms, kernels from the NAS NPB suite, and a nuclear particle transport application written in STAPL, we demonstrate that the PARAGRAPH Executor enables STAPL to ex

关键词： Program Representation Program Composition programming libraries programming Languages Parallel Thesis

来源：评论

学校读者我要写书评

暂无评论

Parallel Program Composition with Paragraphs in Stapl

Parallel Program Composition with Paragraphs in Stapl

引用

作者： Smith, Timmie Texas A&M University

Languages and tools currently available for the development of parallel applications are difficult to learn and use. The Standard Template Adaptive Parallel Library (STAPL) is being developed to make it easier for programmers to implement a parallel application. STAPL is a parallel programming library for C++ that adopts the generic programming philosophy of the C++ Standard Template Library. STAPL provides collections of parallel algorithms (pAlgorithms) and containers (pContainers) that allow a developer to write their application without reimplementing the algorithms and data structures commonly used in parallel computing. pViews in STAPL are abstract data types that provide generic data access operations independently of the type of pContainer used to store the data. Algorithms and applications have a formal, high level representation in STAPL. A computation in STAPL is represented as a parallel task graph, which we call a PARAGRAPH. A PARAGRAPH contains a representation of the algorithm’s input data, the operations that are used to transform individual data elements, and the ordering between the application of operations that transform the same data element. Just as programs are the result of a composition of algorithms, STAPL programs are the result of a composition of PARAGRAPHs. This dissertation develops the PARAGRAPH program representation and its compositional methods. PARAGRAPHs improve the developer’s difficult situation by simplifying what she must specify when writing a parallel algorithm. The performance of the PARAGRAPH is evaluated using parallel generic algorithms, benchmarks from the NAS suite, and a nuclear particle transport application that has been written using STAPL. Our experiments were performed on Cray XT4 and Cray XE6 massively parallel systems and an IBM Power5 cluster, and show that scalable performance beyond 16,000 processors is possible using the PARAGRAPH.

关键词： Program Representation Program Composition programming libraries programming Languages Parallel Thesis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：