检索结果-内蒙古大学图书馆

On the distributed software architecture of a data analysis workflow: A case study

CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE 2022年第9期34卷

作者： Tasgetiren, Nail Tigrak, Umit Bozan, Erdal Gul, Guven Demirci, Emir Saribiyik, Hakan Aktas, Mehmet S. Yildiz Tech Univ Dept Comp Engn Istanbul Turkey Fibabanka Res & Dev Ctr Istanbul Turkey

Hybrid distributed computing software architectures gain great importance in data analysis workflows as the number of available underlying machine learning libraries and data storage systems increase. We argue that there is a need for novel approaches for software architecture designs that can enable machine learning data analysis workflows to run on top of different subsystem libraries. To address this need, we propose a hybrid distributed software architecture in this manuscript. The proposed architecture manages machine learning models for both supervised and unsupervised machine learning data analysis workflows. To show the usability of the proposed architecture, we implement a prototype for the banking sector as a case study. The prototype application includes two data analysis workflows: a workflow for predicting the loan usage tendency of customers, and a workflow for clustering the customers based on the usage patterns of banking loans. The prototype is tested on a large scale banking dataset. Performance tests were carried out to investigate the performance in terms of both responsiveness and scalability of the system. The results obtained reveal the usability of the proposed architecture.

关键词： data analysis workflow distributed software architecture facade design pattern lambda software architecture machine learning workflows

来源：评论

学校读者我要写书评

暂无评论

A practical guide for single-cell transcriptome data analysis in neuroscience

引用

Neuroscience Research 2025年 214卷 9-15页

作者： Hayakawa, Yoshinori Ozaki, Haruka Bioinformatics Laboratory Institute of Medicine University of Tsukuba 1-1-1 Tennodai Ibaraki Tsukuba 305-8577 Japan Center for Artificial Intelligence Research University of Tsukuba 1-1-1 Tennodai Ibaraki Tsukuba 305-8577 Japan

Single-cell RNA sequencing (scRNA-seq) has revolutionized our ability to analyze gene expression at the single-cell level, providing unprecedented insights into cellular heterogeneity, rare cell populations, and dynamic cellular processes. In neuroscience, scRNA-seq has enabled the identification of diverse brain cell types, elucidation of developmental pathways, and discovery of mechanisms underlying neurological diseases. This tutorial provides a practical guide to scRNA-seq data analysis in neuroscience, focusing on the essential workflows and theoretical foundations. Key steps covered include quality control, data preprocessing, integration, cell clustering, and differential expression analysis. Using the Seurat R package, the tutorial demonstrates a comparative analysis approach for identifying differentially expressed genes between conditions, emphasizing the biological interpretation of results. By addressing the unique challenges of scRNA-seq data and illustrating methods for robust analysis, this work aims to enhance the reliability and reproducibility of scRNA-seq studies in neuroscience, supporting the exploration of cellular mechanisms and advancing research into brain function and disease. © 2025 The Authors

关键词： data analysis workflow Neuroscience Practical guide Single-cell RNA sequencing

来源：评论

学校读者我要写书评

暂无评论

A mathematical programming approach for resource allocation of data analysis workflows on heterogeneous clusters

引用

JOURNAL OF SUPERCOMPUTING 2023年第17期79卷 19019-19048页

作者： Mohammadi, Somayeh PourKarimi, Latif Droop, Felix De Mecquenem, Ninon Leser, Ulf Reinert, Knut Freie Univ Dept Math & Comp Sci Berlin Germany Razi Univ Dept Math & Comp Sci Kermanshah Iran Humboldt Univ Dept Math & Comp Sci Berlin Germany

Scientific communities are motivated to schedule their large-scale data analysis workflows in heterogeneous cluster environments because of privacy and financial issues. In such environments containing considerably diverse resources, efficient resource allocation approaches are essential for reaching high performance. Accordingly, this research addresses the scheduling problem of workflows with bag-of-task form to minimize total runtime (makespan). To this aim, we develop a mixed-integer linear programming model (MILP). The proposed model contains binary decision variables determining which tasks should be assigned to which nodes. Also, it contains linear constraints to fulfill the tasks requirements such as memory and scheduling policy. Comparative results show that our approach outperforms related approaches in most cases. As part of the post-optimality analysis, some secondary preferences are imposed on the proposed model to obtain the most preferred optimal solution. We analyze the relaxation of the makespan in the hope of significantly reducing the number of consumed nodes.

关键词： Heterogeneous cluster environments Scheduling data analysis workflow Mixed integer linear programming Makespan minimization Post-optimality analysis

来源：评论

学校读者我要写书评

暂无评论

JTSA: An open source framework for time series abstractions

引用

COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2015年第3期121卷 175-188页

作者： Sacchi, Lucia Capozzi, Davide Bellazzi, Riccardo Larizza, Cristiana Univ Pavia Dept Elect Comp & Biomed Engn I-27100 Pavia Italy BIOMERIS Srl I-27100 Pavia Italy

Background and objective: The evaluation of the clinical status of a patient is frequently based on the temporal evolution of some parameters, making the detection of temporal patterns a priority in data analysis. Temporal abstraction (TA) is a methodology widely used in medical reasoning for summarizing and abstracting longitudinal data. Methods: This paper describes JTSA (Java Time Series Abstractor), a framework including a library of algorithms for time series preprocessing and abstraction and an engine to execute a workflow for temporal data processing. The JTSA framework is grounded on a comprehensive ontology that models temporal data processing both from the data storage and the abstraction computation perspective. The JTSA framework is designed to allow users to build their own analysis workflows by combining different algorithms. Thanks to the modular structure of a workflow, simple to highly complex patterns can be detected. The JTSA framework has been developed in Java 1.7 and is distributed under GPL as a jar file. Results: JTSA provides: a collection of algorithms to perform temporal abstraction and preprocessing of time series, a framework for defining and executing data analysis workflows based on these algorithms, and a GUI for workflow prototyping and testing. The whole JTSA project relies on a formal model of the data types and of the algorithms included in the library. This model is the basis for the design and implementation of the software application. Taking into account this formalized structure, the user can easily extend the JTSA framework by adding new algorithms. Results are shown in the context of the EU project MOSAIC to extract relevant patterns from data coming related to the long term monitoring of diabetic patients. Conclusions: The proof that JTSA is a versatile tool to be adapted to different needs is given by its possible uses, both as a standalone tool for data summarization and as a module to be embedded into other architecture

关键词： Temporal abstractions Time series analysis data analysis workflow Temporal pattern discovery Biomedical data mining software tool

来源：评论

学校读者我要写书评

暂无评论

FLiPPR: A Processor for Limited Proteolysis (LiP) Mass Spectrometry data Sets Built on FragPipe

引用

JOURNAL OF PROTEOME RESEARCH 2024年第7期23卷 2332-2342页

作者： Manriquez-Sandoval, Edgar Brewer, Joy Lule, Gabriela Lopez, Samanta Fried, Stephen D. Johns Hopkins Univ Dept Chem Baltimore MD 21218 USA Johns Hopkins Univ TC Jenkins Dept Biophys Baltimore MD 21218 USA Old Dominion Univ Dept Chem & Biochem Norfolk VA 23529 USA

Here, we present FLiPPR, or FragPipe LiP (limited proteolysis) Processor, a tool that facilitates the analysis of data from limited proteolysis mass spectrometry (LiP-MS) experiments following primary search and quantification in FragPipe. LiP-MS has emerged as a method that can provide proteome-wide information on protein structure and has been applied to a range of biological and biophysical questions. Although LiP-MS can be carried out with standard laboratory reagents and mass spectrometers, analyzing the data can be slow and poses unique challenges compared to typical quantitative proteomics workflows. To address this, we leverage FragPipe and then process its output in FLiPPR. FLiPPR formalizes a specific data imputation heuristic that carefully uses missing data in LiP-MS experiments to report on the most significant structural changes. Moreover, FLiPPR introduces a data merging scheme and a protein-centric multiple hypothesis correction scheme, enabling processed LiP-MS data sets to be more robust and less redundant. These improvements strengthen statistical trends when previously published data are reanalyzed with the FragPipe/FLiPPR workflow. We hope that FLiPPR will lower the barrier for more users to adopt LiP-MS, standardize statistical procedures for LiP-MS data analysis, and systematize output to facilitate eventual larger-scale integration of LiP-MS data.

关键词： limited proteolysis structuralproteomics computational tools data analysis workflow bioinformatics

来源：评论

学校读者我要写书评

暂无评论

Multi-agent System for Mass Spectrometry analysis 8th

引用

8th International Conference on Practical Applications of Agents and Multi-Agent Systems

作者： Reboiro-Jato, Miguel Glez-Pena, Daniel Santos, Hugo M. Diniz, Mario S. Lodeiro, Carlos Capelo, Jose L. Fdez-Riverola, Florentino Univ Vigo Dept Informat SING Grp Ed PolitecnCampus Univ Lagoas S-N Orense 32004 Spain Univ Nova Lisboa Fac Ciencias & Tecnol FCT REQUIMTE Dept Quim P-2829516 Lisbon Portugal Univ Vigo Fac Sci Dept Phys Chem BIOSCOPE Grp Orense Spain Univ Vigo Fac Sci Analytical & Food Chem Dept BIOSCOPE GrpNutrit & Bromatol Area Orense Spain

ISBN: (纸本)9783642124327

This paper presents a framework for rapid distributed application development in mass spectrometry data analysis. The proposed system includes an extensible and flexible workflow that covers the whole data analysis process. This workflow was designed to be easily distributed and parallelized by implementing the underlying framework as a multi-agent system.

关键词： mass spectrometry framework data analysis workflow multi-agent system

来源：评论

学校读者我要写书评

暂无评论

datawindow: Python Library to Simplify the data Science Process 5

Datawindow: Python Library to Simplify the Data Science Proc...

引用

5th IEEE International Conference on Computing, Power, and Communication Technologies, IC2PCT 2024

作者： Lawrence, Daniel Anu Barathi, B.U. Sreeji, S. Sathyabama Institute of Science & Technology Department of Computer Science & Engineering Chennai India

ISBN: (纸本)9798350383522

This paper is a documentation for datawindow, a Python package which has the goal of simplifying the data processing and analysis techniques. datawindow provides eight methods which allows users to quickly and easily handle activities in the data science process such as model building, data cleaning and summarization using user-friendly interfaces. The datawindow keeps user-friendliness and modularity as the main focus in the library's design. datawindow provides the user-friendly experience by utilizing Tkinter for graphical user interfaces. datawindow users Pandas for data processing and Tkinter for the interfaces to help provide clarity to the users. This ensures that the library satisfies user needs and aims to provide the user with a streamlined data science experience. datawindow seeks to refocus the focus of the data analysis and data science process from coding and programming to allowing data workers to focus more on the creative processes and analytical thinking part of the analysis using interfaces. © 2024 IEEE.

关键词： cluster analysis data analysis workflow data cleaning data processing data summarization deep learning exploratory data analysis machine learning modular design numpy pandas python library tkinter user interface

来源：评论

学校读者我要写书评

暂无评论

Bluesky and Raydata: An Integrated Platform for Adaptive Experiment Orchestration

Bluesky and Raydata: An Integrated Platform for Adaptive Exp...

引用

2024 Workshops of the International Conference for High Performance Computing, Networking, Storage and analysis, SC Workshops 2024

作者： Wijesinghe, Hiran Barbour, Andi Wiegart, Lutz Carlin, Evan Einstein-Curtis, Joshua Moeller, Paul Nagler, Rob O'Rourke, Raven Cook, Nathan Rakitin, Max National Synchrotron Light Source II Brookhaven National Laboratory UptonNY United States RadiaSoft LLC BoulderCO United States

ISBN: (纸本)9798350355543

Synchrotron light sources have emerged as vibrant hubs of scientific collaboration, fueling breakthroughs across multiple disciplines. As domain scientists increasingly push the boundaries of their research, the demand for customized beam-lines has grown in response. Recent successes in experiment orchestration and data management solutions such as Bluesky are being adopted widely, yet a significant challenge remains: generalizing support for data analysis workflows to accommodate the unique needs of diverse user communities. Addressing this hurdle could unlock a new era of experimental innovation, enabling the development of sophisticated techniques that are adaptive, automated, and tailored to specific research domains. This work presents a strategic approach to integrating experiment orchestration and data analysis, aimed at empowering scientists to realize their most ambitious goals while maximizing the potential of modern light sources. © 2024 IEEE.

关键词： Bluesky data analysis workflow light sources Raydata

来源：评论

学校读者我要写书评

暂无评论

Detection and identification of authorized and unauthorized GMOs using high-throughput sequencing with the support of a sequence-based GMO database

引用

FOOD CHEMISTRY: MOLECULAR SCIENCES 2022年 4卷 100096页

作者： Saltykova, Assia Van Braekel, Julien Papazova, Nina Fraiture, Marie-Alice Deforce, Dieter Vanneste, Kevin De Keersmaecker, Sigrid C. J. Roosens, Nancy H. Sciensano Transversal Act Appl Genom Brussels Belgium Univ Ghent Dept Plant Biotechnol & Bioinformat Ghent Belgium Univ Ghent Fac Pharmaceut Sci Lab Pharmaceut Biotechnol Ghent Belgium

The increasing number and diversity of genetically modified organisms (GMOs) for the food and feed market calls for the development of advanced methods for their detection and identification. This issue can be addressed by next generation sequencing (NGS). However, the efficiency of NGS-based strategies depends on the availability of bioinformatic methods to find sequences of the transgenic insert and junction regions, which is a challenging topic. To facilitate this task, we have developed Nexplorer, a sequence-based database in which annotated sequences of GM events are stored in a structured, searchable and extractable format. As a proof of concept, we have developed a methodology for the analysis of sequencing data of DNA walking libraries of samples containing GMOs using the database. The efficiency of the method has been tested on datasets representing various scenarios that can be encountered in routine GMO analysis. database-guided analysis allowed obtaining detailed and reliable information with limited hands-on time. As the database allows for efficient analysis of NGS data, it paves the way for the use of NGS sequencing technology to aid routine detection and identification of GMO.

关键词： GMO detection GMO identification1 NGS GMO database data analysis workflow

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：