检索结果-内蒙古大学图书馆

computational reproducibility of Jupyter notebooks from biomedical publications

GIGASCIENCE 2024年第13期13卷 giad113页

作者： Samuel, Sheeba Mietchen, Daniel Friedrich Schiller Univ Jena Heinz Nixdorf Chair Distributed Informat Syst D-07743 Jena Germany Michael Stifel Ctr Jena D-07743 Jena Germany Ronin Inst Montclair NJ 07043 USA Inst Globally Distributed Open Res & Educ IGDORE Rewari India FIZ Karlsruhe Leibniz Inst Informat Infrastruct D-76344 Berlin Germany

Background Jupyter notebooks facilitate the bundling of executable code with its documentation and output in one interactive environment, and they represent a popular mechanism to document and share computational workflows, including for research publications. The reproducibility of computational aspects of research is a key component of scientific reproducibility but has not yet been assessed at scale for Jupyter notebooks associated with biomedical *** We address computational reproducibility at 2 levels: (i) using fully automated workflows, we analyzed the computational reproducibility of Jupyter notebooks associated with publications indexed in the biomedical literature repository PubMed Central. We identified such notebooks by mining the article's full text, trying to locate them on GitHub, and attempting to rerun them in an environment as close to the original as possible. We documented reproduction success and exceptions and explored relationships between notebook reproducibility and variables related to the notebooks or publications. (ii) This study represents a reproducibility attempt in and of itself, using essentially the same methodology twice on PubMed Central over the course of 2 years, during which the corpus of Jupyter notebooks from articles indexed in PubMed Central has grown in a highly dynamic *** Out of 27,271 Jupyter notebooks from 2,660 GitHub repositories associated with 3,467 publications, 22,578 notebooks were written in Python, including 15,817 that had their dependencies declared in standard requirement files and that we attempted to rerun automatically. For 10,388 of these, all declared dependencies could be installed successfully, and we reran them to assess reproducibility. Of these, 1,203 notebooks ran through without any errors, including 879 that produced results identical to those reported in the original notebook and 324 for which our results differed from the originally reported ones. Running the oth

关键词： computational reproducibility Jupyter notebooks PubMed Central GitHub dependency decay Python workflow documentation

来源：评论

学校读者我要写书评

暂无评论

computational reproducibility Within Prognostics and Health Management

引用

Journal of Dynamics, Monitoring and Diagnostics 2023年第1期2卷 52-60页

作者： Tim von Hahn Chris K.Mechefske Department of Mechanical and Materials Engineering Queen’s UniversityKingstonCanada

Scientific research frequently involves the use of computational tools and *** thorough documentation,open-source code,and data–the creation of reproducible computational research(RCR)–helps others understand a researcher’s *** this study,we investigate the state of reproducible computational research,broadly,and from within the field of prognostics and health management(PHM).In a text mining survey of more than 300 articles,we show that fewer than 1%of PHM researchers make their code and data available to *** promote the RCR further,our work also highlights several personal benefits for those engaged in the ***,we introduce an open-source software tool,called PyPHM,to assist PHM researchers in accessing and preprocessing common industrial datasets.

关键词： computational reproducibility open-source prognostics and health management

来源：评论

学校读者我要写书评

暂无评论

A framework for assessing the computational reproducibility of geo-simulation experiments

引用

ENVIRONMENTAL MODELLING & SOFTWARE 2025年 186卷

作者： Zhu, Zhiyi Chen, Min Ren, Guangjin He, Yuanqing Sun, Lingzhi Zhang, Fengyuan Wen, Yongning Yue, Songshan Lu, Guonian Nanjing Normal Univ Key Lab Virtual Geog Environm Minist Educ Nanjing 210023 Jiangsu Peoples R China Nanjing Normal Univ Jiangsu Ctr Collaborat Innovat Geog Informat Resou Nanjing Jiangsu Peoples R China Nanjing Normal Univ State Key Lab Cultivat Base Geog Environm Evolut J Nanjing Jiangsu Peoples R China Minist Nat Resource Geoinformat Mapping Inst 3 Chengdu Sichuan Peoples R China Nanjing Normal Univ Sch Environm Nanjing Peoples R China

Recent advances in computational technologies have enhanced geo-simulation experiments (GSEs), making computational reproducibility assessments increasingly critical. However, existing methods often focus on isolated aspects, lacking a comprehensive framework. This study proposes an integrated framework for assessing reproducibility in GSEs, structured into two parts: (1) evaluating overall computational workflows, and (2) investigating individual processes to identify inconsistencies. The framework employs a detailed assessment model using hierarchical dimensions and metrics that combine quantitative measures (e.g., output consistency) and qualitative evaluations (e.g., clarity of descriptions). These components address both broad and granular aspects of computational processes. The framework is implemented in a prototype system to support reproducibility assessments and demonstrated through practical applications. This systematic approach provides a robust and adaptable method for assessing reproducibility, promoting the resolution of challenges in existing methods.

关键词： computational reproducibility Geo-simulation experiment Assessment framework

来源：评论

学校读者我要写书评

暂无评论

HPC-driven computational reproducibility in numerical relativity codes: a use case study with IllinoisGRMHD

引用

CLASSICAL AND QUANTUM GRAVITY 2024年第2期41卷 025002-025002页

作者： Luo, Yufeng Zhang, Qian Haas, Roland Etienne, Zachariah B. Allen, Gabrielle Univ Wyoming Dept Phys & Astron Laramie WY 82071 USA Univ Wyoming Sch Comp Laramie WY 82071 USA Univ Illinois Dept Phys Urbana IL 61801 USA Univ Illinois NCSA Urbana IL 61801 USA Digital Res Alliance Canada Strategy & Planning Toronto ON M4S3C6 Canada Univ Illinois Dept Astron Urbana IL 61801 USA Univ Idaho Dept Phys Moscow ID 83844 USA

reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produce consistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using the IllinoisGRMHD code that is part of an open-source community software for simulation in relativistic astrophysics, the Einstein Toolkit. We verify that numerical results of simulating a single isolated neutron star with IllinoisGRMHD can be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find that IllinoisGRMHD reproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version of IllinoisGRMHD reproduces these results once we account for bug fixes which have occurred since the original publication.

关键词： high-performance computing computational reproducibility numerical relativity

来源：评论

学校读者我要写书评

暂无评论

Reproscreener: Leveraging LLMs for Assessing computational reproducibility of Machine Learning Pipelines 2

Reproscreener: Leveraging LLMs for Assessing Computational R...

引用

2nd ACM Conference on reproducibility and Replicability (ACM REP)

作者： Bhaskar, Adhithya Stodden, Victoria Univ Southern Calif Los Angeles CA 90007 USA

ISBN: (纸本)9798400705304

The increasing reliance on machine learning models in scientific research and day-to-day applications - and the near-opacity of their associated computational methods - creates a widely recognized need to enable others to verify results coming from Machine Learning Pipelines. In this work we use an empirical approach to build on efforts to define and deploy structured publication standards that allow machine learning research to be automatically assessed and verified, enabling greater reliability and trust in results. To automate the assessment of a set of publication standards for Machine Learning Pipelines we developed Reproscreener;a novel, open-source software tool (see https://***/). We benchmark Reproscreener's automatic reproducibility assessment against a novel manually labeled "gold standard" dataset of machine learning arXiv preprints. Our empirical evaluation has a dual goal: to assess Reproscreener's performance;and to uncover gaps and opportunities in current reproducibility standards. We develop reproducibility assessment metrics we called the Repo Metrics to provide a novel overall assessment of the re-executability potential of the Machine Learning Pipeline, called the ReproScore. We used two approaches to the automatic identification of reproducibility metrics, keywords and LLM tools, and found the reproducibility metric evaluation performance of Large Language Model (LLM) tools superior to keyword associations.

关键词： computational reproducibility Open Code Open Data reproducibility Policy Machine Learning CyberInfrastructure Reproscreener ReproScore

来源：评论

学校读者我要写书评

暂无评论

Epistemic issues in computational reproducibility: software as the elephant in the room

引用

EUROPEAN JOURNAL FOR PHILOSOPHY OF SCIENCE 2021年第2期11卷 1-20页

作者： Hocquet, Alexandre Wieber, Frederic CNRS UMR 7117 Arch Poincare 91 Ave Liberat F-54001 Nancy France Univ Lorraine 91 Ave Liberat F-54001 Nancy France

computational reproducibility (i.e. issues of reproducibility stemming from the computer as a scientific tool) possesses its own dynamics and narratives of crisis. Alongside the difficulties of computing as an ubiquitous yet complex scientific activity, computational reproducibility suffers from a naive expectancy of total reproducibility and a moral imperative to embrace the principles of free software as a non-negotiable epistemic virtue. We argue that the epistemic issues at stake in actual practices of computational reproducibility are best unveiled by focusing on software as a pivotal concept, one that is surprisingly often overlooked in accounts of reproducibility issues. Software is not only about designing and coding but also about maintaining, supporting, distributing, licensing, and governance;it is not only about developers but also about users. We focus on openness debates among computational chemists involved in molecular modeling software packages as empirical grounding for our argument. We then identify and analyse four epistemic characteristics (transparency, consistency, sustainability and inclusivity) as key to the role of software in computational reproducibility.

关键词： computational reproducibility Software computational chemistry Transparency Consistency Sustainability Inclusivity

来源：评论

学校读者我要写书评

暂无评论

computational reproducibility of scientific workflows at extreme scales

引用

INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS 2019年第5期33卷 763-776页

作者： Pouchard, Line Baldwin, Sterling Elsethagen, Todd Jha, Shantenu Raju, Bibi Stephan, Eric Tang, Li Van Dam, Kerstin Kleese Brookhaven Natl Lab Ctr Data Driven Discovery Computat Sci Initiat Upton NY 11973 USA Brookhaven Natl Lab Computat Sci Initiat Upton NY 11973 USA Lawrence Livermore Natl Lab Livermore CA USA Pacific Northwest Natl Lab Richland WA 99352 USA Los Alamos Natl Lab Los Alamos NM USA

We propose an approach for improved reproducibility that includes capturing and relating provenance characteristics and performance metrics. We discuss two use cases: scientific reproducibility of results in the Energy Exascale Earth System Model (E3SM-previously ACME) and performance reproducibility in molecular dynamics workflows on HPC platforms. To capture and persist the provenance and performance data of these workflows, we have designed and developed the Chimbuko and ProvEn frameworks. Chimbuko captures provenance and enables detailed single workflow performance analysis. ProvEn is a hybrid, queryable system for storing and analyzing the provenance and performance metrics of multiple runs in workflow performance analysis campaigns. Workflow provenance and performance data output from Chimbuko can be visualized in a dynamic, multilevel visualization providing overview and zoom-in capabilities for areas of interest. Provenance and related performance data ingested into ProvEn is queryable and can be used to reproduce runs. Our provenance-based approach highlights challenges in extracting information and gaps in the information collected. It is agnostic to the type of provenance data it captures so that both the reproducibility of scientific results and that of performance can be explored with our tools.

关键词： computational reproducibility scientific workflows provenance performance analysis ProvEn Chimbuko

来源：评论

学校读者我要写书评

暂无评论

Containers and orchestration of numerical ocean model for computational reproducibility and portability in public and private clouds: Application of ROMS 3.6

引用

SIMULATION MODELLING PRACTICE AND THEORY 2021年 109卷 102305-102305页

作者： Jung, Kwangwoog Cho, Yang-Ki Tak, Yong-Jin Seoul Natl Univ Sch Earth & Environm Sci Seoul South Korea Seoul Natl Univ Sch Earth & Environm Sci Res Inst Oceanog Seoul South Korea Yonsei Univ Inst Earth Atmosphere Astron 50 Yonsei Ro Seoul South Korea

Various numerical models have been used to understand and predict ocean dynamics. For this reason, many information technology (IT) resources are required for high-resolution global ocean modeling. The development of cloud-computing technologies has enabled earth scientists to easily use numerical ocean models that require high-performance computing (HPC) and messagepassing interface (MPI) software in private and public clouds. Although it is easier today to use computing resources than it was in the past, computational reproducibility and portability in diverse IT environments remain crucial issues. This study proposes a model execution architecture for computational reproducibility, portability, and agility based on container-based virtualization and orchestration technologies. We implement a containerized regional ocean-modeling system (ROMS), an MPI-based numerical ocean model that exists in various public or private cloud environments (e.g., personal computers and multiple-node servers). The preparation time for model setup is greatly reduced using our container-based HPC architecture. Containerization of ROMS is tested for its support of the portability of numerical modeling in a wide range of public-cloud environments. When leveraging an abstraction layer of complex and diverse infrastructure environments, we can run the ocean model more easily while obtaining computational reproducibility using a shareable deployment code. This advancement can be used to guide the containerization of various numerical models and to run them in parallel in public and private cloud-computing environments.

关键词： Numerical ocean model computational reproducibility High-performance computing Cloud computing Infra abstraction Containerization Docker Regional Ocean-modeling System Kubernetes Public and Private Cloud

来源：评论

学校读者我要写书评

暂无评论

Advancing computational reproducibility in the Dataverse Data Repository Platform 3

Advancing Computational Reproducibility in the Dataverse Dat...

引用

3rd International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS)

作者： Trisovic, Ana Durbin, Philip Schlatter, Tania Durand, Gustavo Barbosa, Sonia Brooke, Danny Crosas, Merce Harvard Univ Inst Quantitat Social Sci Cambridge MA 02138 USA

ISBN: (纸本)9781450379779

Recent reproducibility case studies have raised concerns showing that much of the deposited research has not been reproducible. One of their conclusions was that the way data repositories store research data and code cannot fully facilitate reproducibility due to the absence of a runtime environment needed for the code execution. New specialized reproducibility tools provide cloud-based computational environments for code encapsulation, thus enabling research portability and reproducibility. However, they do not often enable research discoverability, standardized data citation, or long-term archival like data repositories do. This paper addresses the shortcomings of data repositories and reproducibility tools and how they could be overcome to improve the current lack of computational reproducibility in published and archived research outputs.

关键词： computational reproducibility open data open code data repository data management data preservation

来源：评论

学校读者我要写书评

暂无评论

computational reproducibility of "Goal relevance and goal conduciveness appraisals lead to differential autonomic reactivity in emotional responding to performance feedback" (Kreibig, Gendolla, & Scherer, 2012): A guide and new evidence

引用

INTERNATIONAL JOURNAL OF PSYCHOPHYSIOLOGY 2017年第0期119卷 93-107页

作者： Kreibig, Sylvia D. Stanford Univ Dept Psychol Stanford CA 94305 USA

The emerging field of the psychophysiology of motivation bears many new findings, but little replication. Using my own data (Kreibig, Gendolla, & Scherer, 2012), I test the reproducibility of this specific study, provide the necessary materials to make the study reproducible, and instantiate proper reproducibility practices that other researchers can use as a road map toward the same goal. In addition, based on re-analyses of the original data, I report new evidence for the motivational effects of emotional responding to performance feedback. Specifically, greater appraisal of goal relevance amplifies the emotional response to events appraised as conducive (i.e., effort mobilization), but not to those appraised as obstructive to a person's goals (i.e., effort withdrawal). I conclude by providing a ten-step road map of best practices to facilitate computational reproducibility for future studies.

关键词： computational reproducibility Replication Psychophysiology Motivation Emotion Appraisal

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：