检索结果-内蒙古大学图书馆

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results

arXiv

引用

arXiv 2021年

作者： Mehta, Raghav Filos, Angelos Baid, Ujjwal Sako, Chiharu McKinley, Richard Rebsamen, Michael Dätwyler, Katrin Meier, Raphael Radojewski, Piotr Murugesan, Gowtham Krishnan Nalawade, Sahil Ganesh, Chandan Wagner, Ben Yu, Fang F. Fei, Baowei Madhuranthakam, Ananth J. Maldjian, Joseph A. Daza, Laura Gómez, Catalina Arbeláez, Pablo Dai, Chengliang Wang, Shuo Reynaud, Hadrien Mo, Yuanhan Angelini, Elsa Guo, Yike Bai, Wenjia Banerjee, Subhashis Pei, Linmin Murat, A.K. Rosas-González, Sarahi Zemmoura, Ilyess Tauber, Clovis Vu, Minh H. Nyholm, Tufve Löfstedt, Tommy Ballestar, Laura Mora Vilaplana, Veronica McHugh, Hugh Talou, Gonzalo Maso Wang, Alan Patel, Jay Chang, Ken Hoebel, Katharina Gidwani, Mishka Arun, Nishanth Gupta, Sharut Aggarwal, Mehak Singh, Praveer Gerstner, Elizabeth R. Kalpathy-Cramer, Jayashree Boutry, Nicolas Huard, Alexis Vidyaratne, Lasitha Rahman, Md Monibor Iftekharuddin, Khan M. Chazalon, Joseph Puybareau, Elodie Tochon, Guillaume Ma, Jun Cabezas, Mariano Llado, Xavier Oliver, Arnau Valencia, Liliana Valverde, Sergi Amian, Mehdi Soltaninejad, Mohammadreza Myronenko, Andriy Hatamizadeh, Ali Feng, Xue Dou, Quan Tustison, Nicholas Meyer, Craig Shah, Nisarg A. Talbar, Sanjay Weber, Marc-André Mahajan, Abhishek Jakab, Andras Wiest, Roland Fathallah-Shaykh, Hassan M. Nazeri, Arash Milchenko, Mikhail Marcus, Daniel Kotrotsou, Aikaterini Colen, Rivka Freymann, John Kirby, Justin Davatzikos, Christos Menze, Bjoern Bakas, Spyridon Gal, Yarin Arbel, Tal McGill University MontrealQC Canada Group University of Oxford Oxford United Kingdom University of Pennsylvania PhiladelphiaPA United States Department of Radiology Perelman School of Medicine The University of Pennsylvania PhiladelphiaPA United States Department of Pathology and Laboratory Medicine Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States University Institute of Diagnostic and Interventional Neuroradiology University of Bern Inselspital Bern University Hospital Bern Switzerland Department of Radiology University of Texas Southwestern Medical Center DallasTX United States Department of Bioengineering University of Texas DallasTX United States Advanced Imaging Research Center University of Texas Southwestern Medical Center DallasTX United States Universidad de los Andes Bogotá Colombia Data Science Institute Imperial College London London United Kingdom NIHR Imperial BRC ITMAT Data Science Group Imperial College London London United Kingdom Department of Brain Sciences Imperial College London London United Kingdom Machine Intelligence Unit Indian Statistical Institute Kolkata India Department of CSE University of Calcutta Kolkata India Department of Information Technology Uppsala University Uppsala Sweden Department of Diagnostic Radiology The University of Pittsburgh Medical Center PittsburghPA United States UMR U1253 iBrain Université de Tours Inserm Tours France Department of Radiation Sciences Umeå University Umeå Sweden Department of Computing Science Umeå University Umeå Sweden Signal Theory and Communications Department Universitat Politècnica de Catalunya Barcelona Tech Barcelona Spain Faculty of Medical and Health Sciences University of Auckland Auckland New Zealand Radiology Department Auckland City Hospital Auckland New Zealand Auckland Bioengineering Institute University of Auckland New Zealand Athinoula A. Martinos Center for Biomedical Imaging Department of Radiology

Deep learning (DL) models have provided state-of-the-art performance in various medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder translating DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties could enable clinical review of the most uncertain regions, thereby building trust and paving the way toward clinical translation. Several uncertainty estimation methods have recently been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentage of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, highlighting the need for uncertainty quantification in medical image analyses. Finally, in favor of transparency and reproducibility, our evaluation code is made publicly available at https://***/RagMeh11/QU-BraTS. © 2021, CC BY.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

Why is the Winner the Best?

Why is the Winner the Best?

引用

Conference on computer vision and Pattern Recognition (CVPR)

作者： M. Eisenmann A. Reinke V. Weru M. D. Tizabi F. Isensee T. J. Adler S. Ali V. Andrearczyk M. Aubreville U. Baid S. Bakas N. Balu S. Bano J. Bernal S. Bodenstedt A. Casella V. Cheplygina M. Daum M. De Bruijne A. Depeursinge R. Dorent J. Egger D. G. Ellis S. Engelhardt M. Ganz N. Ghatwary G. Girard P. Godau A. Gupta L. Hansen K. Harada M. Heinrich N. Heller A. Hering A. Huaulmé P. Jannin A. E. Kavur O. Kodym M. Kozubek J. Li H. Li J. Ma C. Martín-Isla B. Menze A. Noble V. Oreiller N. Padoy S. Pati K. Payette T. Rädsch J. Rafael-Patiño V. Singh Bawa S. Speidel C. H. Sudre K. Van Wijnen M. Wagner D. Wei A. Yamlahi M. H. Yap C. Yuan M. Zenk A. Zia D. Zimmerer D. Aydogan B. Bhattarai L. Bloch R. Brüngel J. Cho C. Choi Q. Dou I. Ezhov C. M. Friedrich C. Fuller R. R. Gaire A. Galdran Á. García Faura M. Grammatikopoulou S. Hong M. Jahanifar I. Jang A. Kadkhodamohammadi I. Kang F. Kofler S. Kondo H. Kuijf M. Li M. Luu T. Martinčič P. Morais M. A. Naser B. Oliveira D. Owen S. Pang J. Park S. Park S. Płotka E. Puybareau N. Rajpoot K. Ryu N. Saeed A. Shephard P. Shi D. Štepec R. Subedi G. Tochon H. R. Torres H. Urien J. L. Vilaça K. A. Wahid H. Wang J. Wang L. Wang X. Wang B. Wiestler M. Wodzinski F. Xia J. Xie Z. Xiong S. Yang Y. Yang Z. Zhao K. Maier-Hein P. F. Jäger A. Kopp-Schneider L. Maier-Hein Division of Intelligent Medical Systems German Cancer Research Center (DKFZ) Heidelberg Germany Helmholtz Imaging German Cancer Research Center (DKFZ) Heidelberg Germany Faculty of Mathematics and Computer Science Heidelberg University Heidelberg Germany Division of Biostatistics German Cancer Research Center (DKFZ) Heidelberg Germany Division of Medical Image Computing German Cancer Research Center (DKFZ) Heidelberg Germany Faculty of Engineering and Physical Sciences School of Computing University of Leeds Leeds UK Institute of Informatics School of Management HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland Sierre Switzerland Department of Nuclear Medicine and Molecular Imaging Lausanne University Hospital Lausanne Switzerland Technische Hochschule Ingolstadt Ingolstadt Germany Center for Artificial Intelligence and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA) University of Pennsylvania Philadelphia PA USA Department of Pathology and Laboratory Medicine Perelman School of Medicine University of Pennsylvania Philadelphia PA USA Department of Radiology Perelman School of Medicine University of Pennsylvania Philadelphia PA USA Department of Radiology University of Washington Seattle WA USA Department of Computer Science Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) University College London London UK Universitat Autònoma de Barcelona & Computer Vision Center Barcelona Spain Division of Translational Surgical Oncology National Center for Tumor Diseases (NCT/UCC) Dresden Dresden Germany Department of Advanced Robotics Istituto Italiano di Tecnologia Italy Department of Electronics Information and Bioengineering Politecnico di Milano Milan Italy IT University of Copenhagen Copenhagen Denmark Department of General Visceral and Transplantation Surgery Heidelberg University Hospital Heidelberg Germany Department of Radiology and Nuc

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.

关键词：

来源：评论

学校读者我要写书评

暂无评论

26th Annual Computational Neuroscience Meeting (CNS*2017): Part 3 Antwerp, Belgium. 15-20 July 2017 Abstracts

引用

BMC NEUROSCIENCE 2017年第SUPPL 1期18卷 95-176页

作者： [Anonymous] Department of Neuroscience Yale University New Haven CT 06520 USA Department Physiology & Pharmacology SUNY Downstate Brooklyn NY 11203 USA NYU School of Engineering 6 MetroTech Center Brooklyn NY 11201 USA Departament de Matemàtica Aplicada Universitat Politècnica de Catalunya Barcelona 08028 Spain Institut de Neurobiologie de la Méditerrannée (INMED) INSERM UMR901 Aix-Marseille Univ Marseille France Center of Neural Science New York University New York NY USA Aix-Marseille Univ INSERM INS Inst Neurosci Syst Marseille France Laboratoire de Physique Théorique et Modélisation CNRS UMR 8089 Université de Cergy-Pontoise 95300 Cergy-Pontoise Cedex France Department of Mathematics and Computer Science ENSAT Abdelmalek Essaadi’s University Tangier Morocco Laboratory of Natural Computation Department of Information and Electrical Engineering and Applied Mathematics University of Salerno 84084 Fisciano SA Italy Department of Medicine University of Salerno 84083 Lancusi SA Italy Dipartimento di Fisica Università degli Studi Aldo Moro Bari and INFN Sezione Di Bari Italy Data Analysis Department Ghent University Ghent Belgium Coma Science Group University of Liège Liège Belgium Cruces Hospital and Ikerbasque Research Center Bilbao Spain BIOtech Department of Industrial Engineering University of Trento and IRCS-PAT FBK 38010 Trento Italy Department of Data Analysis Ghent University Ghent 9000 Belgium The Wellcome Trust Centre for Neuroimaging University College London London WC1N 3BG UK Department of Electronic Engineering NED University of Engineering and Technology Karachi Pakistan Blue Brain Project École Polytechnique Fédérale de Lausanne Lausanne Switzerland Departement of Mathematics Swansea University Swansea Wales UK Laboratory for Topology and Neuroscience at the Brain Mind Institute École polytechnique fédérale de Lausanne Lausanne Switzerland Institute of Mathematics University of Aberdeen Aberdeen Scotland UK Department of Integrativ

来源：评论

学校读者我要写书评

暂无评论

Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge

arXiv

引用

arXiv 2018年

作者： Bakas, Spyridon Reyes, Mauricio Jakab, Andras Bauer, Stefan Rempfler, Markus Crimi, Alessandro Shinohara, Russell Takeshi Berger, Christoph Ha, Sung Min Rozycki, Martin Prastawa, Marcel Alberts, Esther Lipkova, Jana Freymann, John Kirby, Justin Bilello, Michel Fathallah-Shaykh, Hassan M. Wiest, Roland Kirschke, Jan Wiestler, Benedikt Colen, Rivka Kotrotsou, Aikaterini Lamontagne, Pamela Marcus, Daniel Milchenko, Mikhail Nazeri, Arash Weber, Marc-Andr Mahajan, Abhishek Baid, Ujjwal Gerstner, Elizabeth Kwon, Dongjin Acharya, Gagan Agarwal, Manu Alam, Mahbubul Albiol, Alberto Albiol, Antonio Albiol, Francisco J. Alex, Varghese Allinson, Nigel Amorim, Pedro H.A. Amrutkar, Abhijit Anand, Ganesh Andermatt, Simon Arbel, Tal Arbelaez, Pablo Avery, Aaron Azmat, Muneeza Pranjal, B. Bai, Wenjia Banerjee, Subhashis Barth, Bill Batchelder, Thomas Batmanghelich, Kayhan Battistella, Enzo Beers, Andrew Belyaev, Mikhail Bendszus, Martin Benson, Eze Bernal, Jose Bharath, Halandur Nagaraja Biros, George Bisdas, Sotirios Brown, James Cabezas, Mariano Cao, Shilei Cardoso, Jorge M. Carver, Eric N. Casamitjana, Adri Castillo, Laura Silvana Cat, Marcel Cattin, Philippe Cérigues, Albert Chagas, Vinicius S. Chandra, Siddhartha Chang, Yi-Ju Chang, Shiyu Chang, Ken Chazalon, Joseph Chen, Shengcong Chen, Wei Chen, Jefferson W. Chen, Zhaolin Cheng, Kun Choudhury, Ahana Roy Chylla, Roger Clrigues, Albert Colleman, Steven Colmeiro, Ramiro German Rodriguez Combalia, Marc Costa, Anthony Cui, Xiaomeng Dai, Zhenzhen Dai, Lutao Daza, Laura Alexandra Deutsch, Eric Ding, Changxing Dong, Chao Dong, Shidu Dudzik, Wojciech Eaton-Rosen, Zach Egan, Gary Escudero, Guilherme Estienne, Tho Everson, Richard Fabrizio, Jonathan Fan, Yong Fang, Longwei Feng, Xue Ferrante, Enzo Fidon, Lucas Fischer, Martin French, Andrew P. Fridman, Naomi Fu, Huan Fuentes, David Gao, Yaozong Gates, Evan Gering, David Gholami, Amir Gierke, Willi Glocker, Ben Gong, Mingming Gonzlez-Vill, Sandra Grosges, T. Guan, Yuanfang Guo, Sheng Gupta, Sudeep Han, Woo-Sup Han, Il Song Harmuth, Ko Center for Biomedical Image Computing and Analytics University of Pennsylvania PhiladelphiaPA United States Department of Radiology Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Department of Pathology and Laboratory Medicine Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Institute for Surgical Technology and Biomechanics University of Bern Bern Switzerland Center for MR-Research University Children's Hospital Zurich Zurich Switzerland Support Centre for Advanced Neuroimaging Inselspital Institute for Diagnostic and Interventional Neuroradiology Bern University Hospital Bern Switzerland University Hospital of Zurich Zurich Switzerland Center for Clinical Epidemiology and Biostatistics University of Pennsylvania Philadelphia United States Image-Based Biomedical Modeling Group Technical University of Munich Munich Germany Icahn School of Medicine Mount Sinai Health System New YorkNY United States Leidos Biomedical Research Inc. Frederick National Laboratory for Cancer Research FrederickMD21701 United States Cancer Imaging Program National Cancer Institute National Institutes of Health BethesdaMD20814 United States Department of Neurology University of Alabama at Birmingham BirminghamAL United States Department of Diagnostic Radiology University of Texas MD Anderson Cancer Center HoustonTX United States Department of Psychology Washington University St. LouisMO United States Neuroimaging Informatics and Analysis Center Washington University St. LouisMO United States Department of Radiology Washington University St. LouisMO United States Institute of Diagnostic and Interventional Radiology Pediatric Radiology and Neuroradiology University Medical Center Rostock Ernst-Heydemann-Str. 6 Rostock18057 Germany Tata Memorial Centre Homi Bhabha National Institute Mumbai India Shri Guru Gobind Singhji Institute of Engineering and Technology Nanded India NVIDIA Santa Clara

Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset. Copyright © 2018, The Authors. All rights reserved.

关键词： Tumors

来源：评论

学校读者我要写书评

暂无评论

Understanding metric-related pitfalls in image analysis validation

arXiv

引用

arXiv 2023年

作者： Reinke, Annika Tizabi, Minu D. Baumgartner, Michael Eisenmann, Matthias Heckmann-Nötzel, Doreen Kavur, A. Emre Rädsch, Tim Sudre, Carole H. Acion, Laura Antonelli, Michela Arbel, Tal Bakas, Spyridon Benis, Arriel Blaschko, Matthew B. Buettner, Florian Cardoso, M. Jorge Cheplygina, Veronika Chen, Jianxu Christodoulou, Evangelia Cimini, Beth A. Collins, Gary S. Farahani, Keyvan Ferrer, Luciana Galdran, Adrian van Ginneken, Bram Glocker, Ben Godau, Patrick Haase, Robert Hashimoto, Daniel A. Hoffman, Michael M. Huisman, Merel Isensee, Fabian Jannin, Pierre Kahn, Charles E. Kainmueller, Dagmar Kainz, Bernhard Karargyris, Alexandros Karthikesalingam, Alan Kenngott, Hannes Kleesiek, Jens Kofler, Florian Kooi, Thijs Kopp-Schneider, Annette Kozubek, Michal Kreshuk, Anna Kurc, Tahsin Landman, Bennett A. Litjens, Geert Madani, Amin Maier-Hein, Klaus Martel, Anne L. Mattson, Peter Meijering, Erik Menze, Bjoern Moons, Karel G.M. Müller, Henning Nichyporuk, Brennan Nickel, Felix Petersen, Jens Rafelski, Susanne M. Rajpoot, Nasir Reyes, Mauricio Riegler, Michael A. Rieke, Nicola Saez-Rodriguez, Julio Sánchez, Clara I. Shetty, Shravya Summers, Ronald M. Taha, Abdel A. Tiulpin, Aleksei Tsaftaris, Sotirios A. van Calster, Ben Varoquaux, Gaël Yaniv, Ziv R. Jäger, Paul F. Maier-Hein, Lena Faculty of Mathematics and Computer Science Heidelberg University Heidelberg Germany Heidelberg Division of Intelligent Medical Systems Germany NCT Heidelberg A Partnership Between DKFZ University Medical Center Heidelberg Germany Heidelberg Division of Medical Image Computing Germany Heidelberg Division of Intelligent Medical Systems Germany MRC Unit for Lifelong Health and Ageing UCL Centre for Medical Image Computing Department of Computer Science University College London London United Kingdom School of Biomedical Engineering and Imaging Science King’s College London London United Kingdom Instituto de Cálculo CONICET – Universidad de Buenos Aires Buenos Aires Argentina Centre for Medical Image Computing University College London London United Kingdom McGill University Montreal Canada Division of Computational Pathology Dept of Pathology & Laboratory Medicine Indiana University School of Medicine IU Health Information and Translational Sciences Building Indianapolis United States University of Pennsylvania Richards Medical Research Laboratories FL7 PhiladelphiaPA United States Department of Digital Medical Technologies Holon Institute of Technology Holon Israel European Federation for Medical Informatics Le Mont-sur-Lausanne Switzerland Center for Processing Speech and Images Department of Electrical Engineering KU Leuven Leuven Belgium partner site Frankfurt/Mainz a partnership between DKFZ and UCT Frankfurt Marburg Germany Heidelberg Germany Goethe University Frankfurt Department of Medicine Germany Goethe University Frankfurt Department of Informatics Germany and Frankfurt Cancer Insititute Germany Department of Computer Science IT University of Copenhagen Copenhagen Denmark Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V. Dortmund Germany Imaging Platform Broad Institute of MIT and Harvard CambridgeMA United States Centre for Statistics in Medicine University of Oxford Oxford United Kingdom Center for Biomedical In

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately. While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. While focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation. © 2023, CC BY.

关键词： computer vision

来源：评论

学校读者我要写书评

暂无评论

Assessing Trustworthy AI in Times of COVID-19: Deep Learning for Predicting a Multiregional Score Conveying the Degree of Lung Compromise in COVID-19 Patients

IEEE Transactions on Technology and Society

引用

IEEE Transactions on Technology and Society 2022年第4期3卷 272-289页

作者： Allahabadi, Himanshi Amann, Julia Balot, Isabelle Beretta, Andrea Binkley, Charles Bozenhard, Jonas Bruneault, Frederick Brusseau, James Candemir, Sema Cappellini, Luca Alessandro Chakraborty, Subrata Cherciu, Nicoleta Cociancig, Christina Coffee, Megan Ek, Irene Espinosa-Leal, Leonardo Farina, Davide Fieux-Castagnet, Genevieve Frauenfelder, Thomas Gallucci, Alessio Giuliani, Guya Golda, Adam Van Halem, Irmhild Hildt, Elisabeth Holm, Sune Kararigas, Georgios Krier, Sebastien A. Kuhne, Ulrich Lizzi, Francesca Madai, Vince I. Markus, Aniek F. Masis, Serg Mathez, Emilie Wiinblad Mureddu, Francesco Neri, Emanuele Osika, Walter Ozols, Matiss Panigutti, Cecilia Parent, Brendan Pratesi, Francesca Moreno-Sanchez, Pedro A. Sartor, Giovanni Savardi, Mattia Signoroni, Alberto Sormunen, Hanna-Maria Spezzatti, Andy Srivastava, Adarsh Stephansen, Annette F. Theng, Lau Bee Tithi, Jesmin Jahan Tuominen, Jarno Umbrello, Steven Vaccher, Filippo Vetter, Dennis Westerlund, Magnus Wurth, Renee Zicari, Roberto V. Ey Netherlands Enterprise Intelligence Department Amsterdam1083 HP Netherlands Eth Zurich Health Ethics and Policy Lab Department of Health Sciences and Technology Zürich8092 Switzerland Center for Diplomatic and Strategic Studies Postgraduate Studies in Diplomacy and International Relations Paris75015 France Pisa56124 Italy Hackensack Meridian Health Bioethics Center EdisonNJ08820 United States University of Oxford Faculty of Philosophy OxfordOX2 6GG United Kingdom Collège André- Laurendeau Philosophie Department MontrealQCH8N 2J4 Canada Université du Québec À Montréal École des Médias MontrealQCH2L 2C4 Canada Pace University Philosophy Department New YorkNY10038 United States The Ohio State University Wexner Medical Center Department of Radiology ColumbusOH43210 United States Humanitas Research Hospital Department of Radiology Milan20089 Italy Humanitas University Department of Biomedical Sciences Milan20089 Italy University of New England Faculty of Science Agriculture Business and Law ArmidaleNSW2351 Australia University of Technology Sydney Faculty of Engineering and Information Technology SydneyNSW2007 Australia Scuola Superiore Sant'Anna European Centre of Excellence on the Regulation of Robotics and Ai Pisa56127 Italy University of Bremen Group of Computer Architecture Bremen28359 Germany New York University Grossman School of Medicine Division of Infectious Diseases and Immunology Department of Medicine New YorkNY10016 United States Digital Institute Ai Research Section Stockholm16731 Sweden Arcada University of Applied Sciences Department of Business Management and Analytics Helsinki00550 Finland University of Brescia Radiological Sciences and Public Health Department of Medical and Surgical Specialties Brescia25121 Italy Sncf Reseau Sa Ethique Groupe La Plaine93418 France Institute of Diagnostic and Interventional Radiology University Hospital Zurich Zürich8091 Switzerland Eindhoven University of Tech

This article's main contributions are twofold: 1) to demonstrate how to apply the general European Union's High-Level Expert group's (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does 'trustworthy AI' mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of pandemic. The AI system aims to help radiologists to estimate and communicate the severity of damage in a patient's lung from Chest X-rays. It has been experimentally deployed in the radiology department of the ASST Spedali Civili clinic in Brescia, Italy, since December 2020 during pandemic time. The methodology we have applied for our post-hoc assessment, called Z-Inspection®, uses sociotechnical scenarios to identify ethical, technical, and domain-specific issues in the use of the AI system in the context of the pandemic. © 2020 IEEE.

关键词： COVID-19

来源：评论

学校读者我要写书评

暂无评论

Why is the winner the best?

arXiv

引用

arXiv 2023年

作者： Eisenmann, M. Reinke, A. Weru, V. Tizabi, M.D. Isensee, F. Adler, T.J. Ali, S. Andrearczyk, V. Aubreville, M. Baid, U. Bakas, S. Balu, N. Bano, S. Bernal, J. Bodenstedt, S. Casella, A. Cheplygina, V. Daum, M. de Bruijne, M. Depeursinge, A. Dorent, R. Egger, J. Ellis, D.G. Engelhardt, S. Ganz, M. Ghatwary, N. Girard, G. Godau, P. Gupta, A. Hansen, L. Harada, K. Heinrich, M. Heller, N. Hering, A. Huaulmé, A. Jannin, P. Kavur, A.E. Kodym, O. Kozubek, M. Li, J. Li, H. Ma, J. Martín-Isla, C. Menze, B. Noble, A. Oreiller, V. Padoy, N. Pati, S. Payette, K. Rädsch, T. Rafael-Patiño, J. Bawa, V. Singh Speidel, S. Sudre, C.H. van Wijnen, K. Wagner, M. Wei, D. Yamlahi, A. Yap, M.H. Yuan, C. Zenk, M. Zia, A. Zimmerer, D. Aydogan, D. Bhattarai, B. Bloch, L. Brüngel, R. Cho, J. Choi, C. Dou, Q. Ezhov, I. Friedrich, C.M. Fuller, C. Gaire, R.R. Galdran, A. Faura, Á. García Grammatikopoulou, M. Hong, S. Jahanifar, M. Jang, I. Kadkhodamohammadi, A. Kang, I. Kofler, F. Kondo, S. Kuijf, H. Li, M. Luu, M. Martinčič, T. Morais, P. Naser, M.A. Oliveira, B. Owen, D. Pang, S. Park, J. Park, S. Plotka, S. Puybareau, E. Rajpoot, N. Ryu, K. Saeed, N. Shephard, A. Shi, P. Štepec, D. Subedi, R. Tochon, G. Torres, H.R. Urien, H. Vilaça, J.L. Wahid, K.A. Wang, H. Wang, J. Wang, L. Wang, X. Wiestler, B. Wodzinski, M. Xia, F. Xie, J. Xiong, Z. Yang, S. Yang, Y. Zhao, Z. Maier-Hein, K. Jäger, P.F. Kopp-Schneider, A. Maier-Hein, L. Heidelberg Germany Heidelberg Germany Faculty of Mathematics and Computer Science Heidelberg University Heidelberg Germany Heidelberg Germany Heidelberg Germany School of Computing Faculty of Engineering and Physical Sciences University of Leeds Leeds United Kingdom Institute of Informatics School of Management HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland Sierre Switzerland Department of Nuclear Medicine and Molecular Imaging Lausanne University Hospital Lausanne Switzerland Technische Hochschule Ingolstadt Ingolstadt Germany University of Pennsylvania PhiladelphiaPA United States Department of Pathology and Laboratory Medicine Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Department of Radiology Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Department of Radiology University of Washington SeattleWA United States Department of Computer Science University College London London United Kingdom Universitat Autònoma de Barcelona & Computer Vision Center Barcelona Spain Dresden Dresden Germany Department of Advanced Robotics Istituto Italiano di Tecnologia Italy IT University of Copenhagen Copenhagen Denmark Department of General Visceral and Transplantation Surgery Heidelberg University Hospital Heidelberg Germany Biomedical Imaging Group Rotterdam Department of Radiology and Nuclear Medicine Erasmus MC Rotterdam Netherlands Department of Computer Science University of Copenhagen Copenhagen Denmark Sierre Switzerland Harvard Medical School Brigham and Women’s Hospital BostonMA United States School of Biomedical Engineering and Imaging Sciences King’s College London London United Kingdom Essen Germany University of Nebraska Medical Center OmahaNE United States Department of Internal Medicine III Heidelberg University Hospital Heidelberg Germany Neurobiology Research Unit Copenhagen University Hospital Rigshospitalet C

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work. © 2023, CC BY.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：