检索结果-内蒙古大学图书馆

Assessing Trustworthy AI in Times of COVID-19: Deep Learning for Predicting a Multiregional Score Conveying the Degree of Lung Compromise in COVID-19 Patients

学校读者我要写书评

暂无评论

IEEE Transactions on Technology and Society

IEEE Transactions on Technology and Society 2022年第4期3卷 272-289页

作者： Allahabadi, Himanshi Amann, Julia Balot, Isabelle Beretta, Andrea Binkley, Charles Bozenhard, Jonas Bruneault, Frederick Brusseau, James Candemir, Sema Cappellini, Luca Alessandro Chakraborty, Subrata Cherciu, Nicoleta Cociancig, Christina Coffee, Megan Ek, Irene Espinosa-Leal, Leonardo Farina, Davide Fieux-Castagnet, Genevieve Frauenfelder, Thomas Gallucci, Alessio Giuliani, Guya Golda, Adam Van Halem, Irmhild Hildt, Elisabeth Holm, Sune Kararigas, Georgios Krier, Sebastien A. Kuhne, Ulrich Lizzi, Francesca Madai, Vince I. Markus, Aniek F. Masis, Serg Mathez, Emilie Wiinblad Mureddu, Francesco Neri, Emanuele Osika, Walter Ozols, Matiss Panigutti, Cecilia Parent, Brendan Pratesi, Francesca Moreno-Sanchez, Pedro A. Sartor, Giovanni Savardi, Mattia Signoroni, Alberto Sormunen, Hanna-Maria Spezzatti, Andy Srivastava, Adarsh Stephansen, Annette F. Theng, Lau Bee Tithi, Jesmin Jahan Tuominen, Jarno Umbrello, Steven Vaccher, Filippo Vetter, Dennis Westerlund, Magnus Wurth, Renee Zicari, Roberto V. Ey Netherlands Enterprise Intelligence Department Amsterdam1083 HP Netherlands Eth Zurich Health Ethics and Policy Lab Department of Health Sciences and Technology Zürich8092 Switzerland Center for Diplomatic and Strategic Studies Postgraduate Studies in Diplomacy and International Relations Paris75015 France Pisa56124 Italy Hackensack Meridian Health Bioethics Center EdisonNJ08820 United States University of Oxford Faculty of Philosophy OxfordOX2 6GG United Kingdom Collège André- Laurendeau Philosophie Department MontrealQCH8N 2J4 Canada Université du Québec À Montréal École des Médias MontrealQCH2L 2C4 Canada Pace University Philosophy Department New YorkNY10038 United States The Ohio State University Wexner Medical Center Department of Radiology ColumbusOH43210 United States Humanitas Research Hospital Department of Radiology Milan20089 Italy Humanitas University Department of Biomedical Sciences Milan20089 Italy University of New England Faculty of Science Agriculture Business and Law ArmidaleNSW2351 Australia University of Technology Sydney Faculty of Engineering and Information Technology SydneyNSW2007 Australia Scuola Superiore Sant'Anna European Centre of Excellence on the Regulation of Robotics and Ai Pisa56127 Italy University of Bremen Group of Computer Architecture Bremen28359 Germany New York University Grossman School of Medicine Division of Infectious Diseases and Immunology Department of Medicine New YorkNY10016 United States Digital Institute Ai Research Section Stockholm16731 Sweden Arcada University of Applied Sciences Department of Business Management and Analytics Helsinki00550 Finland University of Brescia Radiological Sciences and Public Health Department of Medical and Surgical Specialties Brescia25121 Italy Sncf Reseau Sa Ethique Groupe La Plaine93418 France Institute of Diagnostic and Interventional Radiology University Hospital Zurich Zürich8091 Switzerland Eindhoven University of Tech

This article's main contributions are twofold: 1) to demonstrate how to apply the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does 'trustworthy AI' mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of pandemic. The AI system aims to help radiologists to estimate and communicate the severity of damage in a patient's lung from Chest X-rays. It has been experimentally deployed in the radiology department of the ASST Spedali Civili clinic in Brescia, Italy, since December 2020 during pandemic time. The methodology we have applied for our post-hoc assessment, called Z-Inspection®, uses sociotechnical scenarios to identify ethical, technical, and domain-specific issues in the use of the AI system in the context of the pandemic. © 2020 IEEE.

关键词： COVID-19

A Robust Ensemble Algorithm for Ischemic Stroke Lesion Segmentation: Generalizability and Clinical Utility Beyond the ISLES Challenge

学校读者我要写书评

暂无评论

arXiv 2024年

作者： de la Rosa, Ezequiel Reyes, Mauricio Liew, Sook-Lei Hutton, Alexandre Wiest, Roland Kaesmacher, Johannes Hanning, Uta Hakim, Arsany Zubal, Richard Valenzuela, Waldo Robben, David Sima, Diana M. Anania, Vincenzo Brys, Arne Meakin, James A. Mickan, Anne Broocks, Gabriel Heitkamp, Christian Gao, Shengbo Liang, Kongming Zhang, Ziji Siddiquee, Md Mahfuzur Rahman Myronenko, Andriy Ashtari, Pooya Van Huffel, Sabine Jeong, Hyun-Su Yoon, Chi-Ho Kim, Chulhong Huo, Jiayu Ourselin, Sebastien Sparks, Rachel Clèrigues, Albert Oliver, Arnau Lladó, Xavier Chalcroft, Liam Pappas, Ioannis Bertels, Jeroen Heylen, Ewout Moreau, Juliette Hatami, Nima Frindel, Carole Qayyum, Abdul Mazher, Moona Puig, Domenec Lin, Shao-Chieh Juan, Chun-Jung Hu, Tianxi Boone, Lyndon Goubran, Maged Liu, Yi-Jui Wegener, Susanne Kofler, Florian Ezhov, Ivan Shit, Suprosanna Hernandez Petzsche, Moritz R. Menze, Bjoern Kirschke, Jan S. Wiestler, Benedikt Department of Quantitative Biomedicine University of Zurich Zurich Switzerland Department of Informatics Technical University Munich Germany Icometrix Leuven Belgium ARTORG Center for Biomedical Research University of Bern Bern Switzerland Department of Radiation Oncology University Hospital Bern University of Bern Switzerland University of Bern Bern Switzerland Chan Division of Occupational Science and Occupational Therapy University of Southern California Los AngelesCA United States Stevens Neuroimaging and Informatics Institute Department of Neurology Keck School of Medicine University of Southern California United States University Institute of Diagnostic and Interventional Neuroradiology Inselspital Bern Switzerland University Institute of Diagnostic and Interventional Neuroradiology University Hospital Bern Inselspital University of Bern Bern Switzerland Department of Diagnostic and Interventional neuroradiology University Medical Center Hamburg-Eppendorf Hamburg Germany Department of Medical Imaging Radboud University Medical Center Institute for Health Sciences Nijmegen Netherlands Deepwise AI Lab Beijing China Beijing University of Posts and Telecommunications Bejing China School of Computing and Augmented Intelligence Arizona State University TempeAZ United States NVIDIA Santa ClaraCA United States STADIUS Center for Dynamical Systems Signal Processing and Data Analytics KU Leuven Leuven Belgium Pohang Korea Republic of School of Biomedical Engineering & Imaging Sciences King’s College London United Kingdom Institute of Computer Vision and Robotics University of Girona Spain Wellcome Centre for Human Neuroimaging University College London London United Kingdom Laboratory of Neuro Imaging Stevens Institute for Neuroimaging and Informatics Keck School of Medicine University of Southern California Los Angeles United States KU Leuven Leuven Belgium CREATIS Université Lyon1 CNRS UMR5220 INSERM U1206 INSA-Lyon Villeurbanne696

Diffusion-weighted MRI (DWI) is essential for stroke diagnosis, treatment decisions, and prognosis. However, image and disease variability hinder the development of generalizable AI algorithms with clinical value. We address this gap by presenting a novel ensemble algorithm derived from the 2022 Ischemic Stroke Lesion Segmentation (ISLES) challenge. ISLES’22 provided 400 patient scans with ischemic stroke from various medical centers, facilitating the development of a wide range of cutting-edge segmentation algorithms by the research community. By assessing them against a hidden test set, we identified strengths, weaknesses, and potential biases. Through collaboration with leading teams, we combined top-performing algorithms into an ensemble model that overcomes the limitations of individual solutions. Our ensemble model combines the individual algorithms’ strengths and achieved superior ischemic lesion detection and segmentation accuracy (median Dice score: 0.82, median lesion-wise F1 score: 0.86) on our internal test set compared to individual algorithms. This accuracy generalized well across diverse image and disease variables. Furthermore, the model excelled in extracting clinical biomarkers like lesion types and affected vascular territories. Notably, in a Turing-like test, neuroradiologists consistently preferred the algorithm’s segmentations over manual expert efforts, highlighting increased comprehensiveness and precision. Validation using a real-world external dataset (N=1686) confirmed the model’s generalizability (median Dice score: 0.82, median lesion-wise F1 score: 0.86). The algorithm’s outputs also demonstrated strong correlations with clinical scores (admission NIHSS and 90-day mRS) on par with or exceeding expert-derived results, underlining its clinical relevance. This study offers two key findings. First, we present an ensemble algorithm that detects and segments ischemic stroke lesions on DWI across diverse scenarios on par with expert (neuro)rad

关键词： Diagnosis

Apple vision Pro for Healthcare: "The Ultimate Display"? Entering the Wonderland of Precision Medicine

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Egger, Jan Gsaxner, Christina Chen, Xiaojun Bian, Jiang Kleesiek, Jens Puladi, Behrus Girardetstraße 2 Essen45131 Germany Hufelandstraße 55 Essen45147 Germany Hufelandstraße 55 Essen45147 Germany Department of Oral and Maxillofacial Surgery University Hospital RWTH Aachen Pauwelsstraße 30 Aachen52074 Germany Institute of Computer Graphics and Vision Graz University of Technology Inffeldgasse 16/II Graz8010 Austria Institute of Biomedical Manufacturing and Life Quality Engineering State Key Laboratory of Mechanical System and Vibration School of Mechanical Engineering Shanghai Jiao Tong University Dongchuan Road 800 Minhang District Shanghai200240 China Institute of Medical Robotics Shanghai Jiao Tong University Dongchuan Road 800 Minhang District Shanghai200240 China Health Outcomes & Biomedical Informatics College of Medicine University of Florida GainesvilleFL32610 United States Partner Site Essen Hufelandstraße 55 Essen45147 Germany Department of Physics TU Dortmund University August-Schmidt-Str. 4 Dortmund44227 Germany Institute of Medical Informatics University Hospital RWTH Aachen Pauwelsstraße 30 Aachen52074 Germany

At the Worldwide Developers Conference (WWDC) in June 2023, Apple introduced the vision Pro. The vision Pro is a Mixed Reality (MR) headset, more specifically it is a Virtual Reality (VR) device with an additional Video See-Through (VST) capability. The VST capability turns the vision Pro also into an Augmented Reality (AR) device. The AR feature is enabled by streaming the real world via cameras to the (VR) screens in front of the user’s eyes. This is of course not unique and similar to other devices, like the Varjo XR-3. Nevertheless, the vision Pro has some interesting features, like an inside-out screen that can show the headset wearers’ eyes to "outsiders" or a button on the top, called "Digital Crown", that allows you to seamlessly blend digital content with your physical space by turning it. In addition, it is untethered, except for the cable to the battery, which makes the headset more agile, compared to the Varjo XR-3. This could actually come closer to the "Ultimate Display", which Ivan Sutherland had already sketched in 1965. Not available to the public yet, like the Ultimate Display, we want to take a look into the crystal ball in this perspective to see if it can overcome some clinical challenges that - especially - AR still faces in the medical domain, but also go beyond and discuss if the vision Pro could support clinicians in essential tasks to spend more time with their patients. © 2023, CC BY.

关键词： Augmented reality

Fusion of metadata and dermoscopic images for melanoma detection: Deep learning and feature importance analysis

学校读者我要写书评

暂无评论

Information Fusion 2025年

作者： Misbah Ahmad Imran Ahmed Abdellah Chehri Gwangill Jeon Centre for Machine Vision Bristol Robotics Laboratory University of West of England Frenchy Campus Bristol BS16 1QY Bristol UK Department of Animal and Agriculture Hartpury University Hartpury House Gloucester GL19 3BE Gloucestershire UK School of Computing and Information Science Anglia Ruskin University East Road Cambridge CB1 1PT Cambridgeshire UK Department of Mathematics and Computer Science Royal Military College of Canada (RMC) P.O. Box 17000 Station Forces Kingston K7K 7B4 Ontario Canada Department of Embedded Systems Engineering Incheon National University 119 Academy-ro Incheon 22012 Incheon Republic of Korea

In the era of smart healthcare, integrating multimodal data is essential for improving diagnostic accuracy and enabling personalized care. This study presented a deep learning-based multimodal approach for melanoma detection, leveraging both dermoscopic images and clinical metadata to enhance classification performance. The proposed model integrated a multi-layer convolutional neural network (CNN) to extract image features and combined them with structured metadata, including patient age, gender, and lesion location, through feature-level fusion. The fusion process occurred at the final CNN layer, where high-dimensional image feature vectors were concatenated with processed metadata. The metadata was handled separately through a fully connected neural network comprising multiple dense layers. The final fused representation was passed through additional dense layers, culminating in a classification layer that outputted the probability of melanoma presence. The model was trained end-to-end using the SIIM-ISIC dataset, allowing it to learn a joint representation of image and metadata features for optimal classification. Various data augmentation techniques were applied to dermoscopic images to mitigate class imbalance and improve model robustness. Additionally, exploratory data analysis (EDA) and feature importance analysis were conducted to assess the contribution of each metadata feature to the overall classification. Our fusion-based deep learning architecture outperformed single-modality models, boosting classification accuracy. The presented model achieved an accuracy of 94.5% and an overall F1-score of 0.94, validating its effectiveness in melanoma detection. This study aims to highlight the potential of deep learning-based multimodal fusion in enhancing diagnostic precision, offering a scalable and reliable solution for improved melanoma detection in smart healthcare systems.

关键词：

REFUGE2 CHALLENGE: A TREASURE TROVE FOR MULTI-DIMENSION ANALYSIS AND EVALUATION IN GLAUCOMA SCREENING

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fang, Huihui Li, Fei Wu, Junde Fu, Huazhu Sun, Xu Son, Jaemin Yu, Shuang Zhang, Menglu Yuan, Chenglang Bian, Cheng Lei, Baiying Zhao, Benjian Xu, Xinxing Li, Shaohua Fumero, Francisco Sigut, José Almubarak, Haidar Bazi, Yakoub Guo, Yuanhao Zhou, Yating Baid, Ujjwal Innani, Shubham Guo, Tianjiao Yang, Jie Orlando, José Ignacio Bogunović, Hrvoje Zhang, Xiulan Xu, Yanwu The REFUGE2 Challenge Australia State Key Laboratory of Ophthalmology Zhongshan Ophthalmic Center Sun Yat-Sen University Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science Guangzhou China Intelligent Healthcare Unit Baidu Inc. Beijing China The Institute of High Performance Computing Agency for Science Technology and Research Singapore Yatiris Group PLADEMA Institute CONICET UNICEN Tandil Argentina Christian Doppler Lab for Artificial Intelligence in Retina Department of Ophthalmology and Optometry Medical University of Vienna Vienna Austria VUNO Inc Seoul Korea Republic of Tencent HealthCare Tencent Shenzhen China Computer Vision Institute College of Computer Science and Software Engineering of Shenzhen University Shenzhen China School of Biomedical Engineering Health Science Center Shenzhen University China Xiaohe Healthcare ByteDance Guangdong Guangzhou510000 China School of Biomedical Engineering Shenzhen University China College of Computer Science & Software Engineering Shenzhen University China Department of Computer Science and Systems Engineering Universidad de La Laguna Spain Saudi Electronic University Saudi Arabia King Saud University Saudi Arabia Institute of Automation Chinese Academy of Sciences Beijing China University of Chinese Academy of Sciences Beijing China SGGS Institute of Engineering and Technology India Institute of Medical Robotics Shanghai Jiao Tong University China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

With the rapid development of artificial intelligence (AI) in medical image processing, deep learning in color fundus photography (CFP) analysis is also evolving. Although there are some open-source, labeled datasets of CFPs in the ophthalmology community, large-scale datasets for screening only have labels of disease categories, and datasets with annotations of fundus structures are usually small in size. In addition, labeling standards are not uniform across datasets, and there is no clear information on the acquisition device. Here we release a multi-annotation, multi-quality, and multi-device color fundus image dataset for glaucoma analysis on an original challenge-Retinal Fundus Glaucoma Challenge 2nd Edition (REFUGE2). The REFUGE2 dataset contains 2000 color fundus images with annotations of glaucoma classification, optic disc/cup segmentation, as well as fovea localization. Meanwhile, the REFUGE2 challenge sets three sub-tasks of automatic glaucoma diagnosis and fundus structure analysis and provides an online evaluation framework. Based on the characteristics of multi-device and multi-quality data, some methods with strong generalizations are provided in the challenge to make the predictions more robust. This shows that REFUGE2 brings attention to the characteristics of real-world multi-domain data, bridging the gap between scientific research and clinical application. © 2022, CC BY-NC-ND.

关键词： Color

SocNav1: A dataset to benchmark and learn social navigation conventions∗

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Manso, Luis J. Núñez, Pedro Calderita, Luis V. Faria, Diego R. Bachiller, Pilar Computer Science Department School of Engineering and Applied Science Aston University Birmingham United Kingdom Robotics and Artificial Vision Laboratory - RoboLab Caceres School of Technology Universidad de Extremadura Caceres Extremadura

Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-NET

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Salem, Mostafa Valverde, Sergi Cabezas, Mariano Pareto, Deborah Oliver, Arnau Salvi, Joaquim Rovira, Alex Llado, Xavier Research Institute of Computer Vision and Robotics University of Girona Girona17003 Spain Computer Science Department Faculty of Computers and Information Assiut University Egypt Magnetic Resonance Unit Dept of Radiology Vall d'Hebron University Hospital Spain

In this paper, we propose generating synthetic multiple sclerosis (MS) lesions on MRI images with the final aim to improve the performance of supervised machine learning algorithms, therefore avoiding the problem of the lack of available ground truth. We propose a two-input two-output fully convolutional neural network model for MS lesion synthesis in MRI images. The lesion information is encoded as discrete binary intensity level masks passed to the model and stacked with the input images. The model is trained end-to-end without the need for manually annotating the lesions in the training set. We then perform the generation of synthetic lesions on healthy images via registration of patient images, which are subsequently used for data augmentation to increase the performance for supervised MS lesion detection algorithms. Our pipeline is evaluated on MS patient data from an in-house clinical dataset and the public ISBI2015 challenge dataset. The evaluation is based on measuring the similarities between the real and the synthetic images as well as in terms of lesion detection performance by segmenting both the original and synthetic images individually using a state-of-the-art segmentation framework. We also demonstrate the usage of synthetic MS lesions generated on healthy images as data augmentation. We analyze a scenario of limited training data (one-image training) to demonstrate the effect of the data augmentation on both datasets. Our results significantly show the effectiveness of the usage of synthetic MS lesion images. For the ISBI2015 challenge, our one-image model trained using only a single image plus the synthetic data augmentation strategy showed a performance similar to that of other CNN methods that were fully trained using the entire training set, yielding a comparable human expert rater performance. Copyright © 2019, The Authors. All rights reserved.

关键词： Magnetic resonance imaging

Quantitative susceptibility atlas construction in montreal neurological institute space: Towards histological-consistent Iron-rich deep brain nucleus sub-region identification

学校读者我要写书评

暂无评论

Research Square

Research Square 2021年

作者： He, Chenyu Guan, Xiaojun Zhang, Weimin Li, Jun Liu, Chunlei Wei, Hongjiang Xu, Xiaojun Zhang, Yuyao School of Information Science and Technology ShanghaiTech University 393 Huaxia Road Shanghai201210 China Department of Radiology Second Affiliated Hospital Zhejiang University School of Medicine 88 Jiefang Road Hangzhou Zhejiang 310009 China Electrical Engineering and Computer Science University of California at Berkeley BerkeleyCA94720 United States School of Biomedical Engineering Shanghai Jiao Tong University 800 Dongchuan Road Shanghai200030 China Institute of Medical Robotics Shanghai Jiao Tong University 800 Dongchuan Road Shanghai200030 China Shanghai Engineering Research Center of Intelligent Vision and Imaging ShanghaiTech University 393 Huaxia Road Shanghai201210 China

Iron-rich deep brain nuclei (DBN) of the human brain are involved in various motoric, emotional and cognitive brain functions. The abnormal iron alterations in the DBN are closely associated with multiple neurological and psychiatric diseases. Quantitative susceptibility mapping (QSM) provides the spatial distribution of tissue magnetic susceptibility in the human brain. Compared to traditional structural imaging, QSM has superiority for imaging the iron-rich DBN owing to the susceptibility difference existing between brain tissues. In this study, we construct a Montreal Neurological Institute (MNI) space unbiased QSM human brain atlas via group-wise registration from 100 healthy subjects aged 19-29 years. The atlas construction process is guided by hybrid images that fused from multi-modal Magnetic Resonance Images (MRI), thus named as Multi-modal-fused magnetic Susceptibility (MuSus-100) atlas. The high-quality susceptibility atlas provides extraordinary image contrast between iron-rich DBN with their surroundings. Parcellation maps of DBN and their sub-regions that are highly related to neurological and psychiatric pathology are then manually labeled based on the atlas set with the assistance of an image border-enhancement process. Especially, the bilateral thalamus is delineated into 64 detailed sub-regions referring to the Schaltenbrand and Wahren stereotactic atlas. To our best knowledge, the histological-consistent thalamic nucleus parcellation map is well defined for the first time in MNI space. Comparing with existing atlases emphasized on DBN parcellation, the newly proposed atlas outperforms on atlas-guided individual brain image DBN segmentation accuracy and robustness. Moreover, we apply the proposed DBN parcellation map to conduct detailed identification of the pathology-related iron content alterations in subcortical nuclei for Parkinson Disease (PD) patients. We envision that the MuSus-100 atlas could play a crucial role in improving the accuracy of

关键词： Brain

Why is the winner the best?

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Eisenmann, M. Reinke, A. Weru, V. Tizabi, M.D. Isensee, F. Adler, T.J. Ali, S. Andrearczyk, V. Aubreville, M. Baid, U. Bakas, S. Balu, N. Bano, S. Bernal, J. Bodenstedt, S. Casella, A. Cheplygina, V. Daum, M. de Bruijne, M. Depeursinge, A. Dorent, R. Egger, J. Ellis, D.G. Engelhardt, S. Ganz, M. Ghatwary, N. Girard, G. Godau, P. Gupta, A. Hansen, L. Harada, K. Heinrich, M. Heller, N. Hering, A. Huaulmé, A. Jannin, P. Kavur, A.E. Kodym, O. Kozubek, M. Li, J. Li, H. Ma, J. Martín-Isla, C. Menze, B. Noble, A. Oreiller, V. Padoy, N. Pati, S. Payette, K. Rädsch, T. Rafael-Patiño, J. Bawa, V. Singh Speidel, S. Sudre, C.H. van Wijnen, K. Wagner, M. Wei, D. Yamlahi, A. Yap, M.H. Yuan, C. Zenk, M. Zia, A. Zimmerer, D. Aydogan, D. Bhattarai, B. Bloch, L. Brüngel, R. Cho, J. Choi, C. Dou, Q. Ezhov, I. Friedrich, C.M. Fuller, C. Gaire, R.R. Galdran, A. Faura, Á. García Grammatikopoulou, M. Hong, S. Jahanifar, M. Jang, I. Kadkhodamohammadi, A. Kang, I. Kofler, F. Kondo, S. Kuijf, H. Li, M. Luu, M. Martinčič, T. Morais, P. Naser, M.A. Oliveira, B. Owen, D. Pang, S. Park, J. Park, S. Plotka, S. Puybareau, E. Rajpoot, N. Ryu, K. Saeed, N. Shephard, A. Shi, P. Štepec, D. Subedi, R. Tochon, G. Torres, H.R. Urien, H. Vilaça, J.L. Wahid, K.A. Wang, H. Wang, J. Wang, L. Wang, X. Wiestler, B. Wodzinski, M. Xia, F. Xie, J. Xiong, Z. Yang, S. Yang, Y. Zhao, Z. Maier-Hein, K. Jäger, P.F. Kopp-Schneider, A. Maier-Hein, L. Heidelberg Germany Heidelberg Germany Faculty of Mathematics and Computer Science Heidelberg University Heidelberg Germany Heidelberg Germany Heidelberg Germany School of Computing Faculty of Engineering and Physical Sciences University of Leeds Leeds United Kingdom Institute of Informatics School of Management HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland Sierre Switzerland Department of Nuclear Medicine and Molecular Imaging Lausanne University Hospital Lausanne Switzerland Technische Hochschule Ingolstadt Ingolstadt Germany University of Pennsylvania PhiladelphiaPA United States Department of Pathology and Laboratory Medicine Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Department of Radiology Perelman School of Medicine University of Pennsylvania PhiladelphiaPA United States Department of Radiology University of Washington SeattleWA United States Department of Computer Science University College London London United Kingdom Universitat Autònoma de Barcelona & Computer Vision Center Barcelona Spain Dresden Dresden Germany Department of Advanced Robotics Istituto Italiano di Tecnologia Italy IT University of Copenhagen Copenhagen Denmark Department of General Visceral and Transplantation Surgery Heidelberg University Hospital Heidelberg Germany Biomedical Imaging Group Rotterdam Department of Radiology and Nuclear Medicine Erasmus MC Rotterdam Netherlands Department of Computer Science University of Copenhagen Copenhagen Denmark Sierre Switzerland Harvard Medical School Brigham and Women’s Hospital BostonMA United States School of Biomedical Engineering and Imaging Sciences King’s College London London United Kingdom Essen Germany University of Nebraska Medical Center OmahaNE United States Department of Internal Medicine III Heidelberg University Hospital Heidelberg Germany Neurobiology Research Unit Copenhagen University Hospital Rigshospitalet C

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and postprocessing (66%). The "typical" lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work. © 2023, CC BY.

关键词： Benchmarking