Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Study

作     者：Ng, Matthew Guo, Fumin Biswas, Labonny Petersen, Steffen E. Piechnik, Stefan K. Neubauer, Stefan Wright, Graham 

作者机构：The Physical Sciences Platform Sunnybrook Research Institute Department of Medical Biophysics University of Toronto TorontoON Canada The Wuhan National Laboratory for Optoelectronics and Biomedical Engineering Huazhong University of Science and Technology Wuhan China Physical Sciences Platform Sunnybrook Research Institute Department of Medical Biophysics University of Toronto TorontoON Canada The Physical Sciences Platform Sunnybrook Research Institute TorontoON Canada The William Harvey Research Institute NIHR Barts Biomedical Research Centre Queen Mary University of London United Kingdom Barts Heart Centre St Bartholomew’s Hospital Barts Health NHS Trust West Smithfield London United Kingdom The Division of Cardiovascular Medicine Oxford NIHR Biomedical Research Centre Radcliffe Department of Medicine University of Oxford Oxford United Kingdom 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2020年

核心收录：

主　　题：Quality control 

摘      要：Objective: Convolutional neural networks (CNNs) have demonstrated promise in automated cardiac magnetic resonance image segmentation. However, when using CNNs in a large real-world dataset, it is important to quantify segmentation uncertainty and identify segmentations which could be problematic. In this work, we performed a systematic study of Bayesian and non-Bayesian methods for estimating uncertainty in segmentation neural networks. Methods: We evaluated Bayes by Backprop, Monte Carlo Dropout, Deep Ensembles, and Stochastic Segmentation Networks in terms of segmentation accuracy, probability calibration, uncertainty on out-of-distribution images, and segmentation quality control. Results: We observed that Deep Ensembles outperformed the other methods except for images with heavy noise and blurring distortions. We showed that Bayes by Backprop is more robust to noise distortions while Stochastic Segmentation Networks are more resistant to blurring distortions. For segmentation quality control, we showed that segmentation uncertainty is correlated with segmentation accuracy for all the methods. With the incorporation of uncertainty estimates, we were able to reduce the percentage of poor segmentation to 5% by flagging 31–48% of the most uncertain segmentations for manual review, substantially lower than random review without using neural network uncertainty (reviewing 75–78% of all images). Conclusion: This work provides a comprehensive evaluation of uncertainty estimation methods and showed that Deep Ensembles outperformed other methods in most cases. Significance: Neural network uncertainty measures can help identify potentially inaccurate segmentations and alert users for manual review. © 2020, CC BY-NC-ND.