ML-SIM: universal reconstruction of structured illumination microscopy images using transfer learning

作     者：Christensen, Charles N. Ward, Edward N. Lu, Meng Lio, Pietro Kaminski, Clemens F. 

作者机构：Univ Cambridge Dept Chem Engn & Biotechnol Laser Analyt Grp Philippa Fawcett Dr Cambridge England Univ Cambridge Dept Comp Sci & Technol Artificial Intelligence Grp JJ Thomson Ave Cambridge England 

出 版 物：《BIOMEDICAL OPTICS EXPRESS》 (生物医学光学快报)

年 卷 期：2021年第12卷第5期

页      面：2720-2733页

核心收录：

学科分类：0710[理学-生物学] 070207[理学-光学] 07[理学] 08[工学] 0803[工学-光学工程] 1009[医学-特种医学] 0702[理学-物理学] 

基　　金：Engineering and Physical Sciences Research Council [L015889] Wellcome Trust Medical Research Council [K015850] MRC [MR/K02292X/1, MR/K015850/1] Funding Source: UKRI 

主　　题：Image quality Image resolution Imaging systems Live cell imaging Low light levels Real time imaging 

摘      要：Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible with live-cell imaging. However, the reconstruction of SIM images is often slow, prone to artefacts, and requires multiple parameter adjustments to reflect different hardware or experimental conditions. Here, we introduce a versatile reconstruction method, ML-SIM, which makes use of transfer learning to obtain a parameter-free model that generalises beyond the task of reconstructing data recorded by a specific imaging system for a specific sample type. We demonstrate the generality of the model and the high quality of the obtained reconstructions by application of ML-SIM on raw data obtained for multiple sample types acquired on distinct SIM microscopes. ML-SIM is an end-to-end deep residual neural network that is trained on an auxiliary domain consisting of simulated images, but is transferable to the target task of reconstructing experimental SIM images. By generating the training data to reflect challenging imaging conditions encountered in real systems, ML-SIM becomes robust to noise and irregularities in the illumination patterns of the raw SIM input frames. Since ML-SIM does not require the acquisition of experimental training data, the method can be efficiently adapted to any specific experimental SIM implementation. We compare the reconstruction quality enabled by ML-SIM with current state-of-the-art SIM reconstruction methods and demonstrate advantages in terms of generality and robustness to noise for both simulated and experimental inputs, thus making ML-SIM a useful alternative to traditional methods for challenging imaging conditions. Additionally, reconstruction of a SIM stack is accomplished in less than 200 ms on a modern graphics processing unit, enabling future applications for real-time imaging. Source code and ready-to-use software for the method are available at http