Deep multimodal predictome for studying mental disorders (vol 44, pg 509, 2023)

作     者：Rahaman, M. A. Chen, J. Fu, Z. Lewis, N. Iraji, A. van Erp, T. G. M. Calhoun, V. D. 

作者机构：Department of Computational Science and Engineering Georgia Institute of Technology Atlanta Georgia USA Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS) Georgia State University Georgia Institute of Technology Emory University Atlanta Georgia USA Clinical Translational Neuroscience Laboratory Department of Psychiatry and Human Behavior University of California Irvine Irvine California USA Center for the Neurobiology of Learning and Memory University of California Irvine Irvine California USA 

出 版 物：《HUMAN BRAIN MAPPING》 (人脑图像描记)

年 卷 期：2023年第44卷第14期

页      面：4967-4967页

核心收录：

学科分类：0710[理学-生物学] 1001[医学-基础医学(可授医学、理学学位)] 07[理学] 1010[医学-医学技术（可授医学、理学学位）] 09[农学] 1009[医学-特种医学] 

基　　金：National Institute of Mental Health (NIMH) [R01MH118695  R01MH123610] 

主　　题：functional network connectivity multimodal deep learning resting-state functional and structural MRI saliency schizophrenia classification single nucleotide polymorphism 

摘      要：Characterizing neuropsychiatric disorders is challenging due to heterogeneity in the population. We propose combining structural and functional neuroimaging and genomic data in a multimodal classification framework to leverage their complementary information. Our objectives are two-fold (i) to improve the classification of disorders and (ii) to introspect the concepts learned to explore underlying neural and biological mechanisms linked to mental disorders. Previous multimodal studies have focused on naïve neural networks, mostly perceptron, to learn modality-wise features and often assume equal contribution from each modality. Our focus is on the development of neural networks for feature learning and implementing an adaptive control unit for the fusion phase. Our mid fusion with attention model includes a multilayer feed-forward network, an autoencoder, a bi-directional long short-term memory unit with attention as the features extractor, and a linear attention module for controlling modality-specific influence. The proposed model acquired 92% (p .0001) accuracy in schizophrenia prediction, outperforming several other state-of-the-art models applied to unimodal or multimodal data. Post hoc feature analyses uncovered critical neural features and genes/biological pathways associated with schizophrenia. The proposed model effectively combines multimodal neuroimaging and genomics data for predicting mental disorders. Interpreting salient features identified by the model may advance our understanding of their underlying etiological mechanisms.