Evaluating Web-Based Automatic Transcription for Alzheimer Speech Data: Transcript Comparison and Machine Learning Analysis

作     者：Soroski, Thomas da Cunha Vasco, Thiago Newton-Mason, Sally Granby, Saffrin Lewis, Caitlin Harisinghani, Anuj Rizzo, Matteo Conati, Cristina Murray, Gabriel Carenini, Giuseppe Field, Thalia S. Jang, Hyeju 

作者机构：Vancouver Stroke Program Division of Neurology Faculty of Medicine University of British Columbia Vancouver BC Canada Department of Computer Science Faculty of Science University of British Columbia Vancouver BC Canada School of Computing University of the Fraser Valley Abbotsford BC Canada 

出 版 物：《JMIR Aging》 (JMIR Aging)

年 卷 期：2022年第5卷第3期

页      面：e33460页

主　　题：Alzheimer disease machine learning memory mild cognitive impairment natural language processing neurodegenerative disease speech speech recognition software transcription software 

摘      要：Background: Speech data for medical research can be collected noninvasively and in large volumes. Speech analysis has shown promise in diagnosing neurodegenerative disease. To effectively leverage speech data, transcription is important, as there is valuable information contained in lexical content. Manual transcription, while highly accurate, limits the potential scalability and cost savings associated with language-based screening. Objective: To better understand the use of automatic transcription for classification of neurodegenerative disease, namely, Alzheimer disease (AD), mild cognitive impairment (MCI), or subjective memory complaints (SMC) versus healthy controls, we compared automatically generated transcripts against transcripts that went through manual correction. Methods: We recruited individuals from a memory clinic (“patients) with a diagnosis of mild-to-moderate AD, (n=44, 30%), MCI (n=20, 13%), SMC (n=8, 5%), as well as healthy controls (n=77, 52%) living in the community. Participants were asked to describe a standardized picture, read a paragraph, and recall a pleasant life experience. We compared transcripts generated using Google speech-to-text software to manually verified transcripts by examining transcription confidence scores, transcription error rates, and machine learning classification accuracy. For the classification tasks, logistic regression, Gaussian naive Bayes, and random forests were used. Results: The transcription software showed higher confidence scores (P.05) for speech from healthy controls compared with patients. Classification models using human-verified transcripts significantly (P.001) outperformed automatically generated transcript models for both spontaneous speech tasks. This comparison showed no difference in the reading task. Manually adding pauses to transcripts had no impact on classification performance. However, manually correcting both spontaneous speech tasks led to significantl