Assessing Human Judgment Forecasts in the Rapid Spread of the Mpox Outbreak: Insights and Challenges for Pandemic Preparedness

作     者：McAndrew, Thomas Majumder, Maimuna S. Lover, Andrew A. Venkatramanan, Srini Bocchini, Paolo Besiroglu, Tamay Codi, Allison Dempsey, Gaia Abbott, Sam Chevalier, Sylvain Bosse, Nikos I. Cambeiro, Juan Braun, David 

作者机构：Department of Community and Population Health College of Health Lehigh University BethlehemPA United States Boston Children's Hospital BostonMA United States Harvard Medical School BostonMA United States Dept. of Biostatistics and Epidemiology School of Public Health and Health Sciences University of Massachusetts Amherst AmherstMA United States Biocomplexity Institute and Initiative University of Virginia CharlottesvilleVA United States Department of Civil and Environmental Engineering P.C. Rossin College of Engineering and Applied Science Lehigh University BethlehemPA United States MIT Computer Science and Artificial Intelligence Laboratory CambridgeMA United States Metaculus Santa CruzCA United States Department of Biostatistics Rollins School of Public Health Emory University AtlantaGA United States Centre for Mathematical Modelling of Infectious Diseases London School of Hygiene & Tropical Medicine Keppel St LondonWC1E 7HT United Kingdom Department of Epidemiology Mailman School of Public Health Columbia University New York United States Department of Psychology Lehigh University BethlehemPA United States 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Forecasting 

摘      要：In May 2022, mpox (formerly monkeypox) spread to non-endemic countries rapidly. Human judgment is a forecasting approach that has been sparsely evaluated during the beginning of an outbreak. We collected-between May 19, 2022 and July 31, 2022-1275 forecasts from 442 individuals of six questions about the mpox outbreak where ground truth data are now available. Individual human judgment forecasts and an equally weighted ensemble were evaluated, as well as compared to a random walk, autoregressive, and doubling time model. We found (1) individual human judgment forecasts underestimated outbreak size, (2) the ensemble forecast median moved closer to the ground truth over time but uncertainty around the median did not appreciably decrease, and (3) compared to computational models, for 2-8 week ahead forecasts, the human judgment ensemble outperformed all three models when using median absolute error and weighted interval score;for one week ahead forecasts a random walk outperformed human judgment. We propose two possible explanations: at the time a forecast was submitted, the mode was correlated with the most recent (and smaller) observation that would eventually determine ground truth. Several forecasts were solicited on a logarithmic scale which may have caused humans to generate forecasts with unintended, large uncertainty intervals. To aide in outbreak preparedness, platforms that solicit human judgment forecasts may wish to assess whether specifying a forecast on logarithmic scale matches an individual s intended forecast, support human judgment by finding cues that are typically used to build forecasts, and, to improve performance, tailor their platform to allow forecasters to assign zero probability to events. © 2024, CC BY.