International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from t...
International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multicenter study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), image preprocessing (97%), data curation (79%), and post-processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.
Background: Despite major advances in artificial intelligence (AI) research for healthcare, the deployment and adoption of AI technologies remain limited in clinical practice. In recent years, concerns have been raise...
详细信息
Digital health technologies (smartphones, smartwatches, and other body-worn sensors) can act as novel tools to aid in the diagnosis and remote objective monitoring of an individual's disease symptoms, both in clin...
详细信息
Background: The COVID-19 pandemic highlighted gaps in health surveillance systems, disease prevention, and treatment globally. Among the many factors that might have led to these gaps is the issue of the financing of ...
Background: The COVID-19 pandemic highlighted gaps in health surveillance systems, disease prevention, and treatment globally. Among the many factors that might have led to these gaps is the issue of the financing of national health systems, especially in low-income and middle-income countries (LMICs), as well as a robust global system for pandemic preparedness. We aimed to provide a comparative assessment of global health spending at the onset of the pandemic;characterise the amount of development assistance for pandemic preparedness and response disbursed in the first 2 years of the COVID-19 pandemic;and examine expectations for future health spending and put into context the expected need for investment in pandemic preparedness. Methods: In this analysis of global health spending between 1990 and 2021, and prediction from 2021 to 2026, we estimated four sources of health spending: development assistance for health (DAH), government spending, out-of-pocket spending, and prepaid private spending across 204 countries and territories. We used the Organisation for Economic Co-operation and Development (OECD)'s Creditor Reporting System (CRS) and the WHO Global Health Expenditure database (GHED) to estimate spending. We estimated development assistance for general health, COVID-19 response, and pandemic preparedness and response using a keyword search. Health spending estimates were combined with estimates of resources needed for pandemic prevention and preparedness to analyse future health spending patterns, relative to need. Findings: In 2019, at the onset of the COVID-19 pandemic, US$9·2 trillion (95% uncertainty interval [UI] 9·1–9·3) was spent on health worldwide. We found great disparities in the amount of resources devoted to health, with high-income countries spending $7·3 trillion (95% UI 7·2–7·4) in 2019;293·7 times the $24·8 billion (95% UI 24·3–25·3) spent by low-income countries in 2019. That same year, $43·1 billion in development assistance was provided
The number of international benchmarking competitions is steadily increasing in various fields of machinelearning (ML) research and practice. So far, however, little is known about the common practice as well as bott...
详细信息
Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse mul...
Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.
Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrot...
详细信息
暂无评论