检索结果-内蒙古大学图书馆

Scaling up self-supervised learning for improved surgical foundation models

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Jaspers, Tim J.M. de Jong, Ronald L.P.D. Li, Yiping Kusters, Carolus H.J. Bakker, Franciscus H.A. van Jaarsveld, Romy C. Kuiper, Gino M. van Hillegersberg, Richard Ruurda, Jelle P. Brinkman, Willem M. Pluim, Josien P.W. de With, Peter H.N. Breeuwer, Marcel Khalil, Yasmina Al van der Sommen, Fons Department of Electrical Engineering Video Coding & Architectures Eindhoven University of Technology Eindhoven Netherlands Department of Biomedical Engineering Medical Image Analysis Eindhoven University of Technology Eindhoven Netherlands Department of Surgery University Medical Center Utrecht Utrecht Netherlands Department of Oncological Urology University Medical Center Utrecht Utrecht Netherlands Department of Urology Catharina Hospital Eindhoven Netherlands

Foundation models have revolutionized computer vision by achieving vastly superior performance across diverse tasks through large-scale pretraining on extensive datasets. However, their application in surgical computer vision has been limited. This study addresses this gap by introducing SurgeNetXL, a novel surgical foundation model that sets a new benchmark in surgical computer vision. Trained on the largest reported surgical dataset to date, comprising over 4.7 million video frames, SurgeNetXL achieves consistent top-tier performance across six datasets spanning four surgical procedures and three tasks, including semantic segmentation, phase recognition, and critical view of safety (CVS) classification. Compared with the best-performing surgical foundation models, SurgeNetXL shows mean improvements of 2.4, 9.0, and 12.6 percent for semantic segmentation, phase recognition, and CVS classification, respectively. Additionally, SurgeNetXL outperforms the best-performing imageNet-based variants by 14.4, 4.0, and 1.6 percent in the respective tasks. In addition to advancing model performance, this study provides key insights into scaling pretraining datasets, extending training durations, and optimizing model architectures specifically for surgical computer vision. These findings pave the way for improved generalizability and robustness in data-scarce scenarios, offering a comprehensive framework for future research in this domain. All models and a subset of the SurgeNetXL dataset, including over 2 million video frames, are publicly available at: https://***/TimJaspers0801/SurgeNet. © 2025, CC BY.

关键词： Self-supervised learning

Three-Dimensional Nuclei Synthesis for Fluorescence Microscopy image analysis

学校读者我要写书评

暂无评论

Three-Dimensional Nuclei Synthesis for Fluorescence Microsco...

IEEE International Symposium on Biomedical Imaging

作者： Liming Wu Alain Chen Paul Salama Kenneth W. Dunn Edward J. Delp Video and Image Processing Laboratory School of Electrical and Computer Engineering Purdue University West Lafayette Indiana USA Department of Electrical and Computer Engineering Indiana University-Purdue University Indianapolis Indiana USA Division of Nephrology School of Medicine Indiana University Indianapolis Indiana USA

Three-dimensional tissue cytometry is an important technique for quantitative analysis of cell structures in large fluorescence microscopy volumes. Accurate nuclei detection and segmentation is an important step for 3D tissue cytometry. Deep learning methods have shown promising results for nuclei detection and segmentation. However, manually annotating ground truth for training deep learning methods is labor-intensive and not practical for large 3D volumes. In this paper, we propose a 3D nuclei synthesis method, known as 3DSpCycleGAN, for generating 3D ground truth volumes along with corresponding synthetic microscopy volumes. Experimental results using fluorescence microscopy volumes demonstrate that our method generates more realistic 3D volumes when evaluated both visually and quantitatively than previously reported. We also show that using the synthetic volumes generated by 3DSpCycleGAN as training data improves segmentation accuracy for deep learning segmentation techniques.

关键词：

Beyond MR image Harmonization: Resolution Matters Too

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hays, Savannah P. Remedios, Samuel W. Zuo, Lianrui Mowry, Ellen M. Newsome, Scott D. Calabresi, Peter A. Carass, Aaron Dewey, Blake E. Prince, Jerry L. Image Analysis and Communications Laboratory Department of Electrical and Computer Engineering Johns Hopkins University BaltimoreMD21218 United States Department of Computer Science Johns Hopkins University BaltimoreMD21218 United States Department of Electrical and Computer Engineering Vanderbilt University NashvilleTN37235 United States Department of Neurology Johns Hopkins School of Medicine BaltimoreMD21287 United States

Magnetic resonance (MR) imaging is commonly used in the clinical setting to non-invasively monitor the body. There exists a large variability in MR imaging due to differences in scanner hardware, software, and protocol design. Ideally, a processing algorithm should perform robustly to this variability, but that is not always the case in reality. This introduces a need for image harmonization to overcome issues of domain shift when performing downstream analysis such as segmentation. Most image harmonization models focus on acquisition parameters such as inversion time or repetition time, but they ignore an important aspect in MR imaging—resolution. In this paper, we evaluate the impact of image resolution on harmonization using a pretrained harmonization algorithm. We simulate 2D acquisitions of various slice thicknesses and gaps from 3D acquired, 1mm3 isotropic MR images and demonstrate how the performance of a state-of-the-art image harmonization algorithm varies as resolution changes. We discuss the most ideal scenarios for image resolution including acquisition orientation when 3D imaging is not available, which is common for many clinical scanners. Our results show that harmonization on low-resolution images does not account for acquisition resolution and orientation variations. Super-resolution can be used to alleviate resolution variations but it is not always used. Our methodology can generalize to help evaluate the impact of image acquisition resolution for multiple tasks. Determining the limits of a pretrained algorithm is important when considering preprocessing steps and trust in the results. Copyright © 2024, The Authors. All rights reserved.

关键词： Magnetic resonance imaging

ISGAN: Unsupervised Domain Adaptation With Improved Symmetric GAN for Cross-Modality Multi-Organ Segmentation

学校读者我要写书评

暂无评论

IEEE Journal of Biomedical and Health Informatics 2024年第6期29卷 3874-3885页

作者： Jiapeng Li Yifan Zhang Lisheng Xu Yudong Yao Lin Qi College of Medicine and Biological Information Engineering Northeastern University Shenyang China Engineering Research Center of Medical Imaging and Intelligent Analysis Ministry of Education Shenyang China Key Laboratory of Medical Image Computing Ministry of Education Northeastern University Shenyang China Department of Electrical and Computer Engineering Stevens Institute of Technology Hoboken NJ USA

The differences between cross-modality medical images are significant, so several studies are working on unsupervised domain adaptation (UDA) segmentation, which aims to adapt a segmentation model trained on a labeled source domain to an unlabeled target domain. The conventional UDA segmentation strategy aims to integrate image generation and segmentation. However, conventional image generation modules only consider information from a single domain (source or target), resulting in visual inconsistencies. The image generation module may also lack anatomical constraints, leading to incorrect pseudo-label generation. To address these issues, we propose an improved symmetric generative adversarial network (ISGAN). Unlike conventional approaches that perform domain adaptation only in the source or target domain, ISGAN adopts a symmetric architecture using two-path domain adaptation to reduce the visual difference. In addition, ISGAN adopts a bidirectional training strategy to optimize the image generation and segmentation modules. The bidirectional training strategy introduces the anatomical constraints into the image generation module, thereby reducing the generation of incorrect pseudo labels. Finally, we validate ISGAN on two cross-modality datasets (the MMWHS cardiac dataset and Abdomen dataset). ISGAN delivers promising segmentation and generalization performance compared with state-of-the-art UDA methods.

关键词： image segmentation Training Biomedical imaging image synthesis Magnetic resonance imaging Computed tomography Feature extraction Generative adversarial networks Generators Diffusion models

Fast Low-parameter video Activity Localization in Collaborative Learning Environments

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jatla, Venkatesh Teeparthi, Sravani Egala, Ugesh Pattichis, Sylvia Celedón Pattichis, Marios S. Image and Video Processing and Communications Lab Dept. of Electrical and Computer Engineering University of New Mexico AlbuquerqueNM87131 United States Department of Curriculum and Instruction The University of Texas at Austin AustinTX78712-1293 United States

Research on video activity detection has primarily focused on identifying well-defined human activities in short video segments. The majority of the research on video activity recognition is focused on the development of large parameter systems that require training on large video datasets. This paper develops a low-parameter, modular system with rapid inferencing capabilities that can be trained entirely on limited datasets without requiring transfer learning from large-parameter systems. The system can accurately detect and associate specific activities with the students who perform the activities in real-life classroom videos. Additionally, the paper develops an interactive web-based application to visualize human activity maps over long real-life classroom videos. Long-term video activity detection in real-life classroom videos present unique challenges, such as the need to detect multiple simultaneous activities, rapid transitions between activities, long-term occlusions, durations exceeding 15 minutes, and numerous individuals performing similar activities in the background. Moreover, subtle hand movements further complicate the need to differentiate between actual typing and writing activities as opposed to unrelated hand movements. The system processes the input videos using fast activity initializations and current methods for object detection to determine the location and the the person performing the activities. These regions are then processed through an optimal low-parameter dyadic 3D-CNN classifier to identify the activity. The proposed system processes 1 hour of video in 15 minutes for typing and 50 minutes for writing activities. The system uses several methods to optimize the inference pipeline. For each activity, the system determines an optimal low-parameter 3D CNN architecture selected from a family of low-parameter architectures. The input video is broken into smaller video regions that are transcoded at an optimized frame rate. For inference, a

关键词： Object detection

Long-term Human Participation Assessment In Collaborative Learning Environments Using Dynamic Scene analysis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Shi, Wenjing Tran, Phuong Celedón-Pattichis, Sylvia Pattichis, Marios S. Image and Video Processing and Communications Lab Dept. of Electrical and Computer Engineering University of New Mexico AlbuquerqueNM87131 United States Department of Curriculum and Instruction The University of Texas at Austin AustinTX78712-1293 United States

The paper develops datasets and methods to assess student participation in real-life collaborative learning environments. In collaborative learning environments, students are organized into small groups where they are free to interact within their group. Thus, students can move around freely causing issues with strong pose variation, move out and re-enter the camera scene, or face away from the camera. We formulate the problem of assessing student participation into two subproblems: (i) student group detection against strong background interference from other groups, and (ii) dynamic participant tracking within the group. A massive independent testing dataset of 12,518,250 student label instances, of total duration of 21 hours and 22 minutes of real-life videos, is used for evaluating the performance of our proposed method for student group detection. The proposed method of using multiple image representations is shown to perform equally or better than YOLO on all video instances. Over the entire dataset, the proposed method achieved an F1 score of 0.85 compared to 0.80 for YOLO. Following student group detection, the paper presents the development of a dynamic participant tracking system for assessing student group participation through long video sessions. The proposed dynamic participant tracking system is shown to perform exceptionally well, missing a student in just one out of 35 testing videos. In comparison, a state-of-the-art method fails to track students in 14 out of the 35 testing videos. The proposed method achieves 82.3% accuracy on an independent set of long, real-life collaborative videos. © 2024, CC BY.

关键词： Students

Exploring the Effect of Dataset Diversity in Self-Supervised Learning for Surgical computer Vision

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Jaspers, Tim J.M. de Jong, Ronald L.P.D. Al Khalil, Yasmina Zeelenberg, Tijn Kusters, Carolus H.J. Li, Yiping van Jaarsveld, Romy C. Bakker, Franciscus H.A. Ruurda, Jelle P. Brinkman, Willem M. De With, Peter H.N. van der Sommen, Fons Department of Electrical Engineering Video Coding & Architectures Eindhoven University of Technology Eindhoven Netherlands Department of Biomedical Engineering Medical Image Analysis Eindhoven University of Technology Eindhoven Netherlands Department of Surgery University Medical Center Utrecht Utrecht Netherlands Department of Oncological Urology University Medical Center Utrecht Utrecht Netherlands Department of Urology Catharina Hospital Eindhoven Netherlands

Over the past decade, computer vision applications in minimally invasive surgery have rapidly increased. Despite this growth, the impact of surgical computer vision remains limited compared to other medical fields like pathology and radiology, primarily due to the scarcity of representative annotated data. Whereas transfer learning from large annotated datasets such as imageNet has been conventionally the norm to achieve high-performing models, recent advancements in self-supervised learning (SSL) have demonstrated superior performance. In medical image analysis, in-domain SSL pretraining has already been shown to outperform imageNet-based initialization. Although unlabeled data in the field of surgical computer vision is abundant, the diversity within this data is limited. This study investigates the role of dataset diversity in SSL for surgical computer vision, comparing procedure-specific datasets against a more heterogeneous general surgical dataset across three different downstream surgical applications. The obtained results show that using solely procedure-specific data can lead to substantial improvements of 13.8%, 9.5%, and 36.8% compared to imageNet pretraining. However, extending this data with more heterogeneous surgical data further increases performance by an additional 5.0%, 5.2%, and 2.5%, suggesting that increasing diversity within SSL data is beneficial for model performance. The code and pretrained model weights are made publicly available at https://***/TimJaspers0801/SurgeNet. © 2024, CC BY.

关键词： Self-supervised learning

GEM: Context-Aware Gaze EstiMation with Visual Search Behavior Matching for Chest Radiograph

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Shaonan Chen, Wenting Liu, Jie Luo, Xiaoling Shen, Linlin Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University China Department of Electrical Engineering City University of Hong Kong Hong Kong AI Research Center for Medical Image Analysis and Diagnosis Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China

Gaze estimation is pivotal in human scene comprehension tasks, particularly in medical diagnostic analysis. Eye-tracking technology facilitates the recording of physicians’ ocular movements during image interpretation, thereby elucidating their visual attention patterns and information-processing strategies. In this paper, we initially define the context-aware gaze estimation problem in medical radiology report settings. To understand the attention allocation and cognitive behavior of radiologists during the medical image interpretation process, we propose a context-aware Gaze EstiMation (GEM) network that utilizes eye gaze data collected from radiologists to simulate their visual search behavior patterns throughout the image interpretation process. It consists of a context-awareness module, visual behavior graph construction, and visual behavior matching. Within the context-awareness module, we achieve intricate multimodal registration by establishing connections between medical reports and images. Subsequently, for a more accurate simulation of genuine visual search behavior patterns, we introduce a visual behavior graph structure, capturing such behavior through high-order relationships (edges) between gaze points (nodes). To maintain the authenticity of visual behavior, we devise a visual behavior-matching approach, adjusting the high-order relationships between them by matching the graph constructed from real and estimated gaze points. Extensive experiments on four publicly available datasets demonstrate the superiority of GEM over existing methods and its strong generalizability, which also provides a new direction for the effective utilization of diverse modalities in medical image interpretation and enhances the interpretability of models in the field of medical imaging. https://***/Tiger-SN/GEM. Copyright © 2024, The Authors. All rights reserved.

关键词： Eye movements

A Completely Blind video Quality Evaluator

学校读者我要写书评

暂无评论

IEEE Signal Processing Letters 2022年 29卷 2228-2232页

作者： Qi Zheng Zhengzhong Tu Xiaoyang Zeng Alan C. Bovik Yibo Fan State Key Laboratory of ASIC & System College of Microelectronics Fudan University Shanghai China Laboratory for Image and Video Engineering (LIVE) Department of Electrical and Computer Engineering The University of Texas at Austin Austin TX USA

Automatic video quality assessment of user-generated content (UGC) has gained increased interest recently, due to the ubiquity of shared video clips uploaded and circulated on social media platforms across the globe. Most existing video quality models developed for this vast content are trained on large numbers of samples labeled during large-scale subjective studies, which are often fail to exhibit adequate generalization abilities on unseen data. Thus, it is also desirable to develop opinion-unaware, “completely blind” video quality models, that are free of training, yet can compete with existing learning-based models. Here we propose such a model called VIQE (video Quality Evaluator), which we designed based on a comprehensive analysis of patch- and frame-wise video statistics, as well as of space-time statistical regularities of videos. The statistical features desired from the analysis capture complementary predictive aspects of perceptual quality, which are aggregated to obtain final video quality scores. Extensive experiments on recent large-scale video quality databases demonstrate that VIQE is even competitive with state-of-the-art opinion-aware models. The source code is being made available at https://***/uniqzheng/VIQE .

关键词： video recording Quality assessment Feature extraction Computational modeling Streaming media Distortion Distortion measurement