检索结果-内蒙古大学图书馆

A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Yuli Karetić, Ivan Stegmaier, Johannes Walter, Peter Merhof, Dorit Institute of Imaging and Computer Vision RWTH Aachen University Germany NeuroTX Aachen e.V. Germany Department of Ophthalmology RWTH Aachen University Germany Institute of Image Analysis and Computer Vision University of Regensburg Germany

We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is trained to mimic the biomimetic perceptual model implemented in pulse2percept. The evaluator is a shallow VGG classifier, which is trained with original images. Based on 10,000 test images from the MNIST dataset, we show that the convolutional neural network-based encoder performs significantly better than the trivial downsampling approach, yielding a boost in the weighted F1-Score by 36.17% in the pre-trained classifier with 6×10 electrodes. With this fully neural network-based encoder, the quality of the downstream perceptions can be fine-tuned using gradient descent in an end-to-end fashion. Copyright © 2023, The Authors. All rights reserved.

关键词： Ophthalmology

A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

学校读者我要写书评

暂无评论

A Deep Learning-based in silico Framework for Optimization o...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Yuli Wu Ivan Karetić Johannes Stegmaier Peter Walter Dorit Merhof Institute of Imaging and Computer Vision RWTH Aachen University Germany NeuroTX Aachen e.V Germany Department of Ophthalmology RWTH Aachen University Germany Institute of Image Analysis and Computer Vision University of Regensburg Germany

关键词：

Towards Monocular Shape from Refraction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sulc, Antonin Sato, Imari Goldluecke, Bastian Treibitz, Tali Computer Vision and Image Analysis University of Konstanz Konstanz Germany National Institute of Informatics Tokyo Japan Charney School of Marine Sciences University of Haifa Haifa Israel

Refraction is a common physical phenomenon and has long been researched in computer vision. Objects imaged through a refractive object appear distorted in the image as a function of the shape of the interface between the media. This hinders many computer vision applications, but can be utilized for obtaining the geometry of the refractive interface. Previous approaches for refractive surface recovery largely relied on various priors or additional information like multiple images of the analyzed surface. In contrast, we claim that a simple energy function based on Snell’s law enables the reconstruction of an arbitrary refractive surface geometry using just a single image and known background texture and geometry. In the case of a single point, Snell’s law has two degrees of freedom, therefore to estimate a surface depth, we need additional information. We show that solving for an entire surface at once introduces implicit parameter-free spatial regularization and yields convincing results when an intelligent initial guess is provided. We demonstrate our approach through simulations and real-world experiments, where the reconstruction shows encouraging results in the single-frame monocular setting. © 2023, CC BY.

关键词： Refraction

Tissue concepts: Supervised foundation models in computational pathology

学校读者我要写书评

暂无评论

computers in Biology and Medicine 2025年 186卷 109621-109621页

作者： Nicke, Till Schäfer, Jan Raphael Höfener, Henning Feuerhake, Friedrich Merhof, Dorit Kießling, Fabian Lotz, Johannes Fraunhofer Institute for Digital Medicine MEVIS Bremen/Lübeck/Aachen Germany Institute of Image Analysis and Computer Vision University of Regensburg Regensburg Germany Institute for Pathology Hannover Medical School Hannover Germany Institute for Experimental Molecular Imaging RWTH Aachen University Aachen Germany Institute of Neuropathology Medical Center – University of Freiburg Freiburg Germany

Due to the increasing workload of pathologists, the need for automation to support diagnostic tasks and quantitative biomarker evaluation is becoming more and more apparent. Foundation models have the potential to improve generalizability within and across centers and serve as starting points for data efficient development of specialized yet robust AI models. However, the training of foundation models themselves is usually very expensive in terms of data, computation, and time. This paper proposes a supervised training method that drastically reduces these expenses. The proposed method is based on multi-task learning to train a joint encoder, by combining 16 different classification, segmentation, and detection tasks on a total of 912,000 patches. Since the encoder is capable of capturing the properties of the samples, we term it the Tissue Concepts encoder. To evaluate the performance and generalizability of the Tissue Concepts encoder across centers, classification of whole slide images from four of the most prevalent solid cancers – breast, colon, lung, and prostate – was used. The experiments show that the Tissue Concepts model achieve comparable performance to models trained with self-supervision, while requiring only 6% of the amount of training patches. Furthermore, the Tissue Concepts encoder outperforms an imageNet pre-trained encoder on both in-domain and out-of-domain data. The pre-trained models and will be made available under https://***/FraunhoferMEVIS/MedicalMultitaskModeling. © 2025 The Authors

关键词： Self-supervised learning

Towards Monocular Shape from Refraction 32

学校读者我要写书评

暂无评论

Towards Monocular Shape from Refraction

32nd British Machine vision Conference, BMVC 2021

Refraction is a common physical phenomenon and has long been researched in computer vision. Objects imaged through a refractive object appear distorted in the image as a function of the shape of the interface between the media. This hinders many computer vision applications, but can be utilized for obtaining the geometry of the refractive interface. Previous approaches for refractive surface recovery largely relied on various priors or additional information like multiple images of the analyzed surface. In contrast, we claim that a simple energy function based on Snell's law enables the reconstruction of an arbitrary refractive surface geometry using just a single image and known background texture and geometry. In the case of a single point, Snell's law has two degrees of freedom, therefore to estimate a surface depth, we need additional information. We show that solving for an entire surface at once introduces implicit parameter-free spatial regularization and yields convincing results when an intelligent initial guess is provided. We demonstrate our approach through simulations and real-world experiments, where the reconstruction shows encouraging results in the single-frame monocular setting. © 2021. The copyright of this document resides with its authors.

关键词： Refraction

Self-supervised Few-shot Learning for Semantic Segmentation: An Annotation-free Approach

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Karimijafarbigloo, Sanaz Azad, Reza Merhof, Dorit Institute of Image Analysis and Computer Vision Faculty of Informatics and Data Science University of Regensburg Germany Faculty of Electrical Engineering and Information Technology RWTH Aachen University Aachen Germany Fraunhofer Institute for Digital Medicine MEVIS Bremen Germany

Few-shot semantic segmentation (FSS) offers immense potential in the field of medical image analysis, enabling accurate object segmentation with limited training data. However, existing FSS techniques heavily rely on annotated semantic classes, rendering them unsuitable for medical images due to the scarcity of annotations. To address this challenge, multiple contributions are proposed: First, inspired by spectral decomposition methods, the problem of image decomposition is reframed as a graph partitioning task. The eigenvectors of the Laplacian matrix, derived from the feature affinity matrix of self-supervised networks, are analyzed to estimate the distribution of the objects of interest from the support images. Secondly, we propose a novel self-supervised FSS framework that does not rely on any annotation. Instead, it adaptively estimates the query mask by leveraging the eigenvectors obtained from the support images. This approach eliminates the need for manual annotation, making it particularly suitable for medical images with limited annotated data. Thirdly, to further enhance the decoding of the query image based on the information provided by the support image, we introduce a multi-scale large kernel attention module. By selectively emphasizing relevant features and details, this module improves the segmentation process and contributes to better object delineation. Evaluations on both natural and medical image datasets demonstrate the efficiency and effectiveness of our method. Moreover, the proposed approach is characterized by its generality and model-agnostic nature, allowing for seamless integration with various deep architectures. The code is publicly available at GitHub. © 2023, CC BY.

关键词： Eigenvalues and eigenfunctions

GEM: Context-Aware Gaze EstiMation with Visual Search Behavior Matching for Chest Radiograph

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Shaonan Chen, Wenting Liu, Jie Luo, Xiaoling Shen, Linlin Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University China Department of Electrical Engineering City University of Hong Kong Hong Kong AI Research Center for Medical Image Analysis and Diagnosis Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China

Gaze estimation is pivotal in human scene comprehension tasks, particularly in medical diagnostic analysis. Eye-tracking technology facilitates the recording of physicians’ ocular movements during image interpretation, thereby elucidating their visual attention patterns and information-processing strategies. In this paper, we initially define the context-aware gaze estimation problem in medical radiology report settings. To understand the attention allocation and cognitive behavior of radiologists during the medical image interpretation process, we propose a context-aware Gaze EstiMation (GEM) network that utilizes eye gaze data collected from radiologists to simulate their visual search behavior patterns throughout the image interpretation process. It consists of a context-awareness module, visual behavior graph construction, and visual behavior matching. Within the context-awareness module, we achieve intricate multimodal registration by establishing connections between medical reports and images. Subsequently, for a more accurate simulation of genuine visual search behavior patterns, we introduce a visual behavior graph structure, capturing such behavior through high-order relationships (edges) between gaze points (nodes). To maintain the authenticity of visual behavior, we devise a visual behavior-matching approach, adjusting the high-order relationships between them by matching the graph constructed from real and estimated gaze points. Extensive experiments on four publicly available datasets demonstrate the superiority of GEM over existing methods and its strong generalizability, which also provides a new direction for the effective utilization of diverse modalities in medical image interpretation and enhances the interpretability of models in the field of medical imaging. https://***/Tiger-SN/GEM. Copyright © 2024, The Authors. All rights reserved.

关键词： Eye movements

Utnetpara: A Hybrid CNN-Transformer Architecture with Multi-Scale Fusion for Whole-Slide image Segmentation

学校读者我要写书评

暂无评论

Utnetpara: A Hybrid CNN-Transformer Architecture with Multi-...

IEEE International Symposium on Biomedical Imaging

作者： Boqiang Huang Jiayu Ying Ruizhi Lyu Nadine S. Schaadt Barbara M. Klinkhammer Peter Boor Johannes Lotz Friedrich Feuerhake Dorit Merhof Institute of Image Analysis and Computer Vision University of Regensburg Regensburg Germany Institute of Imaging and Computer Vision RWTH Aachen University Aachen Germany Institute for Pathology Hannover Medical School Hannover Germany Institute of Pathology RWTH Aachen University Hospital Aachen Germany Department of Nephrology and Immunology RWTH Aachen University Hospital Aachen Germany Fraunhofer Institute for Digital Medicine MEVIS Lübeck Germany Institute of Neuropathology Medical Center – University of Freiburg Freiburg Germany Fraunhofer Institute for Digital Medicine MEVIS Bremen Germany

ISBN: (数字)9798350313338

ISBN: (纸本)9798350313345

In medical image segmentation tasks, Convolutional Neural Networks (CNNs) have become an efficient and successful solution, although they have limitations in explicitly modeling long-term dependencies. The Transformer neural network has recently demonstrated its capabilities in image segmentation, although a large amount of data is required for training. In this study, we present a hybrid architecture, UTNetPara, that integrates the Transformer into a U-shaped CNN to improve segmentation accuracy on a medium-sized dataset. Self-attention modules are applied in both the encoder and decoder to enhance the ability to capture long-term dependencies at different scales. Efficient self-attention mechanisms with relative position encoding are employed to reduce the computational cost accordingly. A fully annotated dataset consisting of whole slide images scanned from periodic acid-Schiff stained mouse kidney tissue is used for evaluation. The proposed method is trained to segment the main renal structures: glomerular tuft, glomerulus including Bowman's capsule, tubules, arteries, arterial lumina, and veins. Our experimental results indicate that the UTNetPara has a better segmentation performance than other state-of-the-art models.

关键词： Training image segmentation Veins Neural networks computer architecture Transformers Convolutional neural networks

TISSUE CONCEPTS: SUPERVISED FOUNDATION MODELS IN COMPUTATIONAL PATHOLOGY

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Nicke, Till Schäfer, Jan Raphael Höfener, Henning Feuerhake, Friedrich Merhof, Dorit Kießling, Fabian Lotz, Johannes Fraunhofer Institute for Digital Medicine MEVIS Bremen Germany Fraunhofer Institute for Digital Medicine MEVIS Lübeck Germany Fraunhofer Institute for Digital Medicine MEVIS Aachen Germany Institute for Pathology Hannover Medical School Hannover Germany Institute of Image Analysis and Computer Vision University of Regensburg Regensburg Germany Institute for Experimental Molecular Imaging RWTH Aachen University Aachen Germany

Due to the increasing workload of pathologists, the need for automation to support diagnostic tasks and quantitative biomarker evaluation is becoming more and more apparent. Foundation models have the potential to improve generalizability within and across centers and serve as starting points for data efficient development of specialized yet robust AI models. However, the training foundation models themselves is usually very expensive in terms of data, computation, and time. This paper proposes a supervised training method that drastically reduces these expenses. The proposed method is based on multi-task learning to train a joint encoder, by combining 16 different classification, segmentation, and detection tasks on a total of 912,000 patches. Since the encoder is capable of capturing the properties of the samples, we term it the Tissue Concepts encoder. To evaluate the performance and generalizability of the Tissue Concepts encoder across centers, classification of whole slide images from four of the most prevalent solid cancers - breast, colon, lung, and prostate - was used. The experiments show that the Tissue Concepts model achieve comparable performance to models trained with self-supervision, while requiring only 6% of the amount of training patches. Furthermore, the Tissue Concepts encoder outperforms an imageNet pre-trained encoder on both in-domain and out-of-domain data. The pre-trained models and will be made available under https://***/FraunhoferMEVIS/MedicalMultitaskModeling. © 2024, CC BY-NC-ND.

关键词： Multi-task learning