检索结果-内蒙古大学图书馆

On Moving Object Segmentation from Monocular Video with Transformers

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Homeyer, Christian Schnörr, Christoph Robert Bosch GmbH Corporate Research Computer Vision Lab Hildesheim Germany Image and Pattern Analysis Group Heidelberg University Germany

Moving object detection and segmentation from a single moving camera is a challenging task, requiring an understanding of recognition, motion and 3D geometry. Combining both recognition and reconstruction boils down to a fusion problem, where appearance and motion features need to be combined for classification and segmentation. In this paper, we present a novel fusion architecture for monocular motion segmentation - M3Former, which leverages the strong performance of transformers for segmentation and multi-modal fusion. As reconstructing motion from monocular video is ill-posed, we systematically analyze different 2D and 3D motion representations for this problem and their importance for segmentation performance. Finally, we analyze the effect of training data and show that diverse datasets are required to achieve SotA performance on Kitti and Davis. Copyright © 2024, The Authors. All rights reserved.

关键词： Object detection

Neuroexplicit Diffusion Models for Inpainting of Optical Flow Fields

学校读者我要写书评

暂无评论

arXiv 2024年

Deep learning has revolutionized the field of computer vision by introducing large scale neural networks with millions of parameters. Training these networks requires massive datasets and leads to in-transparent models that can fail to generalize. At the other extreme, models designed from partial differential equations (PDEs) embed specialized domain knowledge into mathematical equations and usually rely on few manually chosen hyperparameters. This makes them transparent by construction and if designed and calibrated carefully, they can generalize well to unseen scenarios. In this paper, we show how to bring model- and data-driven approaches together by combining the explicit PDE-based approaches with convolutional neural networks to obtain the best of both worlds. We illustrate a joint architecture for the task of inpainting optical flow fields and show that the combination of model- and data-driven modeling leads to an effective architecture. Our model outperforms both fully explicit and fully data-driven baselines in terms of reconstruction quality, robustness and amount of required training data. Averaging the endpoint error across different mask densities, our method outperforms the explicit baselines by 11−27%, the GAN baseline by 47% and the Probabilisitic Diffusion baseline by 42%. With that, our method sets a new state of the art for inpainting of optical flow fields from random masks. © 2024, CC BY.

关键词： Network architecture

Neuroexplicit diffusion models for inpainting of optical flow fields 24

学校读者我要写书评

暂无评论

Neuroexplicit diffusion models for inpainting of optical flo...

Proceedings of the 41st International Conference on Machine Learning

作者： Tom Fischer Pascal Peter Joachim Weickert Eddy Ilg Computer Vision and Machine Perception Lab Saarland University Saarbrücken Germany Mathematical Image Analysis Group Saarland University Saarbrücken Germany

Deep learning has revolutionized the field of computer vision by introducing large scale neural networks with millions of parameters. Training these networks requires massive datasets and leads to intransparent models that can fail to generalize. At the other extreme, models designed from partial differential equations (PDEs) embed specialized domain knowledge into mathematical equations and usually rely on few manually chosen hyper-parameters. This makes them transparent by construction and if designed and calibrated carefully, they can generalize well to unseen scenarios. In this paper, we show how to bring model- and data-driven approaches together by combining the explicit PDE-based approaches with convolutional neural networks to obtain the best of both worlds. We illustrate a joint architecture for the task of inpainting optical flow fields and show that the combination of model- and data-driven modeling leads to an effective architecture. Our model outperforms both fully explicit and fully data-driven baselines in terms of reconstruction quality, robustness and amount of required training data. Averaging the endpoint error across different mask densities, our method outperforms the explicit baselines by 11-27%, the GAN baseline by 47% and the Probabilisitic Diffusion baseline by 42%. With that, our method sets a new state of the art for inpainting of optical flow fields from random masks.

关键词：

On Moving Object Segmentation from Monocular Video with Transformers

学校读者我要写书评

暂无评论

On Moving Object Segmentation from Monocular Video with Tran...

International Conference on computer vision Workshops (ICCV Workshops)

作者： Christian Homeyer Christoph Schnörr Robert Bosch GmbH Corporate Research Computer Vision Lab Hildesheim Germany Image and Pattern Analysis Group Heidelberg University Germany Image and Pattern Analysis Group Heidelberg University Germany

Moving object detection and segmentation from a single moving camera is a challenging task, requiring an understanding of recognition, motion and 3D geometry. Combining both recognition and reconstruction boils down to a fusion problem, where appearance and motion features need to be combined for classification and *** this paper, we present a novel fusion architecture for monocular motion segmentation - M 3 Former, which leverages the strong performance of transformers for segmentation and multi-modal fusion. As reconstructing motion from monocular video is ill-posed, we systematically analyze different 2D and 3D motion representations for this problem and their importance for segmentation performance. Finally, we analyze the effect of training data and show that diverse datasets are required to achieve SotA performance on Kitti and Davis. Code will be released upon publication.

关键词：

Inspection Method of Track Facilities Using image analysis of images Taken from Front of Trains

学校读者我要写书评

暂无评论

Quarterly Report of RTRI (Railway Technical Research Institute) 2023年第3期64卷 205-210页

作者： Nagamine, Nozomi Goda, Wataru Maeda, Riho Tsubokawa, Yosuke Kato, So Itoi, Kensuke Head of Image Analysis Laboratory Information and Communication Technology Division Research Areas: Computer Vision Image Processing Signalling Systems Japan Image Analysis Laboratory Information and Communication Technology Division Research Areas: Computer Vision Image Processing Japan Head of Track Geometry and Maintenance Laboratory Track Technology Division Research Areas: Track Measuring Japan Track Geometry and Maintenance Laboratory Track Technology Division Research Areas: Track Measuring Japan Research Areas: Track Measuring Japan

Appropriate maintenance of tracks is vital for the safe operation of railways. Properly managing track facilities is necessary to prevent buckling of rails. Changes in society are creating a shortage of workers, a decrease in skilled engineers, a decrease in passenger income, and an increase in the need to develop low-cost inspection methods which do not require experience. Therefore, we have developed a method using only inexpensive camcorders, which can extract information from images to detect joint gaps, wooden sleeper deterioration and ballast shape, and estimate the kilometer point of these images to locate the installations in question. This paper describes the outline of the method and its application results. © 2023 Ken-yusha Inc.. All rights reserved.

关键词： Inspection

Unsupervised Latent Stain Adaptation for Computational Pathology

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Reisenbüchler, Daniel Luttner, Lucas Schaadt, Nadine S. Feuerhake, Friedrich Merhof, Dorit Institute of Image Analysis and Computer Vision University of Regensburg Regensburg Germany Institute of Pathology Hannover Medical School Hannover Germany Fraunhofer Institute for Digital Medicine MEVIS Bremen Germany

In computational pathology, deep learning (DL) models for tasks such as segmentation or tissue classification are known to suffer from domain shifts due to different staining techniques. Stain adaptation aims to reduce the generalization error between different stains by training a model on source stains that generalizes to target stains. Despite the abundance of target stain data, a key challenge is the lack of annotations. To address this, we propose a joint training between artificially labeled and unlabeled data including all available stained images called Unsupervised Latent Stain Adaptation (ULSA). Our method uses stain translation to enrich labeled source images with synthetic target images in order to increase the supervised signals. Moreover, we leverage unlabeled target stain images using stain-invariant feature consistency learning. With ULSA we present a semi-supervised strategy for efficient stain adaptation without access to annotated target stain data. Remarkably, ULSA is task agnostic in patch-level analysis for whole slide images (WSIs). Through extensive evaluation on external datasets, we demonstrate that ULSA achieves state-of-the-art (SOTA) performance in kidney tissue segmentation and breast cancer classification across a spectrum of staining variations. Our findings suggest that ULSA is an important framework for stain adaptation in computational pathology. © 2024, CC BY.

关键词： image segmentation

A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Yuli Karetić, Ivan Stegmaier, Johannes Walter, Peter Merhof, Dorit Institute of Imaging and Computer Vision RWTH Aachen University Germany NeuroTX Aachen e.V. Germany Department of Ophthalmology RWTH Aachen University Germany Institute of Image Analysis and Computer Vision University of Regensburg Germany

We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is trained to mimic the biomimetic perceptual model implemented in pulse2percept. The evaluator is a shallow VGG classifier, which is trained with original images. Based on 10,000 test images from the MNIST dataset, we show that the convolutional neural network-based encoder performs significantly better than the trivial downsampling approach, yielding a boost in the weighted F1-Score by 36.17% in the pre-trained classifier with 6×10 electrodes. With this fully neural network-based encoder, the quality of the downstream perceptions can be fine-tuned using gradient descent in an end-to-end fashion. Copyright © 2023, The Authors. All rights reserved.

关键词： Ophthalmology

A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

学校读者我要写书评

暂无评论

A Deep Learning-based in silico Framework for Optimization o...

Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

作者： Yuli Wu Ivan Karetić Johannes Stegmaier Peter Walter Dorit Merhof Institute of Imaging and Computer Vision RWTH Aachen University Germany NeuroTX Aachen e.V Germany Department of Ophthalmology RWTH Aachen University Germany Institute of Image Analysis and Computer Vision University of Regensburg Germany

关键词：

Towards Monocular Shape from Refraction

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sulc, Antonin Sato, Imari Goldluecke, Bastian Treibitz, Tali Computer Vision and Image Analysis University of Konstanz Konstanz Germany National Institute of Informatics Tokyo Japan Charney School of Marine Sciences University of Haifa Haifa Israel

Refraction is a common physical phenomenon and has long been researched in computer vision. Objects imaged through a refractive object appear distorted in the image as a function of the shape of the interface between the media. This hinders many computer vision applications, but can be utilized for obtaining the geometry of the refractive interface. Previous approaches for refractive surface recovery largely relied on various priors or additional information like multiple images of the analyzed surface. In contrast, we claim that a simple energy function based on Snell’s law enables the reconstruction of an arbitrary refractive surface geometry using just a single image and known background texture and geometry. In the case of a single point, Snell’s law has two degrees of freedom, therefore to estimate a surface depth, we need additional information. We show that solving for an entire surface at once introduces implicit parameter-free spatial regularization and yields convincing results when an intelligent initial guess is provided. We demonstrate our approach through simulations and real-world experiments, where the reconstruction shows encouraging results in the single-frame monocular setting. © 2023, CC BY.

关键词： Refraction