检索结果-内蒙古大学图书馆

PoseBias: On Dataset Bias and Task Difficulty - Is there an Optimal Camera Position for Facial Image Analysis?

学校读者我要写书评

暂无评论

PoseBias: On Dataset Bias and Task Difficulty - Is there an ...

International Conference on computer vision Workshops (ICCV Workshops)

作者： Mohit Choithwani Sneha Almeida Bernhard Egger Cognitive Computer Vision Lab (CogCoVi) Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

Let's imagine you could choose the position of the camera for a particular face analysis task - where would you put it? In this work, we provide a first analysis based on synthetic training data to provide evidence that this choice is not trivial, not only dependent on the training data and different based on the task. We provide results for two major face analysis tasks, face recognition and landmark detection. For our experiments, we use a 3D Morphable Model as it provides us full control over pose, illumination, and identity to generate idealized training data. Whilst rendered images are not photorealistic we avoid any confounding factors and biases from other sources (e.g. pose bias in training data).Our results show that the optimal camera poses are near frontal but not exactly frontal and dependent on the task. By comparing the results obtained by pose-specific training set to a uniform training distribution without pose bias we show that the accuracy for both tasks not only depends on the bias in the training data but is actually dominated by the difficulty of the task depending on the particular pose.

关键词：

ReWaRD: Retinal Waves for Pre-Training Artificial Neural Networks Mimicking Real Prenatal Development

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Cappell, Benjamin Stoll, Andreas Umah, Williams Chukwudi Egger, Bernhard Cognitive Computer Vision Lab Chair of Visual Computing Friedrich-Alexander-Universität Erlangen-Nürnberg Germany

Computational models trained on a large amount of natural images are the state-of-the-art to study human vision – usually adult vision. Computational models of infant vision and its further development are gaining more and more attention in the community. In this work we aim at the very beginning of our visual experience – pre- and post-natal retinal waves which suggest to be a pre-training mechanism for the primate visual system at a very early stage of development. We see this approach as an instance of biologically plausible data driven inductive bias through pre-training. We built a computational model that mimics this development mechanism by pre-training different artificial convolutional neural networks with simulated retinal wave images. The resulting features of this biologically plausible pre-training closely match the V1 features of the primate visual system. We show that the performance gain by pre-training with retinal waves is similar to a state-of-the art pre-training pipeline. Our framework contains the retinal wave generator, as well as a training strategy, which can be a first step in a curriculum learning based training diet for various models of development. We release code, data and trained networks to build the basis for future work on visual development and based on a curriculum learning approach including prenatal development to support studies of innate vs. learned properties of the primate visual system. An additional benefit of our pre-trained networks for neuroscience or computer vision applications is the absence of biases inherited from datasets like ImageNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Ophthalmology

RENI++: A Rotation-Equivariant, Scale-Invariant, Natural Illumination Prior

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gardner, James A.D. Egger, Bernhard Smith, William A.P. The Department of Computer Science University of York York United Kingdom The Cognitive Computer Vision Lab Friedrich-Alexander-Universität Erlangen-Nürnberg Erlangen Germany

Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representations and, at best, a simplistic prior on the parameters. This results in limitations for the inverse setting in terms of the expressivity of the illumination conditions, especially when taking specular reflections into account. We propose a conditional neural field representation based on a variational auto-decoder and a transformer decoder. We extend Vector Neurons to build equivariance directly into our architecture, and leveraging insights from depth estimation through a scale-invariant loss function, we enable the accurate representation of High Dynamic Range (HDR) images. The result is a compact, rotation-equivariant HDR neural illumination model capable of capturing complex, high-frequency features in natural environment maps. Training our model on a curated dataset of 1.6K HDR environment maps of natural scenes, we compare it against traditional representations, demonstrate its applicability for an inverse rendering task and show environment map completion from partial observations. We share our PyTorch implementation, dataset and trained models at ***/JADGardner/ns_reni. © 2023, CC BY.

关键词： Inverse problems

Rotation-equivariant conditional spherical neural fields for learning a natural illumination prior 22

学校读者我要写书评

暂无评论

Rotation-equivariant conditional spherical neural fields for...

Proceedings of the 36th International Conference on Neural Information Processing Systems

作者： James A. D. Gardner Bernhard Egger William A. P. Smith Department of Computer Science University of York York United Kingdom Cognitive Computer Vision Lab Friedrich-Alexander-Universität Erlangen Germany

ISBN: (纸本)9781713871088

Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representations and, at best, a simplistic prior on the parameters. We propose a conditional neural field representation based on a variational auto-decoder with a SIREN network and, extending Vector Neurons, build equivariance directly into the network. Using this, we develop a rotation-equivariant, high dynamic range (HDR) neural illumination model that is compact and able to express complex, high-frequency features of natural environment maps. Training our model on a curated dataset of 1.6K HDR environment maps of natural scenes, we compare it against traditional representations, demonstrate its applicability for an inverse rendering task and show environment map completion from partial observations. A PyTorch implementation, our dataset and trained models can be found at ***/RENI.

关键词：

Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Gardner, James A.D. Egger, Bernhard Smith, William A.P. Department of Computer Science University of York York United Kingdom Cognitive Computer Vision Lab Friedrich-Alexander-Universität Erlangen Germany

Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representations and, at best, a simplistic prior on the parameters. We propose a conditional neural field representation based on a variational auto-decoder with a SIREN network and, extending Vector Neurons, build equivariance directly into the network. Using this, we develop a rotation-equivariant, high dynamic range (HDR) neural illumination model that is compact and able to express complex, high-frequency features of natural environment maps. Training our model on a curated dataset of 1.6K HDR environment maps of natural scenes, we compare it against traditional representations, demonstrate its applicability for an inverse rendering task and show environment map completion from partial observations. A PyTorch implementation, our dataset and trained models can be found at ***/RENI. © 2022, CC BY.

关键词： Inverse problems

The Sky’s the Limit: Relightable Outdoor Scenes via a Sky-pixel Constrained Illumination Prior and Outside-In Visibility

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gardner, James A.D. Kashin, Evgenii Egger, Bernhard Smith, William A.P. Department of Computer Science The University of York YorkYO10 5DD United Kingdom Cognitive Computer Vision Lab Friedrich-Alexander-Universität Erlangen-Nürnberg Erlangen Germany

Inverse rendering of outdoor scenes from unconstrained image collections is a challenging task, particularly illumination/albedo ambiguities and occlusion of the illumination environment (shadowing) caused by geometry. However, there are many cues in an image that can aid in the disentanglement of geometry, albedo and shadows. Whilst sky is frequently masked out in state-of-the-art methods, we exploit the fact that any sky pixel provides a direct observation of distant lighting in the corresponding direction and, via a neural illumination prior, a statistical cue to derive the remaining illumination environment. The incorporation of our illumination prior is enabled by a novel ‘outside-in’ method for computing differentiable sky visibility based on a neural directional distance function. This is highly efficient and can be trained in parallel with the neural scene representation, allowing gradients from appearance loss to flow from shadows to influence the estimation of illumination and geometry. Our method estimates high-quality albedo, geometry, illumination and sky visibility, achieving state-of-the-art results on the NeRF-OSR relighting benchmark. Our code and models can be found at https://***/JADGardner/neusky. © 2023, CC BY.

关键词： Geometry

Towards Context-Aware Domain Generalization: Understanding the Benefits and Limits of Marginal Transfer Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Müller, Jens Kühmichel, Lars Rohbeck, Martin Radev, Stefan T. Köthe, Ullrich Computer Vision and Learning Lab Heidelberg University Germany Division of Computational Genomics and Systems Genetics DKFZ Germany Department of Cognitive Science Rensselaer Polytechnic Institute NY United States

In this work, we analyze the conditions under which information about the context of an input X can improve the predictions of deep learning models in new domains. Following work in marginal transfer learning in Domain Generalization (DG), we formalize the notion of context as a permutation-invariant representation of a set of data points that originate from the same domain as the input itself. We offer a theoretical analysis of the conditions under which this approach can, in principle, yield benefits, and formulate two necessary criteria that can be easily verified in practice. Additionally, we contribute insights into the kind of distribution shifts for which the marginal transfer learning approach promises robustness. Empirical analysis shows that our criteria are effective in discerning both favorable and unfavorable scenarios. Finally, we demonstrate that we can reliably detect scenarios where a model is tasked with unwarranted extrapolation in out-of-distribution (OOD) domains, identifying potential failure cases. Consequently, we showcase a method to select between the most predictive and the most robust model, circumventing the well-known trade-off between predictive performance and robustness. © 2023, CC BY.

关键词： Deep learning

Convolutional Cross-View Pose Estimation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xia, Zimin Booij, Olaf Kooij, Julian F.P. The Intelligent Vehicles Group Department of Cognitive Robotics Delft University of Technology Leeghwaterstraat Delft2628 CN Netherlands The Computer Vision Lab Delft University of Technology Mekelweg 4 Delft2628 CD Netherlands

We propose a novel end-to-end method for cross-view pose estimation. Given a ground-level query image and an aerial image that covers the query’s local neighborhood, the 3 Degrees-of-Freedom camera pose of the query is estimated by matching its image descriptor to descriptors of local regions within the aerial image. The orientation-aware descriptors are obtained by using a translationally equivariant convolutional ground image encoder and contrastive learning. The Localization Decoder produces a dense probability distribution in a coarse-to-fine manner with a novel Localization Matching Upsampling module. A smaller Orientation Decoder produces a vector field to condition the orientation estimate on the localization. Our method is validated on the VIGOR and KITTI datasets, where it surpasses the state-of-the-art baseline by 72% and 36% in median localization error for comparable orientation estimation accuracy. The predicted probability distribution can represent localization ambiguity, and enables rejecting possible erroneous predictions. Without re-training, the model can infer on ground images with different field of views and utilize orientation priors if available. On the Oxford RobotCar dataset, our method can reliably estimate the ego-vehicle’s pose over time, achieving a median localization error under 1 meter and a median orientation error of around 1 degree at 14 FPS. Copyright © 2023, The Authors. All rights reserved.

关键词： Cameras

Object Detection, Recognition, Deep Learning, and the Universal Law of Generalization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Rustom, Faris B. Öğmen, Haluk Yazdanbakhsh, Arash Department of Psychological and Brain Sciences Computational Neuroscience and Vision Lab Center for Systems Neuroscience Boston University BostonMA United States Department of Electrical & Computer Engineering Laboratory of Perceptual and Cognitive Dynamics University of Denver DenverCO United States

Object detection and recognition are fundamental functions underlying the success of species. Because the appearance of an object exhibits a large variability, the brain has to group these different stimuli under the same object identity, a process of generalization. Does the process of generalization follow some general principles or is it an ad-hoc "bag-of-tricks"? The Universal Law of Generalization provided evidence that generalization follows similar properties across a variety of species and tasks. Here we test the hypothesis that the internal representations underlying generalization reflect the natural properties of object detection and recognition in our environment rather than the specifics of the system solving these problems. By training a deep-neural-network with images of "clear" and "camouflaged" animals, we found that, with a proper choice of category prototypes, the generalization functions are monotone decreasing, similar to the generalization functions of biological systems. Our findings support the hypothesis of the study. Copyright © 2022, The Authors. All rights reserved.

关键词： Object detection