检索结果-内蒙古大学图书馆

Deformable One-shot Face Stylization via DINO Semantic Guidance

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Yang Chen, Zichong Huang, Hui Visual Computing Research Center Shenzhen University China

This paper addresses the complex issue of one-shot face stylization, focusing on the simultaneous consideration of appearance and structure, where previous methods have fallen short. We explore deformation-aware face stylization that diverges from traditional single-image style reference, opting for a real-style image pair instead. The cornerstone of our method is the utilization of a self-supervised vision transformer, specifically DINO-ViT, to establish a robust and consistent facial structure representation across both real and style domains. Our stylization process begins by adapting the StyleGAN generator to be deformation-aware through the integration of spatial transformers (STN). We then introduce two innovative constraints for generator fine-tuning under the guidance of DINO semantics: i) a directional deformation loss that regulates directional vectors in DINO space, and ii) a relative structural consistency constraint based on DINO token self-similarities, ensuring diverse generation. Additionally, style-mixing is employed to align the color generation with the reference, minimizing inconsistent correspondences. This framework delivers enhanced deformability for general one-shot face stylization, achieving notable efficiency with a fine-tuning duration of approximately 10 minutes. Extensive qualitative and quantitative comparisons demonstrate our superiority over state-of-the-art one-shot face stylization methods. Code is available at https://***/zichongc/DoesFS. © 2024, CC BY-NC-SA.

关键词： Semantics

OIL: Observational imitation learning

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Li, Guohao Müller, Matthias Casser, Vincent Smith, Neil Michels, Dominik L. Ghanem, Bernard Visual Computing Center KAUST Thuwal Saudi Arabia

Recent work has explored the problem of autonomous navigation by imitating a teacher and learning an end-to-end policy, which directly predicts controls from raw images. However, these approaches tend to be sensitive to mistakes by the teacher and do not scale well to other environments or vehicles. To this end, we propose Observational Imitation Learning (OIL), a novel imitation learning variant that supports online training and automatic selection of optimal behavior by observing multiple imperfect teachers. We apply our proposed methodology to the challenging problems of autonomous driving and UAV racing. For both tasks, we utilize the Sim4CV simulator [23] that enables the generation of large amounts of synthetic training data and also allows for online learning and evaluation. We train a perception network to predict waypoints from raw image data and use OIL to train another network to predict controls from these waypoints. Extensive experiments demonstrate that our trained network outperforms its teachers, conventional imitation learning (IL) and reinforcement learning (RL) baselines and even humans in simulation. Copyright © 2018, The Authors. All rights reserved.

关键词： Reinforcement learning

Hyperquadrics for shape analysis of 3D nanoscale reconstructions of brain cell nuclear envelopes

学校读者我要写书评

暂无评论

Hyperquadrics for shape analysis of 3D nanoscale reconstruct...

2018 Italian Chapter Conference - Smart Tools and Apps in Computer Graphics, STAG 2018

作者： Agus, M. Calì, C. Tapia Morales, A. Lehväslaiho, H.O. Magistretti, P.J. Gobbetti, E. Hadwiger, M. Thuwal23955-6900 Saudi Arabia Thuwal23955-6900 Saudi Arabia Visual Computing Group Cagliari Italy CSC IT Center for Science Espoo Finland

ISBN: (纸本)9783038680758

Shape analysis of cell nuclei is becoming increasingly important in biology and medicine. Recent results have identified that the significant variability in shape and size of nuclei has an important impact on many biological processes. Current analysis techniques involve automatic methods for detection and segmentation of histology and microscopy images, and are mostly performed in 2D. Methods for 3D shape analysis, made possible by emerging acquisition methods capable to provide nanometric-scale 3D reconstructions, are, however, still at an early stage, and often assume a simple spherical shape. We introduce here a framework for analyzing 3D nanoscale reconstructions of nuclei of brain cells (mostly neurons), obtained by semiautomatic segmentation of electron micrographs. Our method considers an implicit parametric representation customizing the hyperquadrics formulation of convex shapes. Point clouds of nuclear envelopes, extracted from image data, are fitted to our parametrized model, which is then used for performing statistical analysis and shape comparisons. We report on the preliminary analysis of a collection of 92 nuclei of brain cells obtained from a sample of the somatosensory cortex of a juvenile rat. © 2018 The Eurographics Association.

关键词： Cytology

DeepGCNs: Making GCNs go as deep as CNNs

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Li, Guohao Müller, Matthias Qian, Guocheng Delgadillo, Itzel C. Abualshour, Abdulellah Thabet, Ali Ghanem, Bernard Visual Computing Center KAUST Thuwal Saudi Arabia

Convolutional Neural Networks (CNNs) have been very successful at solving a variety of computer vision tasks such as object classification and detection, semantic segmentation, activity understanding, to name just a few. One key enabling factor for their great performance has been the ability to train very deep networks. Despite their huge success in many tasks, CNNs do not work well with non-Euclidean data, which is prevalent in many real-world applications. Graph Convolutional Networks (GCNs) offer an alternative that allows for non-Eucledian data input to a neural network. While GCNs already achieve encouraging results, they are currently limited to architectures with a relatively small number of layers, primarily due to vanishing gradients during training. This work transfers concepts such as residual/dense connections and dilated convolutions from CNNs to GCNs in order to successfully train very deep GCNs. We show the benefit of using deep GCNs (with as many as 112 layers) experimentally across various datasets and tasks. Specifically, we achieve very promising performance in part segmentation and semantic segmentation on point clouds and in node classification of protein functions across biological protein-protein interaction (PPI) graphs. We believe that the insights in this work will open avenues for future research on GCNs and their application to further tasks not explored in this paper. The source code for this work is available at https://***/lightaime/deep_gcns_torch and https://***/lightaime/deep_gcns for PyTorch and TensorFlow implementation respectively. Copyright © 2019, The Authors. All rights reserved.

关键词： Convolution

Uncer2Natural: Uncertainty-Aware Unsupervised Image Denoising 48

学校读者我要写书评

暂无评论

Uncer2Natural: Uncertainty-Aware Unsupervised Image Denoisin...

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Huang, Chenyu Tan, Weimin Shi, Jiaxing Xing, Zhen Yan, Bo Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai China

ISBN: (纸本)9781728163277

Recently, unsupervised image denoising methods learning from paired noisy samples have received increasing attention. These methods build on the idea that the mean of multiple noisy images of the same scene is the ideal clean image. However, these methods ignore the effect of Aleatoric uncertainty in the noisy image (e.g., pixels deviating from the expected distribution). The presence of Aleatoric uncertainty causes degradation of the reconstructed target pixels, resulting in high uncertainty for these pixels (i.e., low confidence), which in turn leads to sub-optimal denoising results. To address this problem, we propose a novel uncertainty-aware unsupervised image denoising method named Uncer2Natural (U2N). It dynamically predicts the Aleatoric uncertainty for each noisy sample and produces satisfactory denoising results by reducing the effect of Aleatoric uncertainty. Extensive experimental results show that U2N outperforms state-of-the- art unsupervised image denoising methods in terms of both quantitative metrics and qualitative visual quality. © 2023 IEEE.

关键词： Pixels

Simulating fire with texture splats

学校读者我要写书评

暂无评论

Simulating fire with texture splats

VIS 2002, IEEE visualisation 2002

作者： Wei, Xiaoming Li, Wei Mueller, Klaus Kaufman, Arie Center for Visual Computing Department of Computer Science State Univ. of New York Stony Brook Stony Brook NY 11794-4400 United States

We propose the use of textured splats as the basic display primitives for an open surface fire model. The high-detail textures help to achieve a smooth boundary of the fire and gain the small-scale turbulence appearance. We utilize the Lattice Boltzmann Model (LBM) to simulate physically-based equations describing the fire evolution and its interaction with the environment (e.g., obstacles, wind and temperature). The property of fuel and non-burning objects are defined on the lattice of the computation domain. A temperature field is also incorporated to model the generation of smoke from the fire due to incomplete combustion. The linear and local characteristics of the LBM enable us to accelerate the computation with graphics hardware to reach real-time simulation speed, while the texture splat primitives enable interactive rendering frame rates.

关键词： Computer graphics

DeeperGCN: All You Need to Train Deeper GCNs

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Guohao Xiong, Chenxin Thabet, Ali Ghanem, Bernard Visual Computing Center KAUST Thuwal Saudi Arabia

Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. These challenges limit the representation power of GCNs on large-scale graphs. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs. We define differentiable generalized aggregation functions to unify different message aggregation operations (e.g. mean, max). We also propose a novel normalization layer namely MsgNorm and a pre-activation version of residual connections for GCNs. Extensive experiments on Open Graph Benchmark (OGB) show DeeperGCN significantly boosts performance over the state-of-the-art on the large scale graph learning tasks of node property prediction and graph property prediction. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolutional neural networks

Deepgcns: Can gcns go as deep as cnns?

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Li, Guohao Müller, Matthias Thabet, Ali Ghanem, Bernard Visual Computing Center KAUST Thuwal Saudi Arabia

Convolutional Neural Networks (CNNs) achieve impressive performance in a wide variety of fields. Their success benefited from a massive boost when very deep CNN models were able to be reliably trained. Despite their merits, CNNs fail to properly address problems with non-Euclidean data. To overcome this challenge, Graph Convolutional Networks (GCNs) build graphs to represent non-Euclidean data, borrow concepts from CNNs, and apply them in training. GCNs show promising results, but they are usually limited to very shallow models due to the vanishing gradient problem (see Figure 1). As a result, most state-of-the-art GCN models are no deeper than 3 or 4 layers. In this work, we present new ways to successfully train very deep GCNs. We do this by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures. Extensive experiments show the positive effect of these deep GCN frameworks. Finally, we use these new concepts to build a very deep 56-layer GCN, and show how it significantly boosts performance (+3:7% mIoU over state-of-the-art) in the task of point cloud semantic segmentation. We believe that the community can greatly benefit from this work, as it opens up many opportunities for advancing GCN-based research. Copyright © 2019, The Authors. All rights reserved.

关键词： Semantics

Motion Matters: Difference-based Multi-scale Learning for Infrared UAV Detection

学校读者我要写书评

暂无评论

Motion Matters: Difference-based Multi-scale Learning for In...

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023

作者： He, Ruian Zhou, Shili Cheng, Ri Sun, Yuqi Tan, Weimin Yan, Bo Shanghai Collaborative Innovation Center of Intelligent Visual Computing Fudan University School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Shanghai China

ISBN: (纸本)9798350302493

Unmanned Aerial Vehicle (UAV) detection in the wild is a challenging task due to the presence of background noise and the varying size of the object. To address these obstacles, we propose a novel learning framework for robust UAV detectors, which we call Difference-based Multi-scale Learning (DML). We argue that motion information matters in UAV detection because of the low recognition in one frame. Our method utilizes the frame difference of multiple previous frames, extracting motion information and blocking background noise. We also fuse multiple spatial-temporal scales for training and inferencing, enabling fusion from different sources. In addition, to better evaluate the performance of UAV detection in different scales, we propose Multi-Scale Average Precision (MSAP) metric to aggregate the detection accuracy over multiple scales. Through extensive experiments, we demonstrate that our proposed approach improves the detection accuracy of baseline models. Notably, we achieve SOTA performance in the 3rd Anti-UAV Challenge, with 2nd place in Track 2 and 4th place in Track 1. © 2023 IEEE.

关键词： Antennas