检索结果-内蒙古大学图书馆

Achieving Efficient and Realistic Full-Radar Simulations and Automatic Data Annotation by Exploiting Ray Meta Data from a Radar Ray Tracing Simulator

学校读者我要写书评

暂无评论

Achieving Efficient and Realistic Full-Radar Simulations and...

IEEE National Conference on Radar

作者： Christian Schüβler Marcel Hoffmann Vanessa Wirth Björn Eskofier Tim Weyrich Marc Stamminger Martin Vossiek Institute of Microwaves and Photonics Friedrich-Alexander-Universität Erlangen-Nürnberg Erlangen Germany Chair of Visual Computing Machine Learning and Data Analytics Lab Chair of Digital Reality

In this work, a novel radar simulation concept for efficiently simulating realistic radar data for range, Doppler, and arbitrary antenna positions is introduced. With the concept, the simulated radar signal can also be automatically annotated by splitting it into multiple parts. Annotations that are almost perfect - including the annotation of exotic effects, such as multi-path - can be produced with this approach. Signal parts originating from different parts of an object can be labelled with it as well. To this end, the computation process used in a Monte Carlo shooting and bouncing rays (SBR) simulator is adapted. By considering the hits of each simulated ray, various meta data can be stored, such as hit position, mesh pointer, object IDs, and many more. This collected meta data can then be utilized to predict path-length changes caused by object motion to obtain Doppler information or to apply specific ray filter rules to obtain radar signals that only fulfil specific conditions, such as multiple bounces, or signals that contain specific object IDs. Using this approach, perfect, and otherwise almost impossible, annotation schemes can be realized.

关键词：

GenRec: Unifying Video Generation and Recognition with Diffusion Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Weng, Zejia Yang, Xitong Xing, Zhen Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Department of Computer Science University of Maryland United States

Video diffusion models are able to generate high-quality videos by learning strong spatial-temporal priors on large-scale datasets. In this paper, we aim to investigate whether such priors derived from a generative process are suitable for video recognition, and eventually joint optimization of generation and recognition. Building upon Stable Video Diffusion, we introduce GenRec, the first unified framework trained with a random-frame conditioning process so as to learn generalized spatial-temporal representations. The resulting framework can naturally supports generation and recognition, and more importantly is robust even when visual inputs contain limited information. Extensive experiments demonstrate the efficacy of GenRec for both recognition and generation. In particular, GenRec achieves competitive recognition performance, offering 75.8% and 87.2% accuracy on SSV2 and K400, respectively. GenRec also performs the best on class-conditioned image-to-video generation, achieving 46.5 and 49.3 FVD scores on SSV2 and EK-100 datasets. Furthermore, GenRec demonstrates extraordinary robustness in scenarios that only limited frames can be observed. Code will be available at https://***/wengzejia1/GenRec. Copyright © 2024, The Authors. All rights reserved.

关键词： Spatio-temporal data

MotionEditor: Editing Video Motion via Content-Aware Diffusion

学校读者我要写书评

暂无评论

MotionEditor: Editing Video Motion via Content-Aware Diffusi...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Shuyuan Tu Qi Dai Zhi-Qi Cheng Han Hu Xintong Han Zuxuan Wu Yu-Gang Jiang Shanahai Key Lab of Intell. Info. Processing School of CS Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing Microsoft Research Asia Camegie Mellon University Huya Inc.

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Existing diffusion-based video editing models have made gorgeous advances for editing attributes of a source video over time but struggle to manipulate the motion information while preserving the original protagonist's appearance and background. To address this, we propose MotionEditor, the first diffusion model for video motion editing. MotionEditor incorporates a novel content-aware motion adapter into ControlNet to capture temporal motion correspondence. While ControlNet enables direct generation based on skeleton poses, it encounters challenges when modifying the source motion in the inverted noise due to contradictory signals between the noise (source) and the condition (reference). Our adapter complements Control-Net by involving source content to transfer adapted control signals seamlessly. Further, we build up a two-branch ar-chitecture (a reconstruction branch and an editing branch) with a high-fidelity attention injection mechanism facilitating branch interaction. This mechanism enables the editing branch to query the key and value from the reconstruction branch in a decoupled manner, making the editing branch retain the original background and protagonist appearance. We also propose a skeleton alignment algorithm to address the discrepancies in pose size and position. Experiments demonstrate the promising motion editing ability of MotionEditor, both qualitatively and quantitatively. To the best of our knowledge, MotionEditor is the first to use diffusion models specifically for video motion editing, considering the origin dynamic background and camera movement.

关键词： Adaptation models Computer vision Heuristic algorithms Noise Dynamics Diffusion models Controllability

CUDA and Applications to Task-based Programming

学校读者我要写书评

暂无评论

CUDA and Applications to Task-based Programming

43rd Annual Conference on European Association for Computer Graphics, EUROGRAPHICS 2022

作者： Kerbl, Bernhard Kenzel, Michael Winter, Martin Steinberger, Markus TU Wien Institute of Visual Computing and Human-Centered Technology Austria Saarland University Computer Graphics Lab Germany Intelligent Cloud Rendering Laboratory Huawei Technologies Austria Graz University of Technology Institute of Computer Graphics and Vision Austria

ISBN: (纸本)9783038681724

Since its inception, the CUDA programming model has been continuously evolving. Because the CUDA toolkit aims to consistently expose cutting-edge capabilities for general-purpose compute jobs to its users, the added features in each new version reflect the rapid changes that we observe in GPU architectures. Over the years, the changes in hardware, growing scope of built-in functions and libraries, as well as an advancing C++ standard compliance have expanded the design choices when coding for CUDA, and significantly altered the directives to achieve peak performance. In this tutorial, we give a thorough introduction to the CUDA toolkit, demonstrate how a contemporary application can benefit from recently introduced features and how they can be applied to task-based GPU scheduling in particular. For instance, we will provide detailed examples of use cases for independent thread scheduling, cooperative groups, and the CUDA standard library, libcu++, which are certain to become an integral part of clean coding for CUDA in the near future. © 2022 The Author(s)

关键词： Regulatory compliance

Neural Image Unfolding: Flattening Sparse Anatomical Structures using Neural Fields

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Rist, Leonhard Stephan, Pluvio Maul, Noah Vorberg, Linda Ditt, Hendrik Sühling, Michael Maier, Andreas Egger, Bernhard Taubmann, Oliver Pattern Recognition Lab Friedrich-Alexander-Universität Erlangen-Nürnberg Germany Computed Tomography Siemens Healthineers AG Forchheim Germany Chair of Visual Computing Friedrich-Alexander-Universität Erlangen-Nürnberg Germany

Tomographic imaging reveals internal structures of 3D objects and is crucial for medical diagnoses. visualizing the morphology and appearance of non-planar sparse anatomical structures that extend over multiple 2D slices in tomographic volumes is inherently difficult but valuable for decision-making and reporting. Hence, various organ-specific unfolding techniques exist to map their densely sampled 3D surfaces to a distortion-minimized 2D representation. However, there is no versatile framework to flatten complex sparse structures including vascular, duct or bone systems. We deploy a neural field to fit the transformation of the anatomy of interest to a 2D overview image. We further propose distortion regularization strategies and combine geometric with intensity-based loss formulations to also display non-annotated and auxiliary targets. In addition to improved versatility, our unfolding technique outperforms mesh-based baselines for sparse structures w.r.t. peak distortion and our regularization scheme yields smoother transformations compared to Jacobian formulations from neural field-based image registration. Copyright © 2024, The Authors. All rights reserved.

关键词： Medical imaging

Riemannian Geometry for Scientific visualization 22

学校读者我要写书评

暂无评论

Riemannian Geometry for Scientific Visualization

SIGGRAPH Asia 2022 Courses

作者： Markus Hadwiger Thomas Theußl Peter Rautek Visual Computing Center King Abdullah University of Science and Technology (KAUST) Thuwal Saudi Arabia Visualization Core Lab King Abdullah University of Science and Technology (KAUST) Thuwal Saudi Arabia

ISBN: (纸本)9781450394741

This tutorial introduces the most important basics of Riemannian geometry and related concepts with a specific focus on applications in scientific visualization. The main concept in Riemannian geometry is the presence of a Riemannian metric on a differentiable manifold, comprising a second-order tensor field that defines an inner product in each tangent space that varies smoothly from point to point. Technically, the metric is what allows defining and computing distances and angles in a coordinate-independent manner. However, even more importantly, it in a sense is really the major structure (on top of topological considerations) that defines the space where scientific data, such as scalar, vector, and tensor fields live.

关键词：

Synthesize, Diagnose, and Optimize: Towards Fine-Grained Vision-Language Understanding

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Peng, Wujian Xie, Sicheng You, Zuyao Lan, Shiyi Wu, Zuxuan Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China NVIDIA United States

Vision language models (VLM) have demonstrated remarkable performance across various downstream tasks. However, understanding fine-grained visual-linguistic concepts, such as attributes and inter-object relationships, remains a significant challenge. While several benchmarks aim to evaluate VLMs in finer granularity, their primary focus remains on the linguistic aspect, neglecting the visual dimension. Here, we highlight the importance of evaluating VLMs from both a textual and visual perspective. We introduce a progressive pipeline to synthesize images that vary in a specific attribute while ensuring consistency in all other aspects. Utilizing this data engine, we carefully design a benchmark, SPEC, to diagnose the comprehension of object size, position, existence, and count. Subsequently, we conduct a thorough evaluation of four leading VLMs on SPEC. Surprisingly, their performance is close to random guess, revealing significant limitations. With this in mind, we propose a simple yet effective approach to optimize VLMs in fine-grained understanding, achieving significant improvements on SPEC without compromising the zero-shot performance. Results on two additional fine-grained benchmarks also show consistent improvements, further validating the transferability of our approach. Code and data are available at https://***/wjpoom/SPEC. Copyright © 2023, The Authors. All rights reserved.

关键词： Zero-shot learning

Learning from Rich Semantics and Coarse Locations for Long-tailed Object Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Meng, Lingchen Dai, Xiyang Yang, Jianwei Chen, Dongdong Chen, Yinpeng Liu, Mengchen Chen, Yi-Ling Wu, Zuxuan Yuan, Lu Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Microsoft United States

Long-tailed object detection (LTOD) aims to handle the extreme data imbalance in real-world datasets, where many tail classes have scarce instances. One popular strategy is to explore extra data with image-level labels, yet it produces limited results due to (1) semantic ambiguity—an image-level label only captures a salient part of the image, ignoring the remaining rich semantics within the image;and (2) location sensitivity—the label highly depends on the locations and crops of the original image, which may change after data transformations like random cropping. To remedy this, we propose RichSem, a simple but effective method, which is robust to learn rich semantics from coarse locations without the need of accurate bounding boxes. RichSem leverages rich semantics from images, which are then served as additional "soft supervision" for training detectors. Specifically, we add a semantic branch to our detector to learn these soft semantics and enhance feature representations for long-tailed object detection. The semantic branch is only used for training and is removed during inference. RichSem achieves consistent improvements on both overall and rare-category of LVIS under different backbones and detectors. Our method achieves state-of-the-art performance without requiring complex training and testing procedures. Moreover, we show the effectiveness of our method on other long-tailed datasets with additional experiments. Code is available at https://***/MengLcool/RichSem. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

AdaDiff: Adaptive Step Selection for Fast Diffusion

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Hui Wu, Zuxuan Xing, Zhen Shao, Jie Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of CS Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China ByteDance Inc China

Diffusion models, as a type of generative models, have achieved impressive results in generating images and videos conditioned on textual conditions. However, the generation process of diffusion models involves denoising for dozens of steps to produce photorealistic images/videos, which is computationally expensive. Unlike previous methods that design "one-size-fits-all" approaches for speed up, we argue denoising steps should be sample-specific conditioned on the richness of input texts. To this end, we introduce AdaDiff, a lightweight framework designed to learn instance-specific step usage policies, which are then used by the diffusion model for generation. AdaDiff is optimized using a policy gradient method to maximize a carefully designed reward function, balancing inference time and generation quality. We conduct experiments on three image generation and two video generation benchmarks and demonstrate that our approach achieves similar results in terms of visual quality compared to the baseline using a fixed 50 denoising steps while reducing inference time by at least 33%, going as high as 40%. Furthermore, our qualitative analysis shows that our method allocates more steps to more informative text conditions and fewer steps to simpler text conditions. Copyright © 2023, The Authors. All rights reserved.

关键词： Benchmarking