检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Li, Ming Yang, Taojiannan Kuang, Huafeng Wu, Jie Wang, Zhaoning Xiao, Xuefeng Chen, Chen Center for Research in Computer Vision University of Central Florida United States ByteDance China

To enhance the controllability of text-to-image diffusion models, existing efforts like ControlNet incorporated image-based conditional controls. In this paper, we reveal that existing methods still face significant challenges in generating images that align with the image conditional controls. To this end, we propose ControlNet++, a novel approach that improves controllable generation by explicitly optimizing pixel-level cycle consistency between generated images and conditional controls. Specifically, for an input conditional control, we use a pre-trained discriminative reward model to extract the corresponding condition of the generated images, and then optimize the consistency loss between the input conditional control and extracted condition. A straightforward implementation would be generating images from random noises and then calculating the consistency loss, but such an approach requires storing gradients for multiple sampling timesteps, leading to considerable time and memory costs. To address this, we introduce an efficient reward strategy that deliberately disturbs the input images by adding noise, and then uses the single-step denoised images for reward fine-tuning. This avoids the extensive costs associated with image sampling, allowing for more efficient reward fine-tuning. Extensive experiments show that ControlNet++ significantly improves controllability under various conditional controls. For example, it achieves improvements over ControlNet by 11.1% mIoU, 13.4% SSIM, and 7.6% RMSE, respectively, for segmentation mask, line-art edge, and depth conditions. All the code, models, demo and organized data have been open sourced on our Github Repo. Copyright © 2024, The Authors. All rights reserved.

关键词： Controllability

来源：评论

学校读者我要写书评

暂无评论

A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition

A Large-scale Study of Spatiotemporal Representation Learnin...

引用

International Conference on computer vision (ICCV)

作者： Andong Deng Taojiannan Yang Chen Chen Center for Research in Computer Vision University of Central Florida USA

The goal of building a benchmark (suite of datasets) is to provide a unified protocol for fair evaluation and thus facilitate the evolution of a specific area. Nonetheless, we point out that existing protocols of action recognition could yield partial evaluations due to several limitations. To comprehensively probe the effectiveness of spatiotemporal representation learning, we introduce BEAR, a new BEnchmark on video Action Recognition. BEAR is a collection of 18 video datasets grouped into 5 categories (anomaly, gesture, daily, sports, and instructional), which covers a diverse set of real-world applications. With BEAR, we thoroughly evaluate 6 common spatiotemporal models pre-trained by both supervised and self-supervised learning. We also report transfer performance via standard finetuning, few-shot finetuning, and unsupervised domain adaptation. Our observation suggests that the current state-of-the-art cannot solidly guarantee high performance on datasets close to real-world applications, and we hope BEAR can serve as a fair and challenging evaluation benchmark to gain insights on building next-generation spatiotemporal learners. Our dataset, code, and models are released at: https://***/AndongDeng/BEAR

关键词：

来源：评论

学校读者我要写书评

暂无评论

Hybrid Active Learning via Deep Clustering for Video Action Detection

Hybrid Active Learning via Deep Clustering for Video Action ...

引用

Conference on computer vision and Pattern Recognition (CVPR)

作者： Aayush J Rana Yogesh S Rawat Center for Research in Computer Vision (CRCV) University of Central Florida

In this work, we focus on reducing the annotation cost for video action detection which requires costly frame-wise dense annotations. We study a novel hybrid active learning (AL) strategy which performs efficient labeling using both intra-sample and inter-sample selection. The intra-sample selection leads to labeling of fewer frames in a video as opposed to inter-sample selection which operates at video level. This hybrid strategy reduces the annotation cost from two different aspects leading to significant labeling cost reduction. The proposed approach utilize Clustering-Aware Uncertainty Scoring (CLAUS), a novel label acquisition strategy which relies on both informativeness and diversity for sample selection. We also propose a novel Spatio-Temporal Weighted (STeW) loss formulation, which helps in model training under limited annotations. The proposed approach is evaluated on UCF-101-24 and J-HMDB-21 datasets demonstrating its effectiveness in significantly reducing the annotation cost where it consistently outperforms other baselines. Project details available at https://***/hybridclaus

关键词：

来源：评论

学校读者我要写书评

暂无评论

$R^{2}$ Former: Unified Retrieval and Reranking Transformer for Place Recognition

$R^{2}$ Former: Unified Retrieval and Reranking Transformer ...

引用

Conference on computer vision and Pattern Recognition (CVPR)

作者： Sijie Zhu Linjie Yang Chen Chen Mubarak Shah Xiaohui Shen Heng Wang ByteDance Center for Research in Computer Vision University of Central Florida

Visual Place Recognition (VPR) estimates the location of query images by matching them with images in a reference database. Conventional methods generally adopt aggregated CNN features for global retrieval and RANSAC-based geometric verification for reranking. However, RANSAC only employs geometric information but ignores other possible information that could be useful for reranking, e.g. local feature correlations, and attention values. In this paper, we propose a unified place recognition framework that handles both retrieval and reranking with a novel transformer model, named $R^{2}$ Former. The proposed reranking module takes feature correlation, attention value, and $xy$ coordinates into account, and learns to determine whether the image pair is from the same location. The whole pipeline is end-to-end trainable and the reranking module alone can also be adopted on other CNN or transformer backbones as a generic component. Remarkably, $R^{2}$ Former significantly outperforms state-of-the-art methods on major VPR datasets with much less inference time and memory consumption. It also achieves the state-of-the-art on the hold-out MSLS challenge set and could serve as a simple yet strong solution for real-world large-scale applications. Experiments also show vision transformer tokens are comparable and sometimes better than CNN local features on local matching. The code is released at https://***/Jeff-Zilence/R2Former.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Skeletonmae: Spatial-Temporal Masked Autoencoders for Self-Supervised Skeleton Action Recognition

Skeletonmae: Spatial-Temporal Masked Autoencoders for Self-S...

引用

2023 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2023

作者： Wu, Wenhan Hua, Yilei Zheng, Ce Wu, Shiqian Chen, Chen Lu, Aidong University of North Carolina at Charlotte Department of Computer Science United States School of Information Science and Engineering Wuhan University of Science and Technology China Center for Research in Computer Vision University of Central Florida United States

ISBN: (纸本)9798350313154

Self-supervised skeleton-based action recognition has attracted more attention in recent years. By utilizing the unlabeled data, more generalizable features can be learned to alleviate the overfitting problem and reduce the demand for massive labeled training data. Inspired by the MAE [1], we propose a spatial-temporal masked autoencoder framework for self-supervised 3D skeleton-based action recognition (SkeletonMAE). Following MAE's masking and reconstruction pipeline, we utilize a skeleton-based encoder-decoder transformer architecture to reconstruct the masked skeleton sequences. A novel masking strategy, named Spatial-Temporal Masking, is introduced in terms of both joint-level and frame-level for the skeleton sequence. This pre-training strategy makes the encoder output generalizable skeleton features with spatial and temporal dependencies. Given the unmasked skeleton sequence, the encoder is fine-tuned for the action recognition task. Extensive experiments show that our SkeletonMAE achieves remarkable performance and outperforms the state-of-the-art methods on both NTU RGB+D 60 and NTU RGB+D 120 datasets. © 2023 IEEE.

关键词： Learning systems

来源：评论

学校读者我要写书评

暂无评论

Defending Poisoning Attacks in Federated Learning via Loss Value Normal Distribution 26

Defending Poisoning Attacks in Federated Learning via Loss V...

引用

26th International Conference on computer Supported Cooperative Work in Design, CSCWD 2023

作者： Han, Fei Zhang, Yao Zhao, Meng Tianjin University of Technology Engineering Research Center of Learning-Based Intelligent System Ministry of Education Key Laboratory of Computer Vision and System of Ministry of Education Tianjin300384 China

ISBN: (纸本)9798350331684

As an emerging distributed machine learning paradigm, federated learning (FL) only shares model updates between clients and servers, and training data are secured on each client's device. Due to local data privacy, FL is vulnerable to poisoning attacks, which causes serious degradation in the model performance. In this paper, we analyze the targeted poisoning attacks of FL and reveal that when the data held by each client of FL is independent and identically distributed (iid), the loss of all non-poisoned models on some clean dataset agrees with a normal distribution (ND), while the loss value of the poisoned model deviates from this distribution. Based on this observation, we propose a method named ND-defense to defend FL targeted poisoning attacks. We set a small clean test dataset on the server, compute the loss of each client model on it, detect the abnormal ones with the proposed loss-value-based inspection algorithm, and then only aggregate the benign clients' models to get the global model. We conducted experiments on three benchmark datasets and compared several state-of-the-art defense methods. The results show that our NDdefense is effective in defending against targeted poisoning attacks. © 2023 IEEE.

关键词： Normal distribution

来源：评论

学校读者我要写书评

暂无评论

Development of methods and algorithms of technical vision for detecting the defect longitudinal crack on sheet metal 6

Development of methods and algorithms of technical vision fo...

引用

6th Scientific School "Dynamics of Complex Networks and their Applications", DCNA 2022

作者： Konstantin, Mortin Maksim, Shamshin Leading Specialist in Computer Vision of the Center for Research Laboratories of the Engineering and Technology Center Moscow Russia The Digital Solutions Laboratory of the Engineering and Technology Center Moscow Russia

ISBN: (数字)9781665461740

ISBN: (纸本)9781665461740

The paper presents a mathematical model of a fuzzy subset of a defect in a digital image and is described as a piecewise constant function. The analysis of the filtering of the flaw detection image is given to ensure the implementation of the quantization algorithm of detection with subsequent adaptive binarization of the obtained result. The developed method makes it possible to detect a sheet metal defect of the longitudinal crack type and calculate various geometric parameters of this defect. This approach allows not only to see the detection of a longitudinal crack, but also to minimize the errors of the second level of false positives on flaw detection images. The above result is compared with the annotation of the flaw detector and with the YOLOv3 neural network. © 2022 IEEE.

关键词： Sheet metal

来源：评论

学校读者我要写书评

暂无评论

GAReT: Cross-view Video Geolocalization with Adapters and Auto-Regressive Transformers

arXiv

引用

arXiv 2024年

作者： Pillai, Manu S. Rizve, Mamshad Nayeem Shah, Mubarak Center for Research in Computer Vision University of Central Florida United States Adobe United States

Cross-view video geo-localization (CVGL) aims to derive GPS trajectories from street-view videos by aligning them with aerial-view images. Despite their promising performance, current CVGL methods face significant challenges. These methods use camera and odometry data, typically absent in real-world scenarios. They utilize multiple adjacent frames and various encoders for feature extraction, resulting in high computational costs. Moreover, these approaches independently predict each street-view frame’s location, resulting in temporally inconsistent GPS trajectories. To address these challenges, in this work, we propose GAReT, a fully transformer-based method for CVGL that does not require camera and odometry data. We introduce GeoAdapter, a transformer-adapter module designed to efficiently aggregate image-level representations and adapt them for video inputs. Specifically, we train a transformer encoder on video frames and aerial images, then freeze the encoder to optimize the GeoAdapter module to obtain video-level representation. To address temporally inconsistent trajectories, we introduce TransRetriever, an encoder-decoder transformer model that predicts GPS locations of street-view frames by encoding top-k nearest neighbor predictions per frame and auto-regressively decoding the best neighbor based on the previous frame’s predictions. Our method’s effectiveness is validated through extensive experiments, demonstrating state-of-the-art performance on benchmark datasets. Our code is available at https://***/manupillai308/GAReT. Copyright © 2024, The Authors. All rights reserved.

关键词： Global positioning system

来源：评论

学校读者我要写书评

暂无评论

Open Vocabulary Multi-Label Video Classification

arXiv

引用

arXiv 2024年

作者： Gupta, Rohit Rizve, Mamshad Nayeem Unnikrishnan, Jayakrishnan Tawari, Ashish Tran, Son Shah, Mubarak Yao, Benjamin Chilimbi, Trishul Center for Research in Computer Vision University of Central Florida United States Amazon United States

Pre-trained vision-language models (VLMs) have enabled significant progress in open vocabulary computer vision tasks such as image classification, object detection and image segmentation. Some recent works have focused on extending VLMs to open vocabulary single label action classification in videos. However, previous methods fall short in holistic video understanding which requires the ability to simultaneously recognize multiple actions and entities e.g., objects in the video in an open vocabulary setting. We formulate this problem as open vocabulary multilabel video classification and propose a method to adapt a pre-trained VLM such as CLIP to solve this task. We leverage large language models (LLMs) to provide semantic guidance to the VLM about class labels to improve its open vocabulary performance with two key contributions. First, we propose an end-to-end trainable architecture that learns to prompt an LLM to generate soft attributes for the CLIP text-encoder to enable it to recognize novel classes. Second, we integrate a temporal modeling module into CLIP's vision encoder to effectively model the spatio-temporal dynamics of video concepts as well as propose a novel regularized finetuning technique to ensure strong open vocabulary classification performance in the video domain. Our extensive experimentation showcases the efficacy of our approach on multiple benchmark datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Towards a Perceptual Evaluation Framework for Lighting Estimation

Towards a Perceptual Evaluation Framework for Lighting Estim...

引用

Conference on computer vision and Pattern Recognition (CVPR)

作者： Justine Giroux Mohammad Reza Karimi Dastjerdi Yannick Hold-Geoffroy Javier Vazquez-Corral Jean-François Lalonde Université Laval Adobe Research Computer Vision Center Universitat Autònoma de Barcelona

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

Progress in lighting estimation is tracked by computing existing image quality assessment (IQA) metrics on images from standard datasets. While this may appear to be a reasonable approach, we demonstrate that doing so does not correlate to human preference when the estimated lighting is used to relight a virtual scene into a real photograph. To study this, we design a controlled psychophysical experiment where human observers must choose their preference amongst rendered scenes lit using a set of lighting estimation algorithms selected from the recent literature, and use it to analyse how these algorithms perform according to human perception. Then, we demonstrate that none of the most popular IQA metrics from the literature, taken individually, correctly represent human perception. Finally, we show that by learning a combination of existing IQA metrics, we can more accurately represent human preference. This provides a new perceptual framework to help evaluate future lighting estimation algorithms. To encourage future research, all (anonymised) perceptual data and code are available at https://***/PerceptionMetric/.

关键词： Measurement Image quality computer vision Codes Lighting Estimation Observers

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：