检索结果-内蒙古大学图书馆

STRNet:Triple-stream Spatiotemporal Relation Network for Action recognition

学校读者我要写书评

暂无评论

International Journal of Automation and computing 2021年第5期18卷 718-730页

作者： Zhi-Wei Xu Xiao-Jun Wu Josef Kittler School of Artificial Intelligence and Computer Science Jiangnan UniversityWuxi 214122China Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Wuxi 214122China Centre for Vision Speech and Signal ProcessingUniversity of SurreyGuildfordGU27XHUK

Learning comprehensive spatiotemporal features is crucial for human action recognition. Existing methods tend to model the spatiotemporal feature blocks in an integrate-separate-integrate form, such as appearance-and-relation network(ARTNet) and spatiotemporal and motion network(STM). However, with blocks stacking up, the rear part of the network has poor interpretability. To avoid this problem, we propose a novel architecture called spatial temporal relation network(STRNet), which can learn explicit information of appearance, motion and especially the temporal relation information. Specifically, our STRNet is constructed by three branches,which separates the features into 1) appearance pathway, to obtain spatial semantics, 2) motion pathway, to reinforce the spatiotemporal feature representation, and 3) relation pathway, to focus on capturing temporal relation details of successive frames and to explore long-term representation dependency. In addition, our STRNet does not just simply merge the multi-branch information, but we apply a flexible and effective strategy to fuse the complementary information from multiple pathways. We evaluate our network on four major action recognition benchmarks: Kinetics-400, UCF-101, HMDB-51, and Something-Something v1, demonstrating that the performance of our STRNet achieves the state-of-the-art result on the UCF-101 and HMDB-51 datasets, as well as a comparable accuracy with the state-of-the-art method on Something-Something v1 and Kinetics-400.

关键词： Action recognition spatiotemporal relation multi-branch fusion long-term representation video classification

Generating Cartoon Images from Face Photos with Cycle-Consistent Adversarial Networks

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2021年第11期69卷 2733-2747页

作者： Tao Zhang Zhanjie Zhang Wenjing Jia Xiangjian He Jie Yang School of Artificial Intelligence and Computer Science Jiangnan UniversityWuxi214000China Key Laboratory of Artificial Intelligence Jiangsu214000China The Global Big Data Technologies Centre University of Technology SydneyUltimoNSW2007Australia The Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong UniversityShanghai201100China

The generative adversarial network(GAN)is first proposed in 2014,and this kind of network model is machine learning systems that can learn to measure a given distribution of data,one of the most important applications is style *** transfer is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output ***-GAN is a classic GAN model,which has a wide range of scenarios in style *** its unsupervised learning characteristics,the mapping is easy to be learned between an input image and an output ***,it is difficult for CYCLE-GAN to converge and generate high-quality *** order to solve this problem,spectral normalization is introduced into each convolutional kernel of the *** convolutional kernel reaches Lipschitz stability constraint with adding spectral normalization and the value of the convolutional kernel is limited to[0,1],which promotes the training process of the proposed ***,we use pretrained model(VGG16)to control the loss of image content in the position of l1 *** avoid overfitting,l1 regularization term and l2 regularization term are both used in the object loss *** terms of Frechet Inception Distance(FID)score evaluation,our proposed model achieves outstanding performance and preserves more discriminative *** results show that the proposed model converges faster and achieves better FID scores than the state of the art.

关键词： Generative adversarial network spectral normalization Lipschitz stability constraint VGG16 l1 regularization term l2 regularization term Frechet inception distance

ZBS: Zero-Shot Background Subtraction via Instance-Level Background Modeling and Foreground Selection

学校读者我要写书评

暂无评论

ZBS: Zero-Shot Background Subtraction via Instance-Level Bac...

Conference on Computer Vision and pattern recognition (CVPR)

作者： Yongqi An Xu Zhao Tao Yu Haiyun Gu Chaoyang Zhao Ming Tang Jinqiao Wang National Laboratory of Pattern Recognition Institute of Automation CAS Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China

Background subtraction (BGS) aims to extract all moving objects in the video frames to obtain binary foreground segmentation masks. Deep learning has been widely used in this field. Compared with supervised-based BGS methods, unsupervised methods have better generalization. However, previous unsupervised deep learning BGS algorithms perform poorly in sophisticated scenarios such as shadows or night lights, and they cannot detect objects outside the pre-defined categories. In this work, we propose an unsuper-vised BGS algorithm based on zero-shot object detection called Zero-shot Background Subtraction (ZBS). The proposed method fully utilizes the advantages of zero-shot object detection to build the open-vocabulary instance-level background model. Based on it, the foreground can be effectively extracted by comparing the detection results of new frames with the background model. ZBS performs well for sophisticated scenarios, and it has rich and extensible categories. Furthermore, our method can easily generalize to other tasks, such as abandoned object detection in unseen environments. We experimentally show that ZBS surpasses state-of-the-art unsupervised BGS methods by 4.70% F-Measure on the CDnet 2014 dataset. The code is released at https://***/CASIA-IVA-Lab/ZBS.

关键词：

ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： An, Yongqi Zhao, Xu Yu, Tao Guo, Haiyun Zhao, Chaoyang Tang, Ming Wang, Jinqiao National Laboratory of Pattern Recognition Institute of Automation CAS Beijing China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China

Background subtraction (BGS) aims to extract all moving objects in the video frames to obtain binary foreground segmentation masks. Deep learning has been widely used in this field. Compared with supervised-based BGS methods, unsupervised methods have better generalization. However, previous unsupervised deep learning BGS algorithms perform poorly in sophisticated scenarios such as shadows or night lights, and they cannot detect objects outside the predefined categories. In this work, we propose an unsupervised BGS algorithm based on zero-shot object detection called Zero-shot Background Subtraction (ZBS). The proposed method fully utilizes the advantages of zero-shot object detection to build the open-vocabulary instance-level background model. Based on it, the foreground can be effectively extracted by comparing the detection results of new frames with the background model. ZBS performs well for sophisticated scenarios, and it has rich and extensible categories. Furthermore, our method can easily generalize to other tasks, such as abandoned object detection in unseen environments. We experimentally show that ZBS surpasses state-of-the-art unsupervised BGS methods by 4.70% F-Measure on the CDnet 2014 dataset. The code is released at https://***/CASIA-IVA-Lab/ZBS. Copyright © 2023, The Authors. All rights reserved.

关键词： Object detection

Feature Space Renormalization for Semi-supervised Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Sun, Jun Mao, Zhongjie Li, Chao Zhou, Chao Wu, Xiao-Jun The School of Artificial Intelligence and Computer Science Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University 1800 Lihu Avenue Jiangsu Wuxi214122 China The Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence Jiangnan University 1800 Lihu Avenue Jiangsu Wuxi214122 China

Semi-supervised learning (SSL) has been proven to be a powerful method for leveraging unlabelled data to alleviate models' dependence on large labelled datasets. The common framework among recent approaches is to train the model on a large amount of unlabelled data with consistency regularization to constrain the model predictions to be invariant to input perturbation. However, the existing SSL frameworks still have room for improvement in the consistency regularization method. Instead of regularizing category predictions in the label space as in existing frameworks, this paper proposes a feature space renormalization (FSR) mechanism for SSL. First, we propose a feature space renormalization mechanism to substitute for the commonly used consistency regularization mechanism to learn better discriminative features. To apply this mechanism, we start by building a basic model and an empirical model and then introduce our mechanism to renormalize the feature learning of the basic model with the guidance of the empirical model. Second, we combine the proposed mechanism with pseudo-labelling to obtain a novel effective SSL model named FreMatch. The experimental results show that our method can achieve better performance on a variety of standard SSL benchmark datasets, and the proposed feature space renormalization mechanism can also enhance the performance of other SSL approaches. Copyright © 2023, The Authors. All rights reserved.

关键词： Benchmarking

Gravitational Wave Detection Based on Squeeze-and-excitation Shrinkage Networks and Multiple Detector Coherent SNR

学校读者我要写书评

暂无评论

Research in Astronomy and Astrophysics 2022年第11期22卷 84-96页

作者： Rui-Qing Yan Wei Liu Zong-Yao Yin Rong Ma Si-Ying Chen Dan Hu Dan Wu Xian-Chuan Yu School of Artificial Intelligence Beijing Normal UniversityBeijing 100875China National Laboratory of Pattern Recognition Institute of AutomationChinese Academy of SciencesBeijing 100190China Department of Radiology and BRIC University of North Carolina at Chapel HillChapel HillNC 27599 United States of America National Astronomical Observatories Chinese Academy of SciencesBeijing 100101China

Deep learning techniques have been applied to the detection of gravitational wave signals in the past few *** existing methods focus on the data obtained by a single ***,the signal-to-noise ratio(SNR)of gravitational wave signals in a single detector is pretty low,making it hard for deep neural networks to learn effective ***,how to use the observation signals obtained by multiple detectors in deep learning methods is a serious *** simulate binary neutron star signals from multiple detectors,including the Advanced LIGO and Virgo *** calculate coherent SNR of multiple detectors using a fully coherent allsky search method and obtain the coherent SNR data required for our proposed deep learning *** by the principle of attention network Squeeze-and-Excitation Networks(SENet)and the soft thresholding shrinkage function,we propose a novel Squeeze-and-Excitation Shrinkage(SES)module to better extract effective *** we use this module to establish a gravitational wave squeeze-and-excitation shrinkage network(GWSESNet)detection *** train and validate the performance of our model on the coherent SNR data *** model obtains satisfactory classification accuracy and can excellently complete the task of gravitational wave detection.

关键词： methods:data analysis methods:statistical gravitational waves

Learning to Rank Pre-trained Vision-Language Models for Downstream Tasks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ding, Yuhe Jiang, Bo Zheng, Aihua Xu, Qin Liang, Jian School of Computer Science and Technology Anhui University China School of Artificial Intelligence Anhui University China New Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences China

Vision language models (VLMs) like CLIP show stellar zero-shot capability on classification benchmarks. However, selecting the VLM with the highest performance on the unlabeled downstream task is non-trivial. Existing VLM selection methods focus on the class-name-only setting, relying on a supervised large-scale dataset and large language models, which may not be accessible or feasible during deployment. This paper introduces the problem of unsupervised vision-language model selection, where only unsupervised downstream datasets are available, with no additional information provided. To solve this problem, we propose a method termed Visual-tExtual Graph Alignment (VEGA), to select VLMs without any annotations by measuring the alignment of the VLM between the two modalities on the downstream task. VEGA is motivated by the pretraining paradigm of VLMs, which aligns features with the same semantics from the visual and textual modalities, thereby mapping both modalities into a shared representation space. Specifically, we first construct two graphs on the vision and textual features, respectively. VEGA is then defined as the overall similarity between the visual and textual graphs at both node and edge levels. Extensive experiments across three different benchmarks, covering a variety of application scenarios and downstream datasets, demonstrate that VEGA consistently provides reliable and accurate estimates of VLMs’ performance on unlabeled downstream tasks. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Xuannan Cui, Xing Li, Peipei Li, Zekun Huang, Huaibo Xia, Shuhan Zhang, Miaoxuan Zou, Yueying He, Ran School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China School of Computer Science University of California Santa Barbara United States State Key Laboratory of Multimodal Artificial Intelligence Systems CASIA New Laboratory of Pattern Recognition CASIA School of Artificial Intelligence University of Chinese Academy of Sciences Beijing100190 China

The rapid evolution of multimodal foundation models has led to significant advancements in cross-modal understanding and generation across diverse modalities, including text, images, audio, and video. However, these models remain susceptible to jailbreak attacks, which can bypass built-in safety mechanisms and induce the production of potentially harmful content. Consequently, understanding the methods of jailbreak attacks and existing defense mechanisms is essential to ensure the safe deployment of multimodal generative models in real-world scenarios, particularly in security-sensitive applications. To provide comprehensive insight into this topic, this survey reviews jailbreak and defense in multimodal generative models. First, given the generalized lifecycle of multimodal jailbreak, we systematically explore attacks and corresponding defense strategies across four levels: input, encoder, generator, and output. Based on this analysis, we present a detailed taxonomy of attack methods, defense mechanisms, and evaluation frameworks specific to multimodal generative models. Additionally, we cover a wide range of input-output configurations, including modalities such as Any-to-Text, Any-to-Vision, and Any-to-Any within generative systems. Finally, we highlight current research challenges and propose potential directions for future research. The open-source repository corresponding to this work can be found at https://***/liuxuannan/Awesome-Multimodal-Jailbreak. Copyright © 2024, The Authors. All rights reserved.

关键词： Generative adversarial networks

Learning long-term temporal contexts using skip RNN for continuous emotion recognition

学校读者我要写书评

暂无评论

Virtual Reality & Intelligent Hardware 2021年第1期3卷 55-64页

作者： Jian HUANG Bin LIU Jianhua TAO National Laboratory of Pattern Recognition Institute of AutomationChinese Academy of SciencesBeijing 100049China CAS Center for Excellence in Brain Science and Intelligence Technology Beijing 100049China School of Artificial Intelligence University of Chinese Academy of SciencesBeijing 100049China

Background Continuous emotion recognition as a function of time assigns emotional values to every frame in a *** long-term temporal context information is essential for continuous emotion recognition *** For this purpose,we employ a window of feature frames in place of a single frame as inputs to strengthen the temporal modeling at the feature *** ideas of frame skipping and temporal pooling are utilized to alleviate the resulting *** the model level,we leverage the skip recurrent neural network to model the long-term temporal variability by skipping trivial information for continuous emotion *** The experimental results using the AVEC 2017 database demonstrate that our proposed methods are beneficial to a performance ***,the skip long short-term memory(LSTM)model can focus on the critical emotional state when training the models,thereby achieving a better performance than the LSTM model and other methods.

关键词： Continuous emotion recognition Skip RNN Temporal contexts Redundancy