检索结果-内蒙古大学图书馆

Disentangling Spatial and Temporal Learning for Efficient image-to-Video Transfer Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Qing, Zhiwu Zhang, Shiwei Huang, Ziyuan Zhang, Yingya Gao, Changxin Zhao, Deli Sang, Nong Key Laboratory of Image Processing Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Alibaba Group China ARC National University of Singapore Singapore

Recently, large-scale pre-trained language-image models like CLIP have shown extraordinary capabilities for understanding spatial contents, but naively transferring such models to video recognition still suffers from unsatisfactory temporal modeling capabilities. Existing methods insert tunable structures into or in parallel with the pre-trained model, which either requires back-propagation through the whole pre-trained model and is thus resource-demanding, or is limited by the temporal reasoning capability of the pre-trained structure. In this work, we present DiST, which disentangles the learning of spatial and temporal aspects of videos. Specifically, DiST uses a dual-encoder structure, where a pre-trained foundation model acts as the spatial encoder, and a lightweight network is introduced as the temporal encoder. An integration branch is inserted between the encoders to fuse spatio-temporal information. The disentangled spatial and temporal learning in DiST is highly efficient because it avoids the back-propagation of massive pre-trained parameters. Meanwhile, we empirically show that disentangled learning with an extra network for integration benefits both spatial and temporal understanding. Extensive experiments on five benchmarks show that DiST delivers better performance than existing state-of-the-art methods by convincing gaps. When pretraining on the large-scale Kinetics-710, we achieve 89.7% on Kinetics-400 with a frozen ViT-L model, which verifies the scalability of DiST. Codes and models can be found in https://***/alibaba-mmai-research/DiST. Copyright © 2023, The Authors. All rights reserved.

关键词： Backpropagation

VideoLCM: Video Latent Consistency Model

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xiang Zhang, Shiwei Zhang, Han Liu, Yu Zhang, Yingya Gao, Changxin Sang, Nong Key Laboratory of Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Alibaba Group China Shanghai Jiao Tong University China

Consistency models have demonstrated powerful capability in efficient image generation and allowed synthesis within a few sampling steps, alleviating the high computational cost in diffusion models. However, the consi...

关键词：

3D Cinemagraphy from a Single image

学校读者我要写书评

暂无评论

3D Cinemagraphy from a Single Image

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Xingyi Li Zhiguo Cao Huiqiang Sun Jianming Zhang Ke Xian Guosheng Lin Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology S-Lab Nanyang Technological University Adobe Research

We present 3D Cinemagraphy, a new technique that mar-ries 2D image animation with 3D photography. Given a single still image as input, our goal is to generate a video that contains both visual content animation and camera motion. We empirically find that naively combining existing 2D image animation and 3D photography methods leads to obvious artifacts or inconsistent animation. Our key insight is that representing and animating the scene in 3D space offers a natural solution to this task. To this end, we first convert the input image into feature-based layered depth images using predicted depth values, followed by unprojecting them to a feature point cloud. To animate the scene, we perform motion estimation and lift the 2D motion into the 3D scene flow. Finally, to resolve the problem of hole emer-gence as points move forward, we propose to bidirectionally displace the point cloud as per the scene flow and synthe-size novel views by separately projecting them into target image planes and blending the results. Extensive experiments demonstrate the effectiveness of our method. A user study is also conducted to validate the compelling rendering results of our method.

关键词：

A Physiological Signal Emotion Recognition Method Based on Domain Adaptation and Incremental Learning

学校读者我要写书评

暂无评论

A Physiological Signal Emotion Recognition Method Based on D...

Cyber-Energy Systems and intelligent Energy (ICCSIE), International Conference on

作者： Junnan Li Xiaoping Wang the School of Artificial Intelligence and Automation and the Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan P. R. China

Temporal concept shift (TCS) is an unavoidable problem in physiological signal-based emotion recognition tasks, i.e., the data distribution of physiological signals is constantly changing over time, which gradually degrades the model accuracy. To this end, we propose a method based on a combination of domain adaptation and incremental learning to reduce the impact of temporal concept drift. In this paper, domain adaptation is used to reduce the distribution differences and incremental learning is used to prevent the learned knowledge from being forgotten. Finally, we validate the effectiveness of our approach on two real datasets.

关键词：

Alignment-Based Adversarial Training (ABAT) for Improving the Robustness and Accuracy of EEG-Based BCIs

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, Xiaoqing Wang, Ziwei Wu, Dongrui The Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Shenzhen Huazhong University of Science and Technology Research Institute Shenzhen China

Machine learning has achieved great success in electroencephalogram (EEG) based brain-computer interfaces (BCIs). Most existing BCI studies focused on improving the decoding accuracy, with only a few considering the adversarial security. Although many adversarial defense approaches have been proposed in other application domains such as computer vision, previous research showed that their direct extensions to BCIs degrade the classification accuracy on benign samples. This phenomenon greatly affects the applicability of adversarial defense approaches to EEG-based BCIs. To mitigate this problem, we propose alignment-based adversarial training (ABAT), which performs EEG data alignment before adversarial training. Data alignment aligns EEG trials from different domains to reduce their distribution discrepancies, and adversarial training further robustifies the classification boundary. The integration of data alignment and adversarial training can make the trained EEG classifiers simultaneously more accurate and more robust. Experiments on five EEG datasets from two different BCI paradigms (motor imagery classification, and event related potential recognition), three convolutional neural network classifiers (EEGNet, ShallowCNN and DeepCNN) and three different experimental settings (offline within-subject cross-block/-session classification, online cross-session classification, and pre-trained classifiers) demonstrated its effectiveness. It is very intriguing that adversarial attacks, which are usually used to damage BCI systems, can be used in ABAT to simultaneously improve the model accuracy and robustness. © 2024, CC BY.

关键词： Adversarial machine learning

Fast Candidate Region Extraction for SAR Ship Target 37

学校读者我要写书评

暂无评论

Fast Candidate Region Extraction for SAR Ship Target

37th Youth Academic Annual Conference of Chinese Association of Automation, YAC 2022

作者： Zhang, Panpan Luo, Haibo Xu, Zheng He, Miao Shenyang Institute of Automation Shenyang110016 China Institutes for Robotics and Intelligent Manufacturing Shenyang110016 China University of Chinese Academy of Sciences Beijing100049 China Key Laboratory of Opto-Electronic Information Processing Shenyang110016 China The Key Lab of Image Understanding and Computer Vision Shenyang110016 China

ISBN: (纸本)9781665465366

At present, deep learning technology is widely used in ship target detection in synthetic aperture radar (SAR) images. However, high-resolution remote sensing SAR images cover a larger area and have larger image sizes. To be able to use the deep learning model for training and testing, the image needs to be cropped to the appropriate size. In a high-resolution SAR ship image, the ship target usually takes only a small part of the whole image. As a result, only part of the cropped image contains the target, and most of the rest are background regions. This will cause a lot of computational redundancy in the model inference stage. To solve this problem, a fast candidate region extraction algorithm is proposed in this paper for ship target extraction. The algorithm consists of three parts: firstly, extract the saliency map of the SAR ship image, secondly, perform median filtering on the segmented image, and thirdly, extract candidate regions. The superiority of the algorithm was demonstrated by experiments on the AIR-SARShip-1.0 dataset. © 2022 IEEE.

关键词： Synthetic aperture radar

A novel method of constructing high-dimensional digital chaotic systems on finite-state automata

学校读者我要写书评

暂无评论

Chinese Physics B 2020年第9期29卷 233-243页

作者： Jun Zheng Han-Ping Hu School of Artificial Intelligence and Automation Huazhong University of Science and TechnologyWuhan 430074China Key Laboratory of Image Information Processing and Intelligent Control Ministry of EducationWuhan 430074China

When chaotic systems are implemented on finite precision machines, it will lead to the problem of dynamical degradation. Aiming at this problem, most previous related works have been proposed to improve the dynamical degradation of low-dimensional chaotic maps. This paper presents a novel method to construct high-dimensional digital chaotic systems in the domain of finite computing precision. The model is proposed by coupling a high-dimensional digital system with a continuous chaotic system. A rigorous proof is given that the controlled digital system is chaotic in the sense of Devaney's definition of chaos. Numerical experimental results for different high-dimensional digital systems indicate that the proposed method can overcome the degradation problem and construct high-dimensional digital chaos with complicated dynamical properties. Based on the construction method, a kind of pseudorandom number generator (PRNG) is also proposed as an application.

关键词： high-dimensional digital chaotic system dynamical degradation anti-control pseudorandom number generator

Focusing on diacritics to improve Vietnamese scene text detection

学校读者我要写书评

暂无评论

Focusing on diacritics to improve Vietnamese scene text dete...

International Joint Conference on Neural Networks (IJCNN)

作者： Wenhui Huang Shaoliang Shi Yimin Wen Guilin University of Electronic Technology Guilin China China Academy of Science and Technology Development GuangXi Branch Nanning China Guangxi Key Laboratory of Image and Graphic Intelligent Processing Guilin University of Electronic Technology Guilin China

ISBN: (数字)9798350359312

ISBN: (纸本)9798350359329

Scene text detection is usually recognized as one of the fundamental challenges in computer vision and robotics. While the detection of scene text for English and Chinese has achieved great progress, Vietnamese scene text detection is still a relatively new field. In Vietnamese scene text detection, the diacritics of Vietnamese characters are often too small to be detected, making it difficult to accurately extract their features. The bounding boxes are usually placed close to Latin letters, resulting in incomplete covering for diacritics. To overcome these challenges, we propose a method of focusing on diacritics to detect Vietnamese scene text. Firstly, we propose detailed information embedding to enhance the features of diacritics. Secondly, we introduce IoU-v to allow the proposals to match the boundary of Vietnamese text instances more accurately. Lastly, in mask branch, we incorporate a diacritics feature fusion module to further enhance the features of diacritics. The experimental results illustrate the proposed method outperforms the existing approaches. The code of the proposed algorithm is available at https://***/mlmmwym/FD2VSTD.

关键词： Computer vision Codes Text recognition Neural networks Text detection Focusing Feature extraction

HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xiang Zhang, Shiwei Qing, Zhiwu Zuo, Zhengrong Gao, Changxin Jin, Rong Sang, Nong Key Laboratory of Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Alibaba Group China Twitter United States

Few-shot action recognition is a challenging but practical problem aiming to learn a model that can be easily adapted to identify new action categories with only a few labeled samples. Recent attempts mainly focus on learning deep representations for each video individually under the episodic meta-learning regime and then performing temporal alignment to match query and support videos. However, they still suffer from two drawbacks: (i) learning individual features without considering the entire task may result in limited representation capability, and (ii) existing alignment strategies are sensitive to noises and misaligned instances. To handle the two limitations, we propose a novel Hybrid Relation guided temporal Set Matching (HyRSM++) approach for few-shot action recognition. The core idea of HyRSM++ is to integrate all videos within the task to learn discriminative representations and involve a robust matching technique. To be specific, HyRSM++ consists of two key components, a hybrid relation module and a temporal set matching metric. Given the basic representations from the feature extractor, the hybrid relation module is introduced to fully exploit associated relations within and cross videos in an episodic task and thus can learn task-specific embeddings. Subsequently, in the temporal set matching metric, we carry out the distance measure between query and support videos from a set matching perspective and design a bidirectional Mean Hausdorff Metric to improve the resilience to misaligned instances. In addition, we explicitly exploit the temporal coherence in videos to regularize the matching process. In this way, HyRSM++ facilitates informative correlation exchanged among videos and enables flexible predictions under the data-limited scenario. Furthermore, we extend the proposed HyRSM++ to deal with the more challenging semi-supervised few-shot action recognition and unsupervised few-shot action recognition tasks. Experimental results on multiple benchmark

关键词： Benchmarking