检索结果-内蒙古大学图书馆

Self-slimmed vision Transformer

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zong, Zhuofan Li, Kunchang Song, Guanglu Wang, Yali Qiao, Yu Leng, Biao Liu, Yu School of Computer Science and Engineering Beihang University China SenseTime Research China ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society China Shanghai AI Laboratory China

vision transformers (ViTs) have become the popular structures and outperformed convolutional neural networks (CNNs) on various vision tasks. However, such powerful transformers bring a huge computation burden, because of the exhausting token-to-token comparison. The previous works focus on dropping insignificant tokens to reduce the computational cost of ViTs. But when the dropping ratio increases, this hard manner will inevitably discard the vital tokens, which limits its efficiency. To solve the issue, we propose a generic self-slimmed learning approach for vanilla ViTs, namely SiT. Specifically, we first design a novel Token Slimming Module (TSM), which can boost the inference efficiency of ViTs by dynamic token aggregation. As a general method of token hard dropping, our TSM softly integrates redundant tokens into fewer informative ones. It can dynamically zoom visual attention without cutting off discriminative token relations in the images, even with a high slimming ratio. Furthermore, we introduce a concise Feature Recalibration Distillation (FRD) framework, wherein we design a reverse version of TSM (RTSM) to recalibrate the unstructured token in a flexible auto-encoder manner. Due to the similar structure between teacher and student, our FRD can effectively leverage structure knowledge for better convergence. Finally, we conduct extensive experiments to evaluate our SiT. It demonstrates that our method can speed up ViTs by 1.7× with negligible accuracy drop, and even speed up ViTs by 3.6× while maintaining 97% of their performance. Surprisingly, by simply arming LV-ViT with our SiT, we achieve new state-of-the-art performance on ImageNet. Code is available at https://***/Sense-X/SiT. © 2021, CC BY.

关键词： Distillation

Collaborative Multi-View Convolutions With Gating For Accurate And Fast Volumetric Medical Image Segmentation

学校读者我要写书评

暂无评论

Collaborative Multi-View Convolutions With Gating For Accura...

IEEE International Symposium on Biomedical Imaging

作者： Cheng Li Jin Ye Junjun He Shanshan Wang Lixu Gu Yu Qiao Paul C. Lauterbur Research Center for Biomedical Imaging SIAT CAS Shenzhen China Shenzhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab SIAT CAS Shenzhen China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China School of Biomedical Engineering/the Institute of Medical Robotics Shanghai Jiao Tong University Shanghai China

ISBN: (纸本)9781665412469;9781665429474

Due to their high capacity in capturing 3D spatial information, 3D Fully Convolutional Neural Networks (3D FCNs), especially 3D U-Net, are prevalent for volumetric medical image segmentation. However, 3D convolutions are much more computationally complex than 2D convolutions and thus, are more prone to overfitting. This paper proposes Collaborative Multi-View convolutions (CMV convs) that can keep the model complexity similar to those employing 2D convolutions while capturing the 3D spatial context like 3D convolutions. Specifically, CMV convs simultaneously extract information from three orthogonal views with three parameter-shared 2D convolutions. A Global-Guided Gating mechanism (3G) is further designed that selectively passes information from CMV convs to the next stage. Combined with 3G, a CMV conv become a G-CMV conv that constitutes a plug-and-play module, which can be easily integrated into various 3D CNNs for image segmentation. Extensive experiments utilizing BraTS18 dataset have been conducted. Our method achieves competitive results compared to state-of-the-art methods with over 10× fewer parameters than 3D-UNet.

关键词： Solid modeling Image segmentation Interpolation Three-dimensional displays Computational modeling Biological system modeling Collaboration

BasicVSR: The search for essential components in video super-resolution and beyond

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Chan, Kelvin C.K. Wang, Xintao Yu, Ke Dong, Chao Loy, Chen Change S-Lab Nanyang Technological University Singapore Applied Research Center Tencent PCG CUHK – SenseTime Joint Lab Chinese University of Hong Kong Shenzhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society China

Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension. Complex designs are not uncommon. In this study, we wish to untangle the knots and reconsider some most essential components for VSR guided by four basic functionalities, i.e., Propagation, Alignment, Aggregation, and Upsampling. By reusing some existing components added with minimal redesigns, we show a succinct pipeline, BasicVSR, that achieves appealing improvements in terms of speed and restoration quality in comparison to many state-of-the-art algorithms. We conduct systematic analysis to explain how such gain can be obtained and discuss the pitfalls. We further show the extensibility of BasicVSR by presenting an information-refill mechanism and a coupled propagation scheme to facilitate information aggregation. The BasicVSR and its extension, IconVSR, can serve as strong baselines for future VSR approaches. © 2020, CC BY.

关键词： Optical resolving power

A comprehensive study on temporal modeling for online action detection

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Wen Peng, Xiaojiang Qiao, Yu Cheng, Jian School of Information and Communication Engineering University of Electronic Science and Technology of China Chengdu Sichuan611731 China ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences

—Online action detection (OAD) is a practical yet challenging task, which has attracted increasing attention in recent years. A typical OAD system mainly consists of three modules: a frame-level feature extractor which is usually based on pre-trained deep Convolutional Neural Networks (CNNs), a temporal modeling module, and an action classifier. Among them, the temporal modeling module is crucial which aggregates discriminative information from historical and current features. Though many temporal modeling methods have been developed for OAD and other topics, their effects are lack of investigation on OAD fairly. This paper aims to provide a comprehensive study on temporal modeling for OAD including four meta types of temporal modeling methods, i.e. temporal pooling, temporal convolution, recurrent neural networks, and temporal attention, and uncover some good practices to produce a state-of-the-art OAD system. Many of them are explored in OAD for the first time, and extensively evaluated with various hyper parameters. Furthermore, based on our comprehensive study, we present several hybrid temporal modeling methods, which outperform the recent state-of-the-art methods with sizable margins on THUMOS-14 and TVSeries. Copyright © 2020, The Authors. All rights reserved.

关键词： Convolution

Dual-AI: Dual-path Actor Interaction Learning for Group Activity recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Mingfei Zhang, David Junhao Wang, Yali Yan, Rui Yao, Lina Chang, Xiaojun Qiao, Yu ReLER AAII UTS United States National University of Singapore Singapore ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China RMIT University Australia University of New South Wales Australia Shanghai AI Laboratory Shanghai China

Learning spatial-temporal relation among multiple actors is crucial for group activity recognition. Different group activities often show the diversified interactions between actors in the video. Hence, it is often difficult to model complex group activities from a single view of spatial-temporal actor evolution. To tackle this problem, we propose a distinct Dual-path Actor Interaction (Dual-AI) framework, which flexibly arranges spatial and temporal transformers in two complementary orders, enhancing actor relations by integrating merits from different spatiotemporal paths. Moreover, we introduce a novel Multi-scale Actor Contrastive Loss (MAC-Loss) between two interactive paths of Dual-AI. Via self-supervised actor consistency in both frame and video levels, MAC-Loss can effectively distinguish individual actor representations to reduce action confusion among different actors. Consequently, our Dual-AI can boost group activity recognition by fusing such discriminative features of different actors. To evaluate the proposed approach, we conduct extensive experiments on the widely used benchmarks, including Volleyball [24], Collective Activity [12], and NBA datasets [56]. The proposed Dual-AI achieves state-of-the-art performance on all these datasets. It is worth noting the proposed Dual-AI with 50% training data outperforms a number of recent approaches with 100% training data. This confirms the generalization power of Dual-AI for group activity recognition, even under the challenging scenarios of limited supervision. Copyright © 2022, The Authors. All rights reserved.

关键词： Machine learning

DCT-phase statistics for forged IMEI numbers and air ticket detection

学校读者我要写书评

暂无评论

Expert Systems with Applications 2021年 164卷 114014-114014页

作者： Nandanwar, Lokesh Shivakumara, Palaiahnakote Kanchan, Swati Basavaraja, V. Guru, D.S. Pal, Umapada Lu, Tong Blumenstein, Michael Faculty of Computer Science and Information Technology University of Malaya Malaysia Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India Department of Studies in Computer Science University of Mysore Karnataka India National Key Lab for Novel Software Technology Nanjing University China Sydney Australia

New tools have been developing with the intention of having more flexibility and greater user-friendliness for editing the images and documents in digital technologies, but, unfortunately, they are also being used for manipulating and tampering information. Examples of such crimes include creating forged International Mobile Equipment Identity (IMEI) numbers which are embedded on mobile packages and inside smart mobile cases for illicit activities. Another example of such crimes is altering the name or date on air tickets for breaching security at the airport. This paper presents a new expert system for detecting forged IMEI numbers as well as altered air ticket images. The proposed method derives the phase spectrum using the Discrete Cosine Transform (DCT) to highlight the suspicious regions;it is unlike the phase spectrum from a Fourier transform, which is ineffective due to power spectrum noise. From the phase spectrum, our method extracts phase statistics to study the effect of distortions introduced by forgery operations. This results in feature vectors, which are fed to a Support Vector Machine (SVM) classifier for detection of forged IMEI numbers and air ticket images. Experimental results on our dataset of forged IMEI numbers (which is created by us for this work), on altered air tickets, on benchmark datasets of video caption text (which is tampered text), and on altered receipts of the ICPR 2018 FDC dataset, show that the proposed method is robust across different datasets. Furthermore, comparative studies of the proposed method with the existing methods on the same datasets show that the proposed method outperforms the existing methods. The dataset created will be available freely on request to the authors. © 2020 Elsevier Ltd

关键词： Discrete cosine transforms

New texture-spatial features for keyword spotting in video images

学校读者我要写书评

暂无评论

New texture-spatial features for keyword spotting in video i...

Asian Conference on pattern recognition (ACPR)

作者： Palaiahnakote Shivakumara Guozhu Liang Sangheeta Roy Umapada Pal Tong Lu Faculty of Computer Science and Information Technology University of Malaya Kuala Lumpur Malaysia National Key Lab for Novel Software Technology Nanjing University Nanjing China Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India

ISBN: (纸本)9781479961016

keyword spotting in video document images is challenging due to low resolution and complex background of video images. We propose the combination of Texture-Spatial-Features (TSF) for keyword spotting in video images without recognizing them. First, a segmentation method extracts words from text lines in each video image. Then we propose the set of texture features for identifying text candidates in the word image with the help of k-means clustering. The proposed method finds proximity between text candidates to study the spatial arrangement of pixels that result in feature vectors for spotting words in the input frame. The proposed method is evaluated on word images of different fonts, contrasts, backgrounds and font sizes, which are chosen from standard databases such as ICDAR 2013 video and our video data. Experimental results show that the proposed method outperforms the existing method in terms of recall, precision and f-measure.

关键词： Image segmentation Semantics Video signal processing Indexing pattern recognition Spatial resolution

Image quality assessment for perceptual image restoration: A new dataset, benchmark and metric

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Gu, Jinjin Cai, Haoming Chen, Haoyu Ye, Xiaoxing Ren, Jimmy S. Dong, Chao School of Electrical and Information Engineering University of Sydney Australia Chinese University of Hong Kong Shenzhen Hong Kong ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SenseTime Research Qing Yuan Research Institute Shanghai Jiao Tong University Shanghai China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Image quality assessment (IQA) is the key factor for the fast development of image restoration (IR) algorithms. The most recent perceptual IR algorithms based on generative adversarial networks (GANs) have brought in significant improvement on visual performance, but also pose great challenges for quantitative evaluation. Notably, we observe an increasing inconsistency between perceptual quality and the evaluation results. We present two questions: (1) Can existing IQA methods objectively evaluate recent IR algorithms? (2) With the focus on beating current benchmarks, are we getting better IR algorithms? To answer the questions and promote the development of IQA methods, we contribute a large-scale IQA dataset, called Perceptual Image Processing ALgorithms (PIPAL) dataset. Especially, this dataset includes the results of GAN-based IR algorithms, which are missing in previous datasets. We collect more than 1.13 million human judgements to assign subjective scores for PIPAL images using the more reliable "Elo system". Based on PIPAL, we present new benchmarks for both IQA and super-resolution methods. Our results indicate that existing IQA methods cannot fairly evaluate GAN-based IR algorithms. While using appropriate evaluation methods is important, IQA methods should also be updated along with the development of IR algorithms. At last, we shed light on how to improve the IQA performance on GAN-based distortion. Inspired by the find that the existing IQA methods have an unsatisfactory performance on the GAN-based distortion partially because of their low tolerance to spatial misalignment, we propose to improve the performance of an IQA network on GAN-based distortion by explicitly considering this misalignment. We propose the Space Warping Difference Network, which includes the novel l2 pooling layers and Space Warping Difference layers. Experiments demonstrate the effectiveness of the proposed method. © 2020, CC BY.

关键词： Image reconstruction

New Tampered Features for Scene and Caption Text Classification in Video Frame

学校读者我要写书评

暂无评论

New Tampered Features for Scene and Caption Text Classificat...

International Workshop on Frontiers in Handwriting recognition

作者： Sangheeta Roy Palaiahnakote Shivakumara Umapada Pal Tong Lu Chew Lim Tan Faculty of Computer Science and Information Technology University of Malaya Kuala Lumpur Malaysia Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India National Key Lab for Novel Software Technology Nanjing University Nanjing China Department of Computer Science National University of Singapore

ISBN: (纸本)9781509009824

The presence of both caption/graphics/superimposed and scene texts in video frames is the major cause for the poor accuracy of text recognition methods. This paper proposes an approach for identifying tampered information by analyzing the spatial distribution of DCT coefficients in a new way for classifying caption and scene text. Since caption text is edited/superimposed, which results in artificially created texts comparing to scene texts that exist naturally in frames. We exploit this fact to identify the presence of caption and scene texts in video frames based on the advantage of DCT coefficients. The proposed method analyzes the distributions of both zero and non-zero coefficients (only positive values) locally by moving a window, and studies histogram operations over each input text line image. This generates line graphs for respective zero and non-zero coefficient coordinates. We further study the behavior of text lines, namely, linearity and smoothness based on centroid location analysis, and the principal axis direction of each text line for classification. Experimental results on standard datasets, namely, ICDAR 2013 video, 2015 video, YVT video and our own data, show that the performances of text recognition methods are improved significantly after-classification compared to before-classification.

关键词： Discrete cosine transforms Text recognition Image color analysis Linearity Bars Character recognition Mathematical model