检索结果-内蒙古大学图书馆

TTPP: Temporal transformer with progressive prediction for efficient action anticipation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Wen Peng, Xiaojiang Su, Yanzhou Qiao, Yu Cheng, Jian School of Information and Communication Engineering University of Electronic Science and Technology of China Chengdu Sichuan611731 China ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab. Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Video action anticipation aims to predict future action categories from observed frames. Current state-of-the-art approaches mainly resort to recurrent neural networks to encode history information into hidden states, and predict future actions from the hidden representations. It is well known that the recurrent pipeline is inefficient in capturing long-term information which may limit its performance in predication task. To address this problem, this paper proposes a simple yet efficient Temporal Transformer with Progressive Prediction (TTPP) framework, which repurposes a Transformer-style architecture to aggregate observed features, and then leverages a light-weight network to progressively predict future features and actions. Specifically, predicted features along with predicted probabilities are accumulated into the inputs of subsequent prediction. We evaluate our approach on three action datasets, namely TVSeries, THUMOS-14, and TV-Human-Interaction. Additionally we also conduct a comprehensive study for several popular aggregation and prediction strategies. Extensive results show that TTPP not only outperforms the state-of-the-art methods but also more efficient. Copyright © 2020, The Authors. All rights reserved.

关键词： Forecasting

Isoform Function Prediction Based on Heterogeneous Graph Attention Networks

学校读者我要写书评

暂无评论

Isoform Function Prediction Based on Heterogeneous Graph Att...

2023 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2023

作者： Guo, Kuo Li, Yifan Chen, Hao Shen, Hong-Bin Yang, Yang Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Shanghai Jiao Tong University Key Laboratory of System Control and Information Processing Ministry of Education of China Institute of Image Processing and Pattern Recognition Shanghai200240 China Carnegie Mellon University School of Computer Science Computational Biology Department PittsburghPA15213 United States

ISBN: (纸本)9798350337488

Isoforms refer to different mRNA molecules transcribed from the same gene, which can be translated into proteins with varying structures and functions. Predicting the functions of isoforms is an essential topic in bioinformatics as it can provide valuable insights into the intricate mechanisms of gene regulation and biological processes. Conventionally, gene function lab.ls are standardized in Gene Ontology (GO) terms. However, traditional methods for predicting isoform function are largely limited by the absence of isoform-specific lab.ls, sparse annotations, and the vast number of GO terms. To address these issues, we propose HANIso, a deep learning-based method for isoform function prediction. HANIso leverages a pretrained protein language model to extract features from protein sequences. It also integrates heterogeneous information, such as isoform sequence features, GO annotations, and isoform interaction data, using a Heterogeneous Graph Attention Network (HAN). This allows the model to learn the importance of different sources of information and their semantic relationships through the attention mechanism. Our method can predict function lab.ls at both the gene level and isoform level. We conduct experiments on two species datasets, and the results demonstrate that our method outperforms existing methods on both AUROC and AUPRC. HANIso has the potential to overcome the limitations of traditional methods and provide a more accurate and comprehensive understanding of isoform function. © 2023 IEEE.

关键词： alternative splicing gene ontology heterogeneous graph attention network isoform function prediction protein language model

A New Journey from SDRTV to HDRTV

学校读者我要写书评

暂无评论

A New Journey from SDRTV to HDRTV

International Conference on computer vision (ICCV)

作者： Xiangyu Chen Zhengwen Zhang Jimmy S. Ren Lynhoo Tian Yu Qiao Chao Dong ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institute of Advanced Technology Chinese Academy of Sciences SenseTime Research Qing Yuan Research Institute Shanghai Jiao Tong University Shanghai AI Laboratory Shanghai

ISBN: (纸本)9781665428132

Nowadays modern displays are capable to render video content with high dynamic range (HDR) and wide color gamut (WCG). However, most availab.e resources are still in standard dynamic range (SDR). Therefore, there is an urgent demand to transform existing SDR-TV contents into their HDR-TV versions. In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step solution pipeline including adaptive global color mapping, local enhancement and highlight generation. Moreover, the above analysis inspires us to present a lightweight network that utilizes global statistics as guidance to conduct image-adaptive color mapping. In addition, we construct a dataset using HDR videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results achieve state-of-the-art performance in quantitative comparisons and visual quality. The code and dataset are availab.e at https://***/chxy95/HDRTVNet.

关键词： Visualization computer vision Analytical models Codes Image color analysis Pipelines Transforms

New Tampered Features for Scene and Caption Text Classification in Video Frame

学校读者我要写书评

暂无评论

New Tampered Features for Scene and Caption Text Classificat...

International Workshop on Frontiers in Handwriting recognition

作者： Sangheeta Roy Palaiahnakote Shivakumara Umapada Pal Tong Lu Chew Lim Tan Faculty of Computer Science and Information Technology University of Malaya Kuala Lumpur Malaysia Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India National Key Lab for Novel Software Technology Nanjing University Nanjing China Department of Computer Science National University of Singapore

ISBN: (纸本)9781509009824

The presence of both caption/graphics/superimposed and scene texts in video frames is the major cause for the poor accuracy of text recognition methods. This paper proposes an approach for identifying tampered information by analyzing the spatial distribution of DCT coefficients in a new way for classifying caption and scene text. Since caption text is edited/superimposed, which results in artificially created texts comparing to scene texts that exist naturally in frames. We exploit this fact to identify the presence of caption and scene texts in video frames based on the advantage of DCT coefficients. The proposed method analyzes the distributions of both zero and non-zero coefficients (only positive values) locally by moving a window, and studies histogram operations over each input text line image. This generates line graphs for respective zero and non-zero coefficient coordinates. We further study the behavior of text lines, namely, linearity and smoothness based on centroid location analysis, and the principal axis direction of each text line for classification. Experimental results on standard datasets, namely, ICDAR 2013 video, 2015 video, YVT video and our own data, show that the performances of text recognition methods are improved significantly after-classification compared to before-classification.

关键词： Discrete cosine transforms Text recognition Image color analysis Linearity Bars Character recognition Mathematical model

WaveDM: Wavelet-Based Diffusion Models for Image Restoration

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Yi Huang, Jiancheng Liu, Jianzhuang Yan, Mingfu Dong, Yu Lyu, Jiaxi Chen, Chaoqi Chen, Shifeng Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China University of Chinese Academy of Sciences Beijing100039 China The University of Hong Kong Hong Kong

Latest diffusion-based methods for many image restoration tasks outperform traditional models, but they encounter the long-time inference problem. To tackle it, this paper proposes a Wavelet-Based Diffusion Model (WaveDM). WaveDM learns the distribution of clean images in the wavelet domain conditioned on the wavelet spectrum of degraded images after wavelet transform, which is more time-saving in each step of sampling than modeling in the spatial domain. To ensure restoration performance, a unique training strategy is proposed where the low-frequency and high-frequency spectrums are learned using distinct modules. In addition, an Efficient Conditional Sampling (ECS) strategy is developed from experiments, which reduces the number of total sampling steps to around 5. Evaluations on twelve benchmark datasets including image raindrop removal, rain steaks removal, dehazing, defocus deblurring, demoiréing, and denoising demonstrate that WaveDM achieves state-of-the-art performance with the efficiency that is comparable to traditional one-pass methods and over 100× faster than existing image restoration methods using vanilla diffusion models. The code is availab.e at https://***/stayalive16/WaveDM. Copyright © 2023, The Authors. All rights reserved.

关键词： Image reconstruction

MsEDNet: Multi-Scale Deep Saliency Learning for Moving Object Detection

学校读者我要写书评

暂无评论

MsEDNet: Multi-Scale Deep Saliency Learning for Moving Objec...

IEEE International Conference on Systems, Man, and Cybernetics

作者： Prashant W. Patil Subrahmanyam Murala Abhinav Dhall Sachin Chaudhary Indian Institute of Technology Delhi New Delhi Delhi IN Computer Vision and Pattern Recognition Lab Indian Institute of Technology Ropar INDIA Learning Afffect and Semantic Image AnalysIs (LASII) Group Indian Institute of Technology Ropar INDIA

Moving object detection (foreground and background) is an important problem in computer vision. Most of the works in this problem are based on background subtraction. However, these approaches are not able to handle scenarios with infrequent motion of object, illumination changes, shadow, camouflage etc. To overcome these, here a two stage robust and compact method for moving object detection (MOD) is proposed. In first stage, to generate the saliency map, background image is estimated using a temporal histogram technique with the help of several input frames. In the second stage, multiscale encoder-decoder network is used to learn multiscale semantic feature of estimated saliency for foreground extraction. The encoder is used to extract multi-scale features from multi-scale saliency map. The decoder part is designed to learn the mapping of low resolution multi-scale features into high resolution output frame. To observe the efficacy of proposed MsEDNet, experiments are conducted on two benchmark datasets (change detection (CDnet-2014) and Wallflower) for MOD. The precision, recall and F-measure are used as performance parameter for comparison with the existing state-of-the-art methods. Experimental results show a significant improvement in detection accuracy and decrement in execution time as compared to the state-of-the-art methods for MOD.

关键词： Feature extraction Estimation Decoding Object detection Histograms Lighting Semantics

ONLINE ADAPTIVE DICTIONARY LEARNING AND WEIGHTED SPARSE CODING FOR ABNORMALITY DETECTION

学校读者我要写书评

暂无评论

ONLINE ADAPTIVE DICTIONARY LEARNING AND WEIGHTED SPARSE CODI...

IEEE International Conference on Image Processing

作者： Sheng Han Ruiqing Fu Suzhen Wang Xinyu Wu Shenzhen Key Lab for Computer Vision and Pattern Recognition Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Guangdong Provincial Key Lab of Robotics and Intelligent System Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences The Chinese University of Hong Kong

ISBN: (纸本)9781479923427

This paper focuses mainly on adaptive dictionary updating and abnormality detection via weighted space coding in video surveillance. Generally, abnormality analysis conducted on a large amount of video data is very complicated, time-consuming and time-variant. However, our dictionary is very efficient at following up on shifted contents in video and abandoning old inactive information in time. The adaptability characteristic also helps reduce the dictionary's size to a small scale, since it only needs to keep recent or active information. We also introduce a simple, but effective, judgement criterion for abnormal detection based on sparse coding over weighted bases. Because of the condensed dictionary and the simplified judgment criterion, our algorithm performs online learning and online detection with a high speed and a high accuracy in various scenes.

关键词： Dictionary Learning Adaptive Learning Sparse Coding Abnormality Detection Sparse coding anomaly detection video data Dictionaries as Topic video monitoring

Blind super-resolutionwith iterative kernel correction

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Gu, Jinjin Lu, Hannan Zuo, Wangmeng Dong, Chao School of Science and Engineering Chinese University of Hong Kong Shenzhen China School of Computer Science and Technology Harbin Institute of Technology Harbin China ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences

Deep learning based methods have dominated superresolution (SR) field due to their remarkable performance in terms of effectiveness and efficiency. Most of these methods assume that the blur kernel during downsampling is predefined/ known (e.g., bicubic). However, the blur kernels involved in real applications are complicated and unknown, resulting in severe performance drop for the advanced SR methods. In this paper, we propose an Iterative Kernel Correction (IKC) method for blur kernel estimation in blind SR problem, where the blur kernels are unknown. We draw the observation that kernel mismatch could bring regular artifacts (either over-sharpening or over-smoothing), which can be applied to correct inaccurate blur kernels. Thus we introduce an iterative correction scheme - IKC that achieves better results than direct kernel estimation. We further propose an effective SR network architecture using spatial feature transform (SFT) layers to handle multiple blur kernels, named SFTMD. Extensive experiments on synthetic and real-world images show that the proposed IKC method with SFTMD can provide visually favorable SR results and the state-of-the-art performance in blind SR problem. Copyright © 2019, The Authors. All rights reserved.

关键词： Network architecture

Temporal modulation network for controllab.e space-time video super-resolution

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Xu, Gang Xu, Jun Li, Zhen Wang, Liang Sun, Xing Cheng, Ming-Ming College of Computer Science Nankai University Tianjin China School of Statistics and Data Science Nankai University Tianjin China National Lab of Pattern Recognition Institute of Automation CAS Beijing China Youtu Lab. Tencent Shanghai China

Space-time video super-resolution (STVSR) aims to increase the spatial and temporal resolutions of low-resolution and low-frame-rate videos. Recently, deformable convolution based methods have achieved promising STVSR performance, but they could only infer the intermediate frame pre-defined in the training stage. Besides, these methods undervalued the short-term motion cues among adjacent frames. In this paper, we propose a Temporal Modulation Network (TMNet) to interpolate arbitrary intermediate frame(s) with accurate high-resolution reconstruction. Specifically, we propose a Temporal Modulation Block (TMB) to modulate deformable convolution kernels for controllab.e feature interpolation. To well exploit the temporal information, we propose a Locally-temporal Feature Comparison (LFC) module, along with the Bi-directional Deformable ConvLSTM, to extract short-term and long-term motion cues in videos. Experiments on three benchmark datasets demonstrate that our TMNet outperforms previous STVSR methods. The code is availab.e at https://***/CS-GangXu/TMNet. Copyright © 2021, The Authors. All rights reserved.

关键词： Convolution