检索结果-内蒙古大学图书馆

Joint Demosaicing and Denoising With Self Guidance

学校读者我要写书评

暂无评论

Joint Demosaicing and Denoising With Self Guidance

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Lin Liu Xu Jia Jianzhuang Liu Qi Tian CAS Key Laboratory of GIPAS University of Science and Technology of China Noah's Ark Lab Huawei Technologies

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Usually located at the very early stages of the computational photography pipeline, demosaicing and denoising play important parts in the modern camera image processing. Recently, some neural networks have shown the effectiveness in joint demosaicing and denoising (JDD). Most of them first decompose a Bayer raw image into a four-channel RGGB image and then feed it into a neural network. This practice ignores the fact that the green channels are sampled at a double rate compared to the red and the blue channels. In this paper, we propose a self-guidance network (SGNet), where the green channels are initially estimated and then works as a guidance to recover all missing values in the input image. In addition, as regions of different frequencies suffer different levels of degradation in image restoration. We propose a density-map guidance to help the model deal with a wide range of frequencies. Our model outperforms state-of-the-art joint demosaicing and denoising methods on four public datasets, including two real and two synthetic data sets. Finally, we also verify that our method obtains best results in joint demosaicing , denoising and super-resolution.

关键词： Noise reduction Task analysis Channel estimation Image resolution Image restoration Image edge detection Neural networks

Contrastive transformation for self-supervised correspondence learning

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Ning Zhou, Wengang Li, Houqiang CAS Key Laboratory of GIPAS University of Science and Technology of China China Institute of Artificial Intelligence Hefei Comprehensive National Science Center China

In this paper, we focus on the self-supervised learning of visual correspondence using unlabeled videos in the wild. Our method simultaneously considers intra- and inter-video representation associations for reliable correspondence estimation. The intra-video learning transforms the image contents across frames within a single video via the frame pair-wise affinity. To obtain the discriminative representation for instance-level separation, we go beyond the intra-video analysis and construct the inter-video affinity to facilitate the contrastive transformation across different videos. By forcing the transformation consistency between intra- and inter-video levels, the fine-grained correspondence associations are well preserved and the instance-level feature discrimination is effectively reinforced. Our simple framework outperforms the recent self-supervised correspondence methods on a range of visual tasks including video object tracking (VOT), video object segmentation (VOS), pose keypoint tracking, etc. It is worth mentioning that our method also surpasses the fully-supervised affinity representation (e.g., ResNet) and performs competitively against the recent fully-supervised algorithms designed for the specific tasks (e.g., VOT and VOS). Copyright © 2020, The Authors. All rights reserved.

关键词： Machine learning

Incorporating bert into neural machine translation

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Zhu, Jinhua Xia, Yingce Wu, Lijun He, Di Qin, Tao Zhou, Wengang Li, Houqiang Liu, Tie-Yan CAS Key Laboratory of GIPAS EEIS Department University of Science and Technology of China Microsoft Research Sun Yat-sen University School of EECS Peking University

The recently proposed BERT (Devlin et al., 2019) has shown great power on a variety of natural language understanding tasks, such as text classification, reading comprehension, etc. However, how to effectively apply BERT to neural machine translation (NMT) lacks enough exploration. While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning. This motivates us to think how to better leverage BERT for NMT along this direction. We propose a new algorithm named BERT-fused model, in which we first use BERT to extract representations for an input sequence, and then the representations are fused with each layer of the encoder and decoder of the NMT model through attention mechanisms. We conduct experiments on supervised (including sentence-level and document-level translations), semi-supervised and unsupervised machine translation, and achieve state-of-the-art results on seven benchmark datasets. Our code is available at https://***/bert-nmt/bert-nmt. Copyright © 2020, The Authors. All rights reserved.

关键词： Neural machine translation

Iterative Alignment Network for Continuous Sign Language Recognition

学校读者我要写书评

暂无评论

Iterative Alignment Network for Continuous Sign Language Rec...

IEEE/CVF Conference on Computer Vision and Pattern Recognition

作者： Junfu Pu Wengang Zhou Houqiang Li CAS Key Laboratory of GIPAS University of Science and Technology of China

ISBN: (纸本)9781728132945

In this paper, we propose an alignment network with iterative optimization for weakly supervised continuous sign language recognition. Our framework consists of two modules: a 3D convolutional residual network (3D-ResNet) for feature learning and an encoder-decoder network with connectionist temporal classification (CTC) for sequence modelling. The above two modules are optimized in an alternate way. In the encoder-decoder sequence learning network, two decoders are included, i.e., LSTM decoder and CTC decoder. Both decoders are jointly trained by maximum likelihood criterion with a soft Dynamic Time Warping (soft-DTW) alignment constraint. The warping path, which indicates the possible alignment between input video clips and sign words, is used to fine-tune the 3D-ResNet as training labels with classification loss. After fine-tuning, the improved features are extracted for optimization of encoder-decoder sequence learning network in next iteration. The proposed algorithm is evaluated on two large scale continuous sign language recognition benchmarks, i.e., RWTH-PHOENIX-Weather and CSL. Experimental results demonstrate the effectiveness of our proposed method.

关键词： decoders sign language recognition Iterative

SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Pei, Qizhi Wu, Lijun Zhu, Jinhua Xia, Yingce Xie, Shufang Qin, Tao Liu, Haiguang Liu, Tie-Yan Yan, Rui Gaoling School of Artificial Intelligence Renmin University of China No.59 Zhong Guan Cun Avenue Haidian District Beijing100872 China Microsoft Research AI4Science No.5 Dan Ling Street Haidian District Beijing100080 China CAS Key Laboratory of GIPAS EEIS Department University of Science and Technology of China No.96 JinZhai Road Baohe District Anhui Province Hefei230026 China Engineering Research Center of Next-Generation Intelligent Search and Recommendation Ministry of Education China Beijing Key Laboratory of Big Data Management and Analysis Methods China

Accurate prediction of Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery, facilitating the identification of drugs that can effectively interact with specific targets and regulate their activities. While wet experiments remain the most reliable method, they are time-consuming and resource-intensive, resulting in limited data availability that poses challenges for deep learning approaches. Existing methods have primarily focused on developing techniques based on the available DTA data, without adequately addressing the data scarcity issue. To overcome this challenge, we present the SSM-DTA framework, which incorporates three simple yet highly effective strategies: (1) A multi-task training approach that combines DTA prediction with masked language modeling (MLM) using paired drug-target data. (2) A semi-supervised training method that leverages large-scale unpaired molecules and proteins to enhance drug and target representations. This approach differs from previous methods that only employed molecules or proteins in pre-training. (3) The integration of a lightweight cross-attention module to improve the interaction between drugs and targets, further enhancing prediction accuracy. Through extensive experiments on benchmark datasets such as BindingDB, DAVIS, and KIBA, we demonstrate the superior performance of our framework. Additionally, we conduct case studies on specific drug-target binding activities, virtual screening experiments, drug feature visualizations, and real-world applications, all of which showcase the significant potential of our work. In conclusion, our proposed SSM-DTA framework addresses the data limitation challenge in DTA prediction and yields promising results, paving the way for more efficient and accurate drug discovery processes. Our code is available at https://***/QizhiPei/SSM-DTA. Copyright © 2022, The Authors. All rights reserved.

关键词： Forecasting

Progressive learning of low-precision networks

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Zhou, Zhengguang Zhou, Wengang Lv, Xutao Huang, Xuan Wang, Xiaoyu Li, Houqiang CAS Key Laboratory of GIPAS University of Science and Technology of China Intellifusion Inc.

Recent years have witnessed the great advance of deep learning in a variety of vision tasks. Many state-of-theart deep neural networks suffer from large size and high complexity, which makes it difficult to deploy in resourcelimited platforms such as mobile devices. To this end, lowprecision neural networks are widely studied which quantize weights or activations into the low-bit format. Though being efficient, low-precision networks are usually hard to train and encounter severe accuracy degradation. In this paper, we propose a new training strategy through expanding low-precision networks during training and removing the expanded parts for network inference. First, we equip each low-precision convolutional layer with an ancillary full-precision convolutional layer based on a low-precision network structure, which could guide the network to good local minima. Second, a decay method is introduced to reduce the output of the added full-precision convolution gradually, which keeps the resulted topology structure the same to the original low-precision one. Experiments on SVHN, CIFAR and ILSVRC-2012 datasets prove that the proposed method can bring faster convergence and higher accuracy for low-precision neural networks. Copyright © 2019, The Authors. All rights reserved.

关键词： Convolution

Voxel R-CNN: Towards high performance voxel-based 3D object detection

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Deng, Jiajun Shi, Shaoshuai Li, Peiwei Zhou, Wengang Zhang, Yanyong Li, Houqiang CAS Key Laboratory of GIPAS EEIS Department University of Science and Technology of China China Multimedia Laboratory The Chinese University of Hong Kong Hong Kong Institute of Artificial Intelligence Hefei Comprehensive National Science Center China Department of Computer Science University of Science and Technology of China China

Recent advances on 3D object detection heavily rely on how the 3D data are represented, i.e., voxel-based or point-based representation. Many existing high performance 3D detectors are point-based because this structure can better retain precise point positions. Nevertheless, point-level features lead to high computation overheads due to unordered storage. In contrast, the voxel-based structure is better suited for feature extraction but often yields lower accuracy because the input data are divided into grids. In this paper, we take a slightly different viewpoint — we find that precise positioning of raw points is not essential for high performance 3D object detection and that the coarse voxel granularity can also offer sufficient detection accuracy. Bearing this view in mind, we devise a simple but effective voxel-based framework, named Voxel R-CNN. By taking full advantage of voxel features in a two stage approach, our method achieves comparable detection accuracy with state-of-the-art point-based models, but at a fraction of the computation cost. Voxel R-CNN consists of a 3D backbone network, a 2D bird-eye-view (BEV) Region Proposal Network and a detect head. A voxel RoI pooling is devised to extract RoI features directly from voxel features for further refinement. Extensive experiments are conducted on the widely used KITTI Dataset and the more recent Waymo Open Dataset. Our results show that compared to existing voxel-based methods, Voxel R-CNN delivers a higher detection accuracy while maintaining a real-time frame processing rate, i.e., at a speed of 25 FPS on an NVIDIA RTX 2080 Ti GPU. The code is available at https://***/djiajunustc/Voxel-R-CNN. Copyright © 2020, The Authors. All rights reserved.

关键词： Object recognition

Relation distillation networks for video object detection

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Deng, Jiajun Pan, Yingwei Yao, Ting Zhou, Wengang Li, Houqiang Mei, Tao CAS Key Laboratory of GIPAS University of Science and Technology of China Hefei China JD AI Research Beijing China

It has been well recognized that modeling object-toobject relations would be helpful for object detection. Nevertheless, the problem is not trivial especially when exploring the interactions between objects to boost video object detectors. The difficulty originates from the aspect that reliable object relations in a video should depend on not only the objects in the present frame but also all the supportive objects extracted over a long range span of the video. In this paper, we introduce a new design to capture the interactions across the objects in spatio-temporal context. Specifically, we present Relation Distillation Networks (RDN) - a new architecture that novelly aggregates and propagates object relation to augment object features for detection. Technically, object proposals are first generated via Region Proposal Networks (RPN). RDN then, on one hand, models object relation via multi-stage reasoning, and on the other, progressively distills relation through refining supportive object proposals with high objectness scores in a cascaded manner. The learnt relation verifies the efficacy on both improving object detection in each frame and box linking across frames. Extensive experiments are conducted on ImageNet VID dataset, and superior results are reported when comparing to state-of-the-art methods. More remarkably, our RDN achieves 81.8% and 83.2% mAP with ResNet-101 and ResNeXt-101, respectively. When further equipped with linking and rescoring, we obtain to-date the best reported mAP of 83.8% and 84.7%. Copyright © 2019, The Authors. All rights reserved.

关键词： Distillation

Relation Distillation Networks for Video Object Detection

学校读者我要写书评

暂无评论

Relation Distillation Networks for Video Object Detection

International Conference on Computer Vision (ICCV)

作者： Jiajun Deng Yingwei Pan Ting Yao Wengang Zhou Houqiang Li Tao Mei CAS Key Laboratory of GIPAS University of Science and Technology of China Hefei China JD AI Research Beijing China

ISBN: (数字)9781728148038

ISBN: (纸本)9781728148045

It has been well recognized that modeling object-to-object relations would be helpful for object detection. Nevertheless, the problem is not trivial especially when exploring the interactions between objects to boost video object detectors. The difficulty originates from the aspect that reliable object relations in a video should depend on not only the objects in the present frame but also all the supportive objects extracted over a long range span of the video. In this paper, we introduce a new design to capture the interactions across the objects in spatio-temporal context. Specifically, we present Relation Distillation Networks (RDN) - a new architecture that novelly aggregates and propagates object relation to augment object features for detection. Technically, object proposals are first generated via Region Proposal Networks (RPN). RDN then, on one hand, models object relation via multi-stage reasoning, and on the other, progressively distills relation through refining supportive object proposals with high objectness scores in a cascaded manner. The learnt relation verifies the efficacy on both improving object detection in each frame and box linking across frames. Extensive experiments are conducted on ImageNet VID dataset, and superior results are reported when comparing to state-of-the-art methods. More remarkably, our RDN achieves 81.8% and 83.2% mAP with ResNet-101 and ResNeXt-101, respectively. When further equipped with linking and rescoring, we obtain to-date the best reported mAP of 83.8% and 84.7%.

关键词： Proposals Object detection Feature extraction Task analysis Context modeling Detectors Aggregates