检索结果-内蒙古大学图书馆

Few-Shot learning with Complex-valued Neural Networks 31

学校读者我要写书评

暂无评论

Few-Shot Learning with Complex-valued Neural Networks

31st British Machine Vision Conference, BMVC 2020

作者： Liu, Zhen Zhang, Baochang Guo, Guodong Beihang University Beijing China Institute of Deep Learning Baidu Research National Engineering Laboratory for Deep Learning Technology and Application China

Feature representation is fundamental and attracts much attention in few-shot learning. Convolutional neural networks (CNNs) are among the best feature extractors so far in this field, which are successfully combined with metric learning, leading to the state-of-the-art performance. However, the subtle difference among inter-class samples challenges existing CNN based methods, which only use real-valued CNNs that fail to extract more detailed information. In this paper, we introduce complex metric module (CMM) into metric learning, aiming to better measure the inter- and intra-class relations based on both amplitude and phase information. Specifically, building upon the recent episodic training mechanism, our CMM can enhance the representation capacity by extracting robust complex-valued features to facilitate modeling subtle relationships among samples, which can enhance the performance of the few-shot classification task when only few samples are available. Moreover, we introduce a new transductive method into CMM, by considering not only query and support but also query and query relationships to predict classes of unlabeled samples. Experiments on two benchmark datasets show that the proposed CMM significantly improves the performance over other approaches and achieves the state-of-the-art results. © 2020. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.

关键词： Benchmarking

Enhancing Multimodal Information Extraction from Visually Rich Documents with 2D Positional Embeddings

学校读者我要写书评

暂无评论

Enhancing Multimodal Information Extraction from Visually Ri...

Proceedings of the Digital Image Computing: Technqiues and applications (DICTA)

作者： Aresha Arshad Momina Moetesum Adnan Ul Hasan Faisal Shafait School of Electrical Engineering and Computer Science (SEECS) National University of Sciences and Technology (NUST) Islamabad Pakistan Deep Learning Laboratory National Center of Artificial Intelligence (NCAI) Islamabad Pakistan

ISBN: (数字)9798350379037

ISBN: (纸本)9798350379044

Visually rich document understanding involves the interpretation of documents with varied formats and complex layouts, including multi-line entities, presenting a significant challenge. This study addresses these challenges by introducing a document comprehension model based on the LayoutLMv3 architecture, incorporating two-dimensional (2D) positional embeddings to capture both row and column information. Additionally, a multi-stage Transformer network is employed for hierarchical processing of document features. The proposed model is evaluated through extensive experiments on FUNSD dataset, achieving improved performance in both text-centric and image-centric document understanding tasks. Results demonstrate enhanced spatial comprehension and computational efficiency as compared to the state-of-the-art, establishing our approach as a significant contribution to the field of visually rich document understanding.

关键词： Computational modeling Two-dimensional displays Digital images Layout Computer architecture Transformers Information retrieval Encoding Computational efficiency Usability

Recurrent Bilinear Optimization for Binary Neural Networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xu, Sheng Li, Yanjing Wang, Tiancheng Ma, Teli Zhang, Baochang Gao, Peng Qiao, Yu Lü, Jinhu Guo, Guodong Beihang University Beijing China Shanghai Artificial Intelligence Laboratory Shanghai China Zhongguancun Laboratory Beijing China Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China

Binary Neural Networks (BNNs) show great promise for real-world embedded devices. As one of the critical steps to achieve a powerful BNN, the scale factor calculation plays an essential role in reducing the performance gap to their real-valued counterparts. However, existing BNNs neglect the intrinsic bilinear relationship of real-valued weights and scale factors, resulting in a sub-optimal model caused by an insufficient training process. To address this issue, Recurrent Bilinear Optimization is proposed to improve the learning process of BNNs (RBONNs) by associating the intrinsic bilinear variables in the back propagation process. Our work is the first attempt to optimize BNNs from the bilinear perspective. Specifically, we employ a recurrent optimization and Density-ReLU to sequentially backtrack the sparse real-valued weight filters, which will be sufficiently trained and reach their performance limits based on a controllable learning process. We obtain robust RBONNs, which show impressive performance over state-of-the-art BNNs on various models and datasets. Particularly, on the task of object detection, RBONNs have great generalization performance. Our code is open-sourced on https://***/SteveTsui/RBONN. © 2022, CC BY.

关键词： Object detection

Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Yanjing Xu, Sheng Zhang, Baochang Cao, Xianbin Gao, Peng Guo, Guodong Beihang University Beijing China Zhongguancun Laboratory Beijing China Shanghai Artificial Intelligence Laboratory Shanghai China Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China

The large pre-trained vision transformers (ViTs) have demonstrated remarkable performance on various visual tasks, but suffer from expensive computational and memory cost problems when deployed on resource-constrained devices. Among the powerful compression approaches, quantization extremely reduces the computation and memory consumption by low-bit parameters and bit-wise operations. However, low-bit ViTs remain largely unexplored and usually suffer from a significant performance drop compared with the real-valued counterparts. In this work, through extensive empirical analysis, we first identify the bottleneck for severe performance drop comes from the information distortion of the low-bit quantized self-attention map. We then develop an information rectification module (IRM) and a distribution guided distillation (DGD) scheme for fully quantized vision transformers (Q-ViT) to effectively eliminate such distortion, leading to a fully quantized ViTs. We evaluate our methods on popular DeiT and Swin backbones. Extensive experimental results show that our method achieves a much better performance than the prior arts. For example, our Q-ViT can theoretically accelerates the ViT-S by 6.14× and achieves about 80.9% Top-1 accuracy, even surpassing the full-precision counterpart by 1.0% on ImageNet dataset. Our codes and models are attached on https://***/YanjingLi0202/Q-ViT. © 2022, CC BY.

关键词： Vision

A Representation Separation Perspective to Correspondences-free Unsupervised 3D Point Cloud Registration

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Zhiyuan Sun, Jiadai Dai, Yuchao Zhou, Dingfu Song, Xibin He, Mingyi School of Electronics and Information Northwestern Polytechnical University Xi’an China Baidu Research and National Engineering Laboratory Deep Learning Technology and Application Beijing China

3D point cloud registration in remote sensing field has been greatly advanced by deep learning based methods, where the rigid transformation is either directly regressed from the two point clouds (correspondences-free approaches) or computed from the learned correspondences (correspondences-based approaches). Existing correspondences-free methods generally learn the holistic representation of the entire point cloud, which is fragile for partial and noisy point clouds. In this paper, we propose a correspondences-free unsupervised point cloud registration (UPCR) method from the representation separation perspective. First, we model the input point cloud as a combination of pose-invariant representation and pose-related representation. Second, the pose-related representation is used to learn the relative pose w.r.t. a "latent canonical shape" for the source and target point clouds respectively. Third, the rigid transformation is obtained from the above two learned relative poses. Our method not only filters out the disturbance in pose-invariant representation but also is robust to partial-to-partial point clouds or noise. Experiments on benchmark datasets demonstrate that our unsupervised method achieves comparable if not better performance than state-of-the-art supervised registration methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Surface measurement

Region-level Contrastive and Consistency learning for Semi-Supervised Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Jianrong Wu, Tianyi Ding, Chuanghao Zhao, Hongwei Guo, Guodong College of Computer Science and Technology Jilin University Changchun China Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China College of Software Jilin University Changchun China The Institute of Deep Learning Baidu Research China

Current semi-supervised semantic segmentation methods mainly focus on designing pixel-level consistency and contrastive regularization. However, pixel-level regularization is sensitive to noise from pixels with incorrect predictions, and pixel-level contrastive regularization has memory and computational cost with O(pixel num2). To address the issues, we propose a novel region-level contrastive and consistency learning framework (RC2L) for semi-supervised semantic segmentation. Specifically, we first propose a Region Mask Contrastive (RMC) loss and a Region Feature Contrastive (RFC) loss to accomplish region-level contrastive property. Furthermore, Region Class Consistency (RCC) loss and Semantic Mask Consistency (SMC) loss are proposed for achieving region-level consistency. Based on the proposed region-level contrastive and consistency regularization, we develop a region-level contrastive and consistency learning framework (RC2L) for semi-supervised semantic segmentation, and evaluate our RC2L on two challenging benchmarks (PASCAL VOC 2012 and Cityscapes), outperforming the state-of-the-art. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantic Segmentation

Sparse to dense motion transfer for face image animation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhao, Ruiqi Wu, Tianyi Guo, Guodong Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China

Face image animation from a single image has achieved remarkable progress. However, it remains challenging when only sparse landmarks are available as the driving signal. Given a source face image and a sequence of sparse face landmarks, our goal is to generate a video of the face imitating the motion of landmarks. We develop an efficient and effective method for motion transfer from sparse landmarks to the face image. We then combine global and local motion estimation in a unified model to faithfully transfer the motion. The model can learn to segment the moving foreground from the background and generate not only global motion, such as rotation and translation of the face, but also subtle local motion such as the gaze change. We further improve face landmark detection on videos. With temporally better aligned landmark sequences for training, our method can generate temporally coherent videos with higher visual quality. Experiments suggest we achieve results comparable to the state-of-the-art image driven method on the same identity testing and better results on cross identity testing. © 2021, CC BY-NC-ND.

关键词： Animation

Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xue, Fanglei Tan, Zichang Zhu, Yu Ma, Zhongsong Guo, Guodong University of Chinese Academy of Sciences Beijing China Key Laboratory of Space Utilization Technology and Engineering Center for Space Utilization Chinese Academy of Sciences Beijing China Institute of Deep Learning Baidu Research Beijing China National Engineering Laboratory for Deep Learning Technology and Application Beijing China

Facial expression recognition plays an important role in human-computer interaction. In this paper, we propose the Coarse-to-Fine Cascaded network with Smooth Predicting (CFC-SP) to improve the performance of facial expression recognition. CFC-SP contains two core components, namely Coarse-to-Fine Cascaded networks (CFC) and Smooth Predicting (SP). For CFC, it first groups several similar emotions to form a rough category, and then employs a network to conduct a coarse but accurate classification. Later, an additional network for these grouped emotions is further used to obtain fine-grained predictions. For SP, it improves the recognition capability of the model by capturing both universal and unique expression features. To be specific, the universal features denote the general characteristic of facial emotions within a period and the unique features denote the specific characteristic at this moment. Experiments on Aff-Wild2 show the effectiveness of the proposed CFSP. We achieved 3rd place in the Expression Classification Challenge of the 3rd Competition on Affective Behavior Analysis in-the-wild. The code will be released at https://***/BR-IDL/PaddleViT. © 2022, CC BY.

关键词： Forecasting

Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Xue, Fanglei Wang, Qiangchang Tan, Zichang Ma, Zhongsong Guo, Guodong University of Chinese Academy of Sciences The Key Laboratory of Space Utilization Technology and Engineering Center for Space Utilization Chinese Academy of Sciences Beijing China West Virginia University Morgantown United States Institute of Deep Learning Baidu Research National Engineering Laboratory for Deep Learning Technology and Application Beijing China

Facial Expression Recognition (FER) in the wild is an extremely challenging task. Recently, some Vision Transformers (ViT) have been explored for FER, but most of them perform inferiorly compared to Convolutional Neural Networks (CNN). This is mainly because the new proposed modules are difficult to converge well from scratch due to lacking inductive bias and easy to focus on the occlusion and noisy areas. TransFER, a representative transformer-based method for FER, alleviates this with multi-branch attention dropping but brings excessive computations. On the contrary, we present two attentive pooling (AP) modules to pool noisy features directly. The AP modules include Attentive Patch Pooling (APP) and Attentive Token Pooling (ATP). They aim to guide the model to emphasize the most discriminative features while reducing the impacts of less relevant features. The proposed APP is employed to select the most informative patches on CNN features, and ATP discards unimportant tokens in ViT. Being simple to implement and without learnable parameters, the APP and ATP intuitively reduce the computational cost while boosting the performance by ONLY pursuing the most discriminative features. Qualitative results demonstrate the motivations and effectiveness of our attentive poolings. Besides, quantitative results on six in-the-wild datasets outperform other state-of-the-art methods. © 2022, CC BY.

关键词： Convolutional neural networks