检索结果-内蒙古大学图书馆

International Conference on Automatic Face and Gesture Recognition

作者： Ruikui Wang Shishi Qiao Ruiping Wang Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9781728130798

ISBN: (纸本)9781728130804

Video face retrieval (VFR) is an appealing and practical computer vision task, which aims to search particular character from masses of videos like in TV-Series. The challenges of this task mainly lie in two aspects, i.e. faces in such videos contain complex appearance variations with uncontrollab.e shooting environment and searching from big data usually requires high efficiency in both space and time. To fulfill this task, current works typically proceed in a learning to hash manner by fusing single-frame features within a video to obtain the video representation and further embedding it into Hamming space to yield video binary codes. The feature fusion stage has inevitably discarded too much frame information and leads to less discriminative video codes. In this paper, we propose Hybrid Video and Image Hashing (HVIH) to learn more effective binary codes for face videos. Specifically, we fully exploit the dense frame features rather than simply discarding them after the video level fusion and jointly optimize binary codes for the video and its composed frames in adapted supervised manners. To achieve more robust video representation, we introduce a module of video center alignment to ensure the binary codes location of the video and its frames to be as compact and consistent as possible in the Hamming space, which naturally facilitates both tasks of video-to-video retrieval and image-to-video retrieval. Extensive experiments on two challenging video face databases demonstrate the superiority of our approach over the state-of-the-art.

关键词： Binary codes Face recognition Faces Task analysis Feature extraction Databases Robustness

来源：评论

学校读者我要写书评

暂无评论

Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text

Multi-Modal Graph Neural Network for Joint Reasoning on Visi...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Difei Gao Ke Li Ruiping Wang Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Answering questions that require reading texts in an image is challenging for current models. One key difficulty of this task is that rare, polysemous, and ambiguous words frequently appear in images, e.g., names of places, products, and sports teams. To overcome this difficulty, only resorting to pre-trained word embedding models is far from enough. A desired model should utilize the rich information in multiple modalities of the image to help understand the meaning of scene texts, e.g., the prominent text on a bottle is most likely to be the brand. Following this idea, we propose a novel VQA approach, Multi-Modal Graph Neural Network (MM-GNN). It first represents an image as a graph consisting of three sub-graphs, depicting visual, semantic, and numeric modalities respectively. Then, we introduce three aggregators which guide the message passing from one graph to another to utilize the contexts in various modalities, so as to refine the features of nodes. The updated nodes have better features for the downstream question answering module. Experimental evaluations show that our MM-GNN represents the scene texts better and obviously facilitates the performances on two VQA tasks that require reading scene texts.

关键词： Visualization Semantics Task analysis Numerical models Neural networks Cognition Knowledge discovery

来源：评论

学校读者我要写书评

暂无评论

NiuEM: A Nested-iterative Unsupervised Learning Model for Single-particle Cryo-EM Image processing

NiuEM: A Nested-iterative Unsupervised Learning Model for Si...

引用

2020 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2020

作者： Hu, Rui Cai, Jiaming Zheng, Wangjie Yang, Yang Shen, Hong-Bin Shanghai Jiao Tong University Key Lab. of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Department of Computer Science and Engineering Shanghai200240 China Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University Key Laboratory of System Control and Information Processing Ministry of Education of China Shanghai200240 China Shanghai Jiao Tong University Department of Bioinformatics and Biostatistics Shanghai200240 China

ISBN: (纸本)9781728162157

Cryo-electron microscopy (cryo-EM) has become a mainstream technology for solving spatial structures of biomacromolecules, while the processing of cryo-EM images is a very challenging task. One of the great challenges is the high noise in the images. A common method is to cluster the images with close projecting angles to get mean images, which are used for 3D reconstruction. However, due to the extremely low signal-to-noise-ratio, common clustering methods often fail to obtain high-quality mean images, leading to poorly reconstructed structures. In this study, we present a new unsupervised learning framework, called NiuEM, to discriminate images captured from different angles and yield cluster-mean images. NiuEM first generates pseudo-lab.ls and then exploits both contrastive loss and cross-entropy loss for training convolutional layers to learn feature representations. Moreover, the pseudo-lab.ls are updated iteratively to enhance the reliability of lab.ls. We assess the performance of NiuEM on four data sets via both visualized and quantitative experiments. Especially, two kinds of metrics are adopted to measure the performance, regarding the clustering quality and the resolution of reconstructed 3D models, respectively. The experimental results show that NiuEM achieves very competitive clustering accuracy in the comparison with the state-of-the-art image clustering methods. Moreover, the cluster mean images yielded by NiuEM lead to better initial 3D models compared with the mainstream reconstruction tools. © 2020 IEEE.

关键词： Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Locality-aware channel-wise dropout for occluded face recognition

arXiv

引用

arXiv 2021年

作者： He, Mingjie Zhang, Jie Shan, Shiguang Liu, Xiao Wu, Zhongqin Chen, Xilin Key Laboratory of Intelligent Information Processing Chinese Academy of Sciences Institute of Computing Technology CAS Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China CAS Center for Excellence in Brain Science and Intelligence Technology Shanghai200031 China Peng Cheng Laboratory Shenzhen518055 China Beijing100080 China

Face recognition remains a challenging task in unconstrained scenarios, especially when faces are partially occluded. To improve the robustness against occlusion, augmenting the training images with artificial occlusions has been proved as a useful approach. However, these artificial occlusions are commonly generated by adding a black rectangle or several object templates including sunglasses, scarfs and phones, which cannot well simulate the realistic occlusions. In this paper, based on the argument that the occlusion essentially damages a group of neurons, we propose a novel and elegant occlusion-simulation method via dropping the activations of a group of neurons in some elab.rately selected channel. Specifically, we first employ a spatial regularization to encourage each feature channel to respond to local and different face regions. In this way, the activations affected by an occlusion in a local region are more likely to be located in a single feature channel. Then, the locality-aware channel-wise dropout (LCD) is designed to simulate the occlusion by dropping out the entire feature channel. Furthermore, by randomly dropping out several feature channels, our method can well simulate the occlusion of larger area. The proposed LCD can encourage its succeeding layers to minimize the intra-class feature variance caused by occlusions, thus leading to improved robustness against occlusion. In addition, we design an auxiliary spatial attention module by learning a channel-wise attention vector to reweight the feature channels, which improves the contributions of non-occluded regions. Extensive experiments on various benchmarks show that the proposed method outperforms state-of-the-art methods with a remarkable improvement. Copyright © 2021, The Authors. All rights reserved.

关键词： Chemical activation

来源：评论

学校读者我要写书评

暂无评论

Pseudo-Convolutional Policy Gradient for Sequence-to-Sequence Lip-Reading

Pseudo-Convolutional Policy Gradient for Sequence-to-Sequenc...

引用

International Conference on Automatic Face and Gesture Recognition

作者： Mingshuang Luo Shuang Yang Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9781728130798

ISBN: (纸本)9781728130804

Lip-reading aims to infer the speech content from the lip movement sequence and can be seen as a typical sequence-to-sequence (seq2seq) problem which translates the input image sequence of lip movements to the text sequence of the speech content. However, the traditional learning process of seq2seq models always suffers from two problems: the exposure bias resulted from the strategy of “teacher-forcing”, and the inconsistency between the discriminative optimization target (usually the cross-entropy loss) and the final evaluation metric (usually the character/word error rate). In this paper, we propose a novel pseudo-convolutional policy gradient (PCPG) based method to address these two problems. On the one hand, we introduce the evaluation metric (refers to the character error rate in this paper) as a form of reward to optimize the model together with the original discriminative target. On the other hand, inspired by the local perception property of convolutional operation, we perform a pseudo-convolutional operation on the reward and loss dimension, so as to take more context around each time step into account to generate a robust reward and loss for the whole optimization. Finally, we perform a thorough comparison and evaluation on both the word-level and sentence-level benchmarks. The results show a significant improvement over other related methods, and report either a new state-of-the-art performance or a competitive accuracy on all these challenging benchmarks, which clearly proves the advantages of our approach.

关键词： Decoding Measurement Image sequences Computational modeling Task analysis Optimization Error analysis

来源：评论

学校读者我要写书评

暂无评论

Consistency and Diversity induced Human Motion Segmentation

arXiv

引用

arXiv 2022年

作者： Zhou, Tao Fu, Huazhu Gong, Chen Shao, Ling Porikli, Fatih Ling, Haibin Shen, Jianbing The PCA Lab The Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information Ministry of Education School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing210094 China Institute of High Performance Computing Agency for Science Technology and Research Singapore Riyadh Saudi Arabia The Research School of Engineering The Australian National University Australia The Department of Computer Science Stony Brook University Strony Brook NY United States The State Key Laboratory of Internet of Things for Smart City Department of Computer and Information Science University United States

Subspace clustering is a classical technique that has been widely used for human motion segmentation and other related tasks. However, existing segmentation methods often cluster data without guidance from prior knowledge, resulting in unsatisfactory segmentation results. To this end, we propose a novel Consistency and Diversity induced human Motion Segmentation (CDMS) algorithm. Specifically, our model factorizes the source and target data into distinct multi-layer feature spaces, in which transfer subspace learning is conducted on different layers to capture multi-level information. A multi-mutual consistency learning strategy is carried out to reduce the domain gap between the source and target data. In this way, the domain-specific knowledge and domain-invariant properties can be explored simultaneously. Besides, a novel constraint based on the Hilbert Schmidt Independence Criterion (HSIC) is introduced to ensure the diversity of multi-level subspace representations, which enables the complementarity of multi-level representations to be explored to boost the transfer learning performance. Moreover, to preserve the temporal correlations, an enhanced graph regularizer is imposed on the learned representation coefficients and the multi-level representations of the source data. The proposed model can be efficiently solved using the Alternating Direction Method of Multipliers (ADMM) algorithm. Extensive experimental results on public human motion datasets demonstrate the effectiveness of our method against several state-of-the-art approaches. Copyright © 2022, The Authors. All rights reserved.

关键词： Motion analysis

来源：评论

学校读者我要写书评

暂无评论

Deformation Flow Based Two-Stream Network for Lip Reading

Deformation Flow Based Two-Stream Network for Lip Reading

引用

International Conference on Automatic Face and Gesture Recognition

作者： Jingyun Xiao Shuang Yang Yuanhang Zhang Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9781728130798

ISBN: (纸本)9781728130804

Lip reading is the task of recognizing speech content by analyzing movements in the lip region when people are speaking. Based on the continuity in adjacent frames in the speaking process, and the consistency in motion patterns among different people when they pronounce the same phoneme, we model lip movements as a sequence of apparent deformations in the lip region during the speaking process. Specifically, we introduce a Deformation Flow Network (DFN) to learn the deformation flow between adjacent frames, which directly captures the motion information within the lip region. The learned deformation flow is then combined with the original grayscale frames with a two-stream network to perform lip reading. To make the two streams learn from each other in the learning process, we introduce a bidirectional knowledge distillation loss to train the two branches jointly. Owing to the complementary cues provided by different branches, the two-stream network shows substantial improvement over using either single branch. A thorough experimental evaluation on two large-scale lip reading benchmarks is presented with detailed analysis. The results accord with our motivation, and show that our method achieves state-of-the-art or comparable performance on these two challenging datasets.

关键词： Strain Lips Face recognition Task analysis Training Decoding Gray-scale

来源：评论

学校读者我要写书评

暂无评论

M3F: Multi-Modal Continuous Valence-Arousal Estimation in the Wild

M3F: Multi-Modal Continuous Valence-Arousal Estimation in th...

引用

International Conference on Automatic Face and Gesture Recognition

作者： Yuan-Hang Zhang Rulin Huang Jiabei Zeng Shiguang Shan Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9781728130798

ISBN: (纸本)9781728130804

In this paper, we propose a multi-modal multi-feature (M$^{3}F$) approach for in-the-wild valence-arousal estimation. In the proposed M$^{3}F$ framework, we fuse both visual features from videos and acoustic features from the audio tracks to estimate the valence and arousal. We follow a CNN-RNN paradigm, where the spatio-temporal visual features are extracted with a 3D convolutional network and/or a pretrained 2D convolutional network, and a bidirectional recurrent neural network. We evaluated the M$^{3}F$ framework on the validation set provided by the Affective Behavior Analysis in-the-wild (ABAW) Challenge, held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020, and it significantly outperforms the baseline method.

关键词： Feature extraction Visualization Three-dimensional displays Two dimensional displays Videos Training Estimation

来源：评论

学校读者我要写书评

暂无评论

PAS-Net: Pose-based and Appearance-based Spatiotemporal Networks Fusion for Action Recognition

PAS-Net: Pose-based and Appearance-based Spatiotemporal Netw...

引用

International Conference on Automatic Face and Gesture Recognition

作者： Changzhen Li Jie Zhang Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9781728130798

ISBN: (纸本)9781728130804

Human poses play important roles in action analysis. However, most state-of-the-art approaches in action recognition ignore the importance of human poses and rarely leverage the pose information for further improving the recognition performance. In this paper, we propose a novel network architecture, which simultaneously considers the appearance information and pose knowledge for robust action recognition. We explore various architectures for fusing the appearance and pose information rather than simply averaging scores at the final layer. Moreover, a novel training strategy is proposed to reduce the influence of overfitting for limited training data. Extensive experiments show that our method achieves competitive performance on the popular benchmarks, i.e., UCF-101 and HMDB-51.

关键词： Feature extraction Three-dimensional displays Spatiotemporal phenomena Computer architecture Two dimensional displays Fuses Training

来源：评论

学校读者我要写书评

暂无评论

Poisoning attack against estimating from pairwise comparisons

arXiv

引用

arXiv 2021年

作者： Ma, Ke Xu, Qianqian Zeng, Jinshan Cao, Xiaochun Huang, Qingming The School of Computer Science and Technology University of Chinese Academy of Sciences Beijing100049 China The Artificial Intelligence Research Center Peng Cheng Laboratory Shenzhen518055 China The Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China The School of Computer and Information Engineering Jiangxi Normal University Jiangxi Nanchang330022 China Institute of Information Engineering Chinese Academy of Sciences Beijing100093 China The School of Cyber Security University of Chinese Academy of Sciences Beijing100049 China The Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China The School of Computer Science and Technology University of Chinese Academy of Sciences Beijing100049 China The Key Laboratory of Big Data Mining and Knowledge Management The School of Economics and Management University of Chinese Academy of Sciences Beijing100049 China

As pairwise ranking becomes broadly employed for elections, sports competitions, recommendation, information retrieval and so on, attackers have strong motivation and incentives to manipulate or disrupt the ranking list. They could inject malicious comparisons into the training data to fool the target ranking algorithm. Such a technique is called "poisoning attack" in regression and classification tasks. In this paper, to the best of our knowledge, we initiate the first systematic investigation of data poisoning attack on the pairwise ranking algorithms, which can be generally formalized as the dynamic and static games between the ranker and the attacker, and can be modeled as certain kinds of integer programming problems mathematically. To break the computational hurdle of the underlying integer programming problems, we reformulate them into the distributionally robust optimization (DRO) problems, which are computational tractable. Based on such DRO formulations, we propose two efficient poisoning attack algorithms and establish the associated theoretical guarantees including the existence of Nash equilibrium and the generalization ability bounds. The effectiveness of the suggested poisoning attack strategies is demonstrated by a series of toy simulations and several real data experiments. These experimental results show that the proposed methods can significantly reduce the performance of the ranker in the sense that the correlation between the true ranking list and the aggregated results with toxic data can be decreased dramatically. © 2021, CC BY-NC-ND.

关键词： Toys

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：