检索结果-内蒙古大学图书馆

DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Jiang, Xiaoze Yu, Jing Sun, Yajing Qin, Zengchang Zhu, Zihao Hu, Yue Wu, Qi Institute of Information Engineering Chinese Academy of Sciences Beijing China Intelligent Computing and Machine Learning Lab School of ASEE Beihang University Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China AI Research Codemao Inc University of Adelaide Australia

Visual Dialogue task requires an agent to be engaged in a conversation with human about an image. The ability of generating detailed and non-repetitive responses is crucial for the agent to achieve human-like conversation. In this paper, we propose a novel generative decoding architecture to generate high-quality responses, which moves away from decoding the whole encoded semantics towards the design that advocates both transparency and flexibility. In this architecture, word generation is decomposed into a series of attention-based information selection steps, performed by the novel recurrent Deliberation, Abandon and Memory (DAM) module. Each DAM module performs an adaptive combination of the response-level semantics captured from the encoder and the word-level semantics specifically selected for generating each word. Therefore, the responses contain more detailed and non-repetitive descriptions while maintaining the semantic accuracy. Furthermore, DAM is flexible to cooperate with existing visual dialogue encoders and adaptive to the encoder structures by constraining the information selection mode in DAM. We apply DAM to three typical encoders and verify the performance on the VisDial v1.0 dataset. Experimental results show that the proposed models achieve new state-of-the-art performance with high-quality responses. The code is available at https://***/JXZe/DAM. Copyright © 2020, The Authors. All rights reserved.

关键词： Signal encoding

Multi-level network for high-speed multi-person pose estimation

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Huang, Ying Zhuang, Jiankai Qin, Zengchang Alibaba Business School Hangzhou Normal University Hangzhou Intelligent Computing and Machine Learning Lab School of Asee Beihang University Beijing

In multi-person pose estimation, the left/right joint type discrimination is always a hard problem because of the similar appearance. Traditionally, we solve this problem by stacking multiple refinement modules to increase network's receptive fields and capture more global context, which can also increase a great amount of computation. In this paper, we propose a Multi-level Network (MLN) that learns to aggregate features from lower-level (left/right information), upper-level (localization information), joint-limb level (complementary information) and global-level (context) information for discrimination of joint type. Through feature reuse and its intrarelation, MLN can attain comparable performance to other conventional methods while runtime speed retains at 42 FPS. Copyright © 2019, The Authors. All rights reserved.

关键词： Joints (anatomy)

Generative cooperative net for image generation and data augmentation 7th

学校读者我要写书评

暂无评论

Generative cooperative net for image generation and data aug...

7th International Symposium on Integrated Uncertainty in Knowledge Modelling and Decision Making, IUKM 2019

作者： Xu, Qiangeng Qin, Zengchang Wan, Tao Intelligent Computing and Machine Learning Lab School of ASEE Beihang University Beijing China Department of Computer Science University of Southern California Los Angeles United States Keep Labs Keep Inc. Beijing China School of Biological Science and Medical Engineering Beihang University Beijing China

ISBN: (纸本)9783030148140

How to build a good model for image generation given an abstract concept is one of fundamental problems in computer vision. In this paper, we explore a generative model for the task of generating fictitious images with desired features. We propose the Generative Cooperative Net (GCN) for image generation. The idea is similar to generative adversarial networks (GANs) except that the generators and discriminators are trained to work accordingly. We conducted experiments on hand-written digit generation and facial expression generation. In experimental studies, we found that GCN’s two cooperative counterparts (the generator and the classifier) can work together nicely and achieve promising results. Such generative model can be used as a data-augmentation tool. Our experiment of applying this method to an emotion classification task shows that the synthesis images can help to improve classification accuracy. It is easy to set up and generate a very large synthesized dataset. © Springer Nature Switzerland AG 2019.

关键词： Generative adversarial networks

Followmeup sports: new benchmark for 2D human keypoint recognition

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Huang, Ying Sun, Bin Kan, Haipeng Zhuang, Jiankai Qin, Zengchang Alibaba Business School Hangzhou Normal University Hangzhou China Keep Inc. Beijing China Intelligent Computing and Machine Learning Lab School of ASEE Beihang University Beijing China

Human pose estimation has made significant advancement in recent years. However, the existing datasets are limited in their coverage of pose variety. In this paper, we introduce a novel benchmark"FollowMeUp Sports" that makes an important advance in terms of specific postures, self-occlusion and class balance, a contribution that we feel is required for future development in human body models. This comprehensive dataset was collected using an established taxonomy of over 200 standard workout activities with three different shot angles. The collected videos cover a wider variety of specific workout activities than previous datasets including push-up, squat and body moving near the ground with severe self-occlusion or occluded by some sport equipment and outfits. Given these rich images, we perform a detailed analysis of the leading human pose estimation approaches gaining insights for the success and failures of these methods. Copyright © 2019, The Authors. All rights reserved.

关键词： Sports

DualVD: An adaptive dual encoding model for deep visual understanding in visual dialogue

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Jiang, Xiaoze Yu, Jing Qin, Zengchang Zhuang, Yingying Zhang, Xingxing Hu, Yue Wu, Qi Institute of Information Engineering Chinese Academy of Sciences Beijing China Intelligent Computing and Machine Learning Lab School of ASEE Beihang University Beijing China Microsoft Research Asia Beijing China University of Adelaide Australia

Different from Visual Question Answering task that requires to answer only one question about an image, Visual Dialogue involves multiple questions which cover a broad range of visual content that could be related to any objects, relationships or semantics. The key challenge in Visual Dialogue task is thus to learn a more comprehensive and semantic-rich image representation which may have adaptive attentions on the image for variant questions. In this research, we propose a novel model to depict an image from both visual and semantic perspectives. Specifically, the visual view helps capture the appearance-level information, including objects and their relationships, while the semantic view enables the agent to understand high-level visual semantics from the whole image to the local regions. Futhermore, on top of such multiview image features, we propose a feature selection framework which is able to adaptively capture question-relevant information hierarchically in fine-grained level. The proposed method achieved state-of-the-art results on benchmark Visual Dialogue datasets. More importantly, we can tell which modality (visual or semantic) has more contribution in answering the current question by visualizing the gate values. It gives us insights in understanding of human cognition in Visual Dialogue. Copyright © 2019, The Authors. All rights reserved.

关键词： Semantics

Scene Graph Reasoning with Prior Visual Relationship for Visual Question Answering

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yang, Zhuoqian Qin, Zengchang Yu, Jing Hu, Yue Intelligent Computing and Machine Learning Lab School of Automation Science and Electrical Engineering Beihang University Institute of Information Engineering Chinese Academy of Sciences

One of the key issues of Visual Question Answering (VQA) is to reason with semantic clues in the visual content under the guidance of the question, how to model relational semantics still remains as a great challenge. To fully capture visual semantics, we propose to reason over a structured visual representation – scene graph, with embedded objects and inter-object relationships. This shows great benefit over vanilla vector representations and implicit visual relationship learning. Based on existing visual relationship models, we propose a visual relationship encoder that projects visual relationships into a learned deep semantic space constrained by visual context and language priors. Upon the constructed graph, we propose a Scene Graph Convolutional Network (SceneGCN) to jointly reason the object properties and relational semantics for the correct answer. We demonstrate the model’s effectiveness and interpretability on the challenging GQA dataset and the classical VQA 2.0 dataset, remarkably achieving state-of-the-art 54.56% accuracy on GQA compared to the existing best model. Copyright © 2018, The Authors. All rights reserved.

关键词： Graph neural networks

Pixel level data augmentation for semantic image segmentation using generative adversarial networks

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Liu, Shuangting Zhang, Jiaqi Chen, Yuxin Liu, Yifan Qin, Zengchang Wan, Tao Intelligent Computing and Machine Learning Lab School of ASEE Beihang University China Keep Labs Keep Inc School of Biological Science and Medical Engineering Beihang University

Semantic segmentation is one of the basic topics in computer vision, it aims to assign semantic labels to every pixel of an image. Unbalanced semantic label distribution could have a negative influence on segmentation accuracy. In this paper, we investigate using data augmentation approach to balance the semantic label distribution in order to improve segmentation performance. We propose using generative adversarial networks (GANs) to generate realistic images for improving the performance of semantic segmentation networks. Experimental results show that the proposed method can not only improve segmentation performance on those classes with low accuracy, but also obtain 1.3% to 2.1% increase in average segmentation accuracy. It shows that this augmentation method can boost accuracy and be easily applicable to any other segmentation models. Copyright © 2018, The Authors. All rights reserved.

关键词： Generative adversarial networks

Semantic modeling of textual relationships in cross-modal retrieval

学校读者我要写书评

暂无评论

arXiv 2018年

作者： Yu, Jing Yang, Chenghao Qin, Zengchang Yang, Zhuoqian Hu, Yue Zhang, Weifeng Institute of Information Engineering Chinese Academy of Sciences China Intelligent Computing and Machine Learning Lab Beihang University China College of Mathematics Physics and Information Engineering Jiaxing University China

Feature modeling of different modalities is a basic problem in current research of cross-modal information retrieval. Existing models typically project texts and images into one embedding space, in which semantically similar information will have a shorter distance. Semantic modeling of textural relationships is notoriously difficult. In this paper, we propose an approach to model texts using a featured graph by integrating multi-view textual relationships including semantic relations, statistical co-occurrence, and prior relations in knowledge base. A dualpath neural network is adopted to learn multi-modal representations of information and cross-modal similarity measure jointly. We use a Graph Convolutional Network (GCN) for generating relation-aware text representations, and use a Convolutional Neural Network (CNN) with nonlinearities for image representations. The cross-modal similarity measure is learned by distance metric learning. Experimental results show that, by leveraging the rich relational semantics in texts, our model can outperform the state-of-the-art models by 3.4% and 6.3% on accuracy on two benchmark datasets. Copyright © 2018, The Authors. All rights reserved.

关键词： Knowledge graph

Emotion classification with data augmentation using generative adversarial networks 22nd

学校读者我要写书评

暂无评论

Emotion classification with data augmentation using generati...

22nd Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2018

作者： Zhu, Xinyue Liu, Yifan Li, Jiahong Wan, Tao Qin, Zengchang Intelligent Computing and Machine Learning Lab School of ASEE Beihang University Beijing China School of Electronic Engineering Bejing University of Posts and Telecommunications Beijing China Beijing San Kuai Yun Technology Co. Ltd. Beijing China School of Biological Science and Medical Engineering Beijing Advanced Innovation Centre for Biomedical Engineering Beihang University Beijing China

ISBN: (纸本)9783319930398

It is a difficult task to classify images with multiple class labels using only a small number of labeled examples, especially when the label (class) distribution is imbalanced. Emotion classification is such an example of imbalanced label distribution, because some classes of emotions like disgusted are relatively rare comparing to other labels like happy or sad. In this paper, we propose a data augmentation method using generative adversarial networks (GAN). It can complement and complete the data manifold and find better margins between neighboring classes. Specifically, we design a framework using a CNN model as the classifier and a cycle-consistent adversarial networks (CycleGAN) as the generator. In order to avoid gradient vanishing problem, we employ the least-squared loss as adversarial loss. We also propose several evaluation methods on three benchmark datasets to validate GAN’s performance. Empirical results show that we can obtain 5%–10% increase in the classification accuracy after employing the GAN-based data augmentation techniques. © Springer International Publishing AG, part of Springer Nature 2018.

关键词： Generative adversarial networks