检索结果-内蒙古大学图书馆

More Grounded Image Captioning by Distilling Image-Text Matching Model

学校读者我要写书评

暂无评论

More Grounded Image Captioning by Distilling Image-Text Matc...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yuanen Zhou Meng Wang Daqing Liu Zhenzhen Hu Hanwang Zhang Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education) School of Computer Science and Information Engineering Hefei University of Technology University of Science and Technology of China Nanyang Technological University

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Visual attention not only improves the performance of image captioners, but also serves as a visual interpretation to qualitatively measure the caption rationality and model transparency. Specifically, we expect that a captioner can fix its attentive gaze on the correct objects while generating the corresponding words. This ability is also known as grounded image captioning. However, the grounding accuracy of existing captioners is far from *** improve the grounding accuracy while retaining the captioning quality, it is expensive to collect the word-region alignment as strong *** this end, we propose a Part-of-Speech (POS) enhanced image-text matching model (SCAN[24]): POS-SCAN, as the effective knowledge distillation for more grounded image captioning. The benefits are two-fold: 1) given a sentence and an image, POS-SCAN can ground the objects more accurately than SCAN; 2) POS-SCAN serves as a word-region alignment regularization for the captioner's visual attention module. By showing benchmark experimental results, we demonstrate that conventional image captioners equipped with POS-SCAN can significantly improve the grounding accuracy without strong supervision. Last but not the least, we explore the indispensable Self-Critical Sequence Training (SCST[46]) in the context of grounded image captioning and show that the image-text matching score can serve as a reward for more grounded captioning 1 .

关键词： Visualization Grounding Task analysis Training Measurement Computational modeling Image edge detection

Recovering Accurate Labeling Information from Partially Valid data for Effective Multi-Label Learning

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Li, Ximing Wang, Yang College of Computer Science and Technology Jilin University China Key Laboratory of Symbolic Computation Knowledge Engineering of Ministry of Education Jilin University China Key Laboratory of Knowledge Engineering with Big Data Ministry of Education Hefei University of Technology China School of Computer Sci & Information Engineering Hefei University of Technology China

Partial Multi-label Learning (PML) aims to induce the multi-label predictor from datasets with noisy supervision, where each training instance is associated with several candidate labels but only partially valid. To address the noisy issue, the existing PML methods basically recover the ground-truth labels by leveraging the ground-truth confidence of the candidate label, i.e., the likelihood of a candidate label being a ground-truth one. However, they neglect the information from non-candidate labels, which potentially contributes to the ground-truth label recovery. In this paper, we propose to recover the ground-truth labels, i.e., estimating the ground-truth confidences, from the label enrichment, composed of the relevance degrees of candidate labels and irrelevance degrees of non-candidate labels. Upon this observation, we further develop a novel two-stage PML method, namely Partial MultiLabel Learning with Label Enrichment-Recovery (PML3ER), where in the first stage, it estimates the label enrichment with unconstrained label propagation, then jointly learns the ground-truth confidence and multi-label predictor given the label enrichment. Experimental results validate that PML3ER outperforms the state-of-the-art PML methods. Copyright © 2020, The Authors. All rights reserved.

关键词： Recovery

Iterative context-aware graph inference for visual dialog

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Guo, Dan Wang, Hui Zhang, Hanwang Zha, Zheng-Jun Wang, Meng Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology School of Computer Science and Information Engineering Hefei University of Technology Nanyang Technological University University of Science and Technology of China

Visual dialog is a challenging task that requires the comprehension of the semantic dependencies among implicit visual and textual contexts. This task can refer to the relation inference in a graphical model with sparse contexts and unknown graph structure (relation descriptor), and how to model the underlying context-aware relation inference is critical. To this end, we propose a novel Context-Aware Graph (CAG) neural network. Each node in the graph corresponds to a joint semantic feature, including both object-based (visual) and history-related (textual) context representations. The graph structure (relations in dialog) is iteratively updated using an adaptive top-K message passing mechanism. Specifically, in every message passing step, each node selects the most K relevant nodes, and only receives messages from them. Then, after the update, we impose graph attention on all the nodes to get the final graph embedding and infer the answer. In CAG, each node has dynamic relations in the graph (different related K neighbor nodes), and only the most relevant nodes are attributive to the context-aware relational graph inference. Experimental results on VisDial v0.9 and v1.0 datasets show that CAG outperforms comparative methods. Visualization results further validate the interpretability of our method. Copyright © 2020, The Authors. All rights reserved.

关键词： Message passing

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Fan, Zhaoxin Zhu, Yazhi He, Yulin Sun, Qi Liu, Hongyan He, Jun Key Laboratory of Data Engineering and Knowledge Engineering of MOE School of Information Renmin University of China Beijing China No. 59 Zhongguancun Street Haidian Dist Beijing100872 China Institute of Information Science Beijing Jiaotong University No.3 Shangyuancun Haidian Dist. Beijing China School of Economics and Management Tsinghua University Haidian Dist Beijing100084 China

Object pose detection and tracking has recently attracted increasing attention due to its wide applications in many areas, such as autonomous driving, robotics, and augmented reality. Among methods for object pose detection and tracking, deep learning is the most promising one that has shown better performance than others. However, survey study about the latest development of deep learning-based methods is lacking. Therefore, this study presents a comprehensive review of recent progress in object pose detection and tracking that belongs to the deep learning technical route. To achieve a more thorough introduction, the scope of this study is limited to methods taking monocular RGB/RGBD data as input and covering three kinds of major tasks: instance-level monocular object pose detection, category-level monocular object pose detection, and monocular object pose tracking. In our work, metrics, datasets, and methods of both detection and tracking are presented in detail. Comparative results of current state-of-the-art methods on several publicly available datasets are also presented, together with insightful observations and inspiring future research directions. Copyright © 2021, The Authors. All rights reserved.

关键词： Augmented reality

data-Driven Single Image Deraining: A Comprehensive Review and New Perspectives

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2021年

作者： Zhang, Zhao Wei, Yanyan Zhang, Haijun Yang, Yi Yan, Shuicheng Wang, Meng School of Computer Science and Information Engineering Hefei University of Technology Hefei230009 China Key Laboratory of Knowledge Engineering with Big Data Ministry of Education Hefei University of Technology Hefei230009 China Shenzhen China Centre for Artificial Intelligence University of Technology Sydney SydneyNSW Australia Department of Electrical and Computer Engineering National University of Singapore Singapore Hefei University of Technology China

Single Image Deraining task aims at recovering the rain-free background from an image degraded by rain streaks and rain accumulation. For the powerful fitting ability of deep neural networks and massive training data, data-driven deep SID methods obtained significant improvement over traditional ones. Current SID methods usually focus on improving the deraining performance by proposing different kinds of deraining networks, while neglecting the interpretation of the solving process. As a result, the generalization ability may still be limited in real-world scenarios, and the deraining results also cannot effectively improve the performance of subsequent high-level tasks (e.g., object detection). To explore these issues, we in this paper re-examine the three important factors (i.e., data, rain model and network architecture) for the SID problem, and specifically analyze them by proposing new and more reasonable criteria (i.e., general vs. specific, synthetical vs. mathematical, black-box vs. white-box). We also study the relationship of the three factors from a new perspective of data, and reveal two different solving paradigms (explicit vs. implicit) for the SID task. We further discuss the current mainstream data-driven SID methods from five aspects, i.e., training strategy, network pipeline, domain knowledge, data preprocessing, and objective function, and some useful conclusions are summarized by statistics. Besides, we profoundly studied one of the three factors, i.e., data, and measured the performance of current methods on different datasets through extensive experiments to reveal the effectiveness of SID data. Finally, with the comprehensive review and in-depth analysis, we draw some valuable conclusions and suggestions for future research. © 2021, CC BY.

关键词： Object detection

Iterative Context-Aware Graph Inference for Visual Dialog

学校读者我要写书评

暂无评论

Iterative Context-Aware Graph Inference for Visual Dialog

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Dan Guo Hui Wang Hanwang Zhang Zheng-Jun Zha Meng Wang Key Laboratory of Knowledge Engineering with Big Data Hefei University of Technology School of Computer Science and Information Engineering Hefei University of Technology Nanyang Technological University University of Science and Technology of China

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

关键词： Visualization History Task analysis Semantics Message passing Neural networks Cognition

Research on Influencing Factors of College Students' Intention of Online Health Information Behavior Based on Social Cognitive Theory 2

学校读者我要写书评

暂无评论

Research on Influencing Factors of College Students' Intenti...

2019 2nd International Conference on Advanced Algorithms and Control engineering, ICAACE 2019

作者： Kong, Jia Deng, Sanhong Zhang, Yue School of Information Management Nanjing University Nanjing210023 China Jiangsu Key Laboratory of Data Engineering and Knowledge Service Nanjing University Nanjing210023 China

Based on the Social Cognition Theory, we constructed the influencing factors model of college students' intention of online health information behavior from three levels of individual, society and information systems, which was analyzed by using the PLS-SEM. Research shows that self-efficacy, information service quality, social support, information literacy and information quality significantly influence college students' intention of online health information behavior. However, the influence of perceived risk and information system quality is not significant. © 2019 IOP Publishing Ltd. All rights reserved.

关键词： Information systems

A review on vision-based analysis for automatic dietary assessment

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Wang, Wei Min, Weiqing Li, Tianhao Dong, Xiaoxiao Li, Haisheng Jiang, Shuqiang School of Computer and Engineering Beijing Technology and Business University Beijing100048 China The Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China Beijing Key Laboratory of Big Data Technology for Food Safety Beijing100048 China National Engineering Laboratory for Agri-product Quality Traceability Beijing100048 China University of Chinese Academy of Sciences Beijing100049 China

Background: Maintaining a healthy diet is vital to avoid health-related issues, e.g., undernutrition, obesity and many non-communicable diseases. An indispensable part of the health diet is dietary assessment. Traditional manual recording methods are not only burdensome but timeconsuming, and contain substantial biases and errors. Recent advances in Artificial Intelligence (AI), especially computer vision technologies, have made it possible to develop automatic dietary assessment solutions, which are more convenient, less time-consuming and even more accurate to monitor daily food intake. Scope and approach: This review presents Vision-Based Dietary Assessment (VBDA) architectures, including multi-stage architecture and end-to-end one. The multi-stage dietary assessment generally consists of three stages: food image analysis, volume estimation and nutrient derivation. The prosperity of deep learning makes VBDA gradually move to an end-to-end implementation, which applies food images to a single network to directly estimate the nutrition. The recently proposed end-to-end methods are also discussed. We further analyze existing dietary assessment datasets, indicating that one large-scale benchmark is urgently needed, and finally highlight critical challenges and future trends for VBDA. key findings and conclusions: After thorough exploration, we find that multi-task end-to-end deep learning approaches are one important trend of VBDA. Despite considerable research progress, many challenges remain for VBDA due to the meal complexity. We also provide the latest ideas for future development of VBDA, e.g., fine-grained food analysis and accurate volume estimation. This review aims to encourage researchers to propose more practical solutions for VBDA. Copyright © 2021, The Authors. All rights reserved.

关键词： Nutrition

BCS-Net: Boundary, Context and Semantic for Automatic COVID-19 Lung Infection Segmentation from CT Images

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Cong, Runmin Yang, Haowei Jiang, Qiuping Gao, Wei Li, Haisheng Wang, Cong Zhao, Yao Kwong, Sam The Institute of Information Science Beijing Jiaotong University Beijing100044 China Beijing Key Laboratory of Big Data Technology for Food Safety Beijing Technology and Business University Beijing100048 China The Department of Computer Science City University of Hong Kong Hong Kong The Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing100044 China The School of Information Science and Engineering Ningbo University Ningbo315211 China The School of Electronic and Computer Engineering Peking University Shenzhen Graduate School Shenzhen518055 China Peng Cheng Laboratory Shenzhen518055 China The Distributed and Parallel Software Lab Huawei Technologies Shenzhen518129 China The City University of Hong Kong Shenzhen Research Institute Shenzhen51800 China

The spread of COVID-19 has brought a huge disaster to the world, and the automatic segmentation of infection regions can help doctors to make diagnosis quickly and reduce workload. However, there are several challenges for the accurate and complete segmentation, such as the scattered infection area distribution, complex background noises, and blurred segmentation boundaries. To this end, in this paper, we propose a novel network for automatic COVID-19 lung infection segmentation from CT images, named BCS-Net, which considers the boundary, context, and semantic attributes. The BCS-Net follows an encoder-decoder architecture, and more designs focus on the decoder stage that includes three progressively Boundary-Context-Semantic Reconstruction (BCSR) blocks. In each BCSR block, the attention-guided global context (AGGC) module is designed to learn the most valuable encoder features for decoder by highlighting the important spatial and boundary locations and modeling the global context dependence. Besides, a semantic guidance (SG) unit generates the semantic guidance map to refine the decoder features by aggregating multi-scale high-level features at the intermediate resolution. Extensive experiments demonstrate that our proposed framework outperforms the existing competitors both qualitatively and quantitatively. © 2022, CC BY-NC-SA.

关键词： COVID-19