检索结果-内蒙古大学图书馆

More Grounded Image Captioning by Distilling Image-Text Matching Model

学校读者我要写书评

暂无评论

More Grounded Image Captioning by Distilling Image-Text Matc...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yuanen Zhou Meng Wang Daqing Liu Zhenzhen Hu Hanwang Zhang Key Laboratory of Knowledge Engineering with Big Data (Ministry of Education) School of Computer Science and Information Engineering Hefei University of Technology University of Science and Technology of China Nanyang Technological University

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Visual attention not only improves the performance of image captioners, but also serves as a visual interpretation to qualitatively measure the caption rationality and model transparency. Specifically, we expect that a captioner can fix its attentive gaze on the correct objects while generating the corresponding words. This ability is also known as grounded image captioning. However, the grounding accuracy of existing captioners is far from *** improve the grounding accuracy while retaining the captioning quality, it is expensive to collect the word-region alignment as strong *** this end, we propose a Part-of-Speech (POS) enhanced image-text matching model (SCAN[24]): POS-SCAN, as the effective knowledge distillation for more grounded image captioning. The benefits are two-fold: 1) given a sentence and an image, POS-SCAN can ground the objects more accurately than SCAN; 2) POS-SCAN serves as a word-region alignment regularization for the captioner's visual attention module. By showing benchmark experimental results, we demonstrate that conventional image captioners equipped with POS-SCAN can significantly improve the grounding accuracy without strong supervision. Last but not the least, we explore the indispensable Self-Critical Sequence Training (SCST[46]) in the context of grounded image captioning and show that the image-text matching score can serve as a reward for more grounded captioning 1 .

关键词： Visualization Grounding Task analysis Training Measurement Computational modeling Image edge detection

Multi-objective Optimization for Multi-UAV-assisted Mobile Edge Computing

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Geng Wang, Yixian Sun, Zemin Wu, Qingqing Kang, Jiawen Niyato, Dusit Leung, Victor C.M. College of Computer Science and Technology Jilin University Changchun130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering Ministry of Education Jilin University Changchun130012 China College of Computing and Data Science Nanyang Technological University Singapore639798 Singapore Department of Electronic Engineering Shanghai Jiao Tong University Shanghai China School of Automation Guangdong University of Technology Guangzhou510006 China College of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China Department of Electrical and Computer Engineering University of British Columbia VancouverBCV6T 1Z4 Canada

Recent developments in unmanned aerial vehicles (UAVs) and mobile edge computing (MEC) have provided users with flexible and resilient computing services. However, meeting the computing-intensive and latency-sensitive demands of users poses a significant challenge due to the limited resources of UAVs. To address this challenge, we present a multi-objective optimization approach for multi-UAV-assisted MEC systems. First, we formulate a multi-objective optimization problem aiming at minimizing the total task completion delay, reducing the total UAV energy consumption, and maximizing the total amount of offloaded tasks by jointly optimizing task offloading, computation resource allocation, and UAV trajectory control. Since the problem is a mixed-integer non-linear programming (MINLP) and NP-hard problem which is challenging, we propose a joint task offloading, computation resource allocation, and UAV trajectory control (JTORATC) approach to solve the problem. However, since the decision variables of task offloading, computation resource allocation, and UAV trajectory control are coupled with each other, the original problem is split into three sub-problems, i.e., task offloading, computation resource allocation, and UAV trajectory control, which are solved individually to obtain the corresponding decisions. Moreover, the sub-problem of task offloading is solved by using distributed splitting and threshold rounding methods, the sub-problem of computation resource allocation is solved by adopting the Karush-Kuhn-Tucker (KKT) method, and the sub-problem of UAV trajectory control is solved by employing the successive convex approximation (SCA) method. Simulation results show that the proposed JTORATC has superior performance compared to the other benchmark methods. Copyright © 2024, The Authors. All rights reserved.

关键词： Unmanned aerial vehicles (UAV)

Exploring A CAM - Based Approach for Weakly Supervised Fire Detection Task

学校读者我要写书评

暂无评论

Exploring A CAM - Based Approach for Weakly Supervised Fire ...

IEEE International Conference on e-Business engineering (ICEBE)

作者： Lvlong Lai Jian Chen Huichou Huang Qingyao Wu School of Software Engineering South China University of Technology Guangzhou China Key Laboratory of Big Data and Intelligent Robot Ministry of Education City University of Hong Kong Hong Kong China Pazhou Lab Guangzhou China

ISBN: (纸本)9781665444194

Most existing works in fire detection literature use available detectors like Faster RCNN, SSD, YOLO, etc. to localize the fire in images. These approaches work well but require object-level annotation for training, which is created manually and is very expensive. In this paper, we explore the weakly supervised fire detection task (WSFD) in which only the image-level annotation is given. We propose an approach based on class activation map (CAM). The CAM-based approach firstly trains a deep neural network as the classifier for identifying fire and non-fire images. For a fire image in the inference stage, it uses the classifier to create a CAM and then further generates the bounding boxes according to the CAM. To evaluate the effectiveness of our approach, we collect and construct a benchmark dataset named WS-FireNet and conduct comprehensive experiments on it. The experiment results show that in a way the performance of our approach is satisfactory.

关键词： Training Deep learning Annotations IEEE Sections Conferences Neural networks Detectors

GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yaning Yu, Zitong Wang, Tianyi Huang, Xiaobin Shen, Linlin Gao, Zan Ren, Jianfeng Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China School of Computing and Information Technology Great Bay University Dongguan523000 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China Nanyang Technological University 50 Nanyang Ave Block N 4 639798 Singapore Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen518129 China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University China Jinan250014 China Key Laboratory of Computer Vision and System Ministry of Education Tianjin University of Technology Tianjin300384 China School of Computer Science University of Nottingham Ningbo China

The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Thus, benchmarking and advancing techniques detecting digital manipulation become an urgent issue. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology, which does not involve the most recent technologies like diffusion. The diversity and quality of images generated by diffusion models have been significantly improved and thus a much more challenging face forgery dataset shall be used to evaluate SOTA forgery detection literature. In this paper, we propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection, which contains a large number of forgery faces generated by advanced generators such as the diffusion-based model and more detailed labels about the manipulation approaches and adopted generators. In addition to evaluating SOTA approaches on our benchmark, we design an innovative Cross Appearance-Edge Learning (CAEL) detector to capture multi-grained appearance and edge global representations, and detect discriminative and general forgery traces. Moreover, we devise an Appearance-Edge Cross-Attention (AECA) module to explore the various integrations across two domains. Extensive experiment results and visualizations show that our detection model outperforms the state of the arts on different settings like cross-generator, cross-forgery, and cross-dataset evaluations. Code and datasets will be available at https://***/Jenine-321/GenFace. Copyright © 2024, The Authors. All rights reserved.

关键词： Image enhancement

Noise-Tolerant Learning for Audio-Visual Action Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Haochen Zheng, Qinghua Luo, Minnan Miao, Kaiyao Tian, Feng Chen, Yan The National Engineering Lab for Big Data Analytics Xi’an Jiaotong University Xi’an710049 China The School of Computer Science and Technology Xi’an Jiaotong University Xi’an710049 China The Key Laboratory of Intelligent Networks and Network Security Xi’an Jiaotong University Ministry of Education Xi’an710049 China The School of Cyber Science and Engineering Xi’an Jiaotong University Xi’an710049 China

Recently, video recognition is emerging with the help of multi-modal learning, which focuses on integrating distinct modalities to improve the performance or robustness of the model. Although various multi-modal learning methods have been proposed and offer remarkable recognition results, almost all of these methods rely on high-quality manual annotations and assume that modalities among multi-modal data provide semantically relevant information. Unfortunately, the widely used video datasets are usually coarse-annotated or collected from the Internet. Thus, it inevitably contains a portion of noisy labels and noisy correspondence. To address this challenge, we use the audio-visual action recognition task as a proxy and propose a noise-tolerant learning framework to find anti-interference model parameters against both noisy labels and noisy correspondence. Specifically, our method consists of two phases that aim to rectify noise by the inherent correlation between modalities. First, a noise-tolerant contrastive training phase is performed to make the model immune to the possible noisy-labeled data. Despite the benefits brought by contrastive training, it would overfit the noisy correspondence and thus provide false supervision. To alleviate the influence of noisy correspondence, we propose a cross-modal noise estimation component to adjust the consistency between different modalities. As the noisy correspondence existed at the instance level, we further propose a category-level contrastive loss to reduce its interference. Second, in the hybrid-supervised training phase, we calculate the distance metric among features to obtain corrected labels, which are used as complementary supervision to guide the training. Furthermore, due to the lack of suitable datasets, we establish a benchmark of real-world noisy correspondence in audio-visual data by relabeling the Kinetics dataset. Extensive experiments on a wide range of noisy levels demonstrate that our method significant

关键词： Modal analysis

Multi-frame joint enhancement for early interlaced videos

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhao, Yang Ma, Yanbo Chen, Yuan Jia, Wei Wang, Ronggang Liu, Xiaoping The Key Laboratory of Knowledge Engineering with Big Data Ministry of Education Hefei University of Technology Hefei230009 China The School of Computer Science and Information Engineering Hefei University of Technology Hefei230009 China The School of Electronic and Computer Engineering Peking University Shenzhen Graduate School 2199 Lishui Road Shenzhen518055 China Peng Cheng Laboratory Shenzhen518000 China

Early interlaced videos usually contain multiple and interlacing and complex compression artifacts, which significantly reduce the visual quality. Although the high-definition reconstruction technology for early videos has made great progress in recent years, related research on deinterlacing is still lacking. Traditional methods mainly focus on simple interlacing mechanism, and cannot deal with the complex artifacts in real-world early videos. Recent interlaced video reconstruction deep deinterlacing models only focus on single frame, while neglecting important temporal information. Therefore, this paper proposes a multiframe deinterlacing network joint enhancement network for early interlaced videos that consists of three modules, i.e., spatial vertical interpolation module, temporal alignment and fusion module, and final refinement module. The proposed method can effectively remove the complex artifacts in early videos by using temporal redundancy of multi-fields. Experimental results demonstrate that the proposed method can recover high quality results for both synthetic dataset and real-world early interlaced videos. Copyright © 2021, The Authors. All rights reserved.

关键词： Image reconstruction

Entransy functions for steady heat transfer

学校读者我要写书评

暂无评论

Science China(Technological Sciences) 2019年第10期62卷 1726-1734页

作者： CHENG XueTao LIANG XinGang Big Data Center of Maanshan City Maanshan243000China Bureau of Data Resource Management of Maanshan City Maanshan243000China Key Laboratory for Thermal Science and Power Engineering of Ministry of Education School of AerospaceTsinghua UniversityBeijing100084China

In this paper, the entransy functions for steady heat transfer are summarized and discussed based on the variational theory and the entransy theory. The entransy functions for steady convective heat transfer are derived for the first time. In steady heat transfer processes, it is shown that the steady distributions of heat flux and temperature(radiative thermal potential) should make the corresponding entransy functions reach their minimum values when the temperature(radiative thermal potential) or the heat flux of the boundary is given. The extremum entransy dissipation principles and the minimum entransy-dissipation-based thermal resistance principles are compared with the entransy functions. It is shown that the entransy functions can describe a steady state,but cannot directly give a way to optimize heat transfer processes, while the extremum entransy dissipation principles and the minimum entransy-dissipation-based thermal resistance principles act in an opposite way.

关键词： entransy entransy dissipation entransy function steady state heat transfer

Effect of calcium alumina silicate hydrate nano-seeds on the hydration of low clinker cement

学校读者我要写书评

暂无评论

Journal of Building engineering 2023年 66卷

作者： Liu, Xiao Luo, Qifeng Xie, Hui Li, Shiyu Zhang, Jiale Xia, Chunlei Ding, Yungchin Chen, Yichen Gao, Ruijun Wei, Zhongyuan Zhou, Wu Wang, Ziming Cui, Suping Faculty of Materials and Manufacturing Key Laboratory of Advanced Functional Materials Ministry of Education Beijing University of Technology Beijing100124 China Faculty of Materials and Manufacturing National Engineering Laboratory for Industrial Big-data Application Technology Beijing University of Technology Beijing100124 China Beijing Municipal Engineering Research Institute Beijing100037 China Institute of Mineral Resources Engineering National Taipei University of Technology Taipei10608 China China Building Materials Academy Beijing100024 China Tangshan Longyi Tech. Co Ltd. Tangshan063004 China Guangxi Keda Building Material Chemicals Co Ltd. Nanning530221 China

The replacement of cement clinker with supplementary cementitious materials (SCMs) is considered a very effective way of carbon reduction. However, the slow pozzolanic reaction of SCMs weakens the early strength, which hinders its application. In this study, the dedicated nucleating agents for Al-rich SCMs, C-A-S-H nano-seeds, were synthesized by co-precipitation using polycarboxylate superplasticizer (PCE) as dispersant. The performances of four nano-seeds with different Al/Si and their effects on hydration in 50% metakaolin cement (MK50) were investigated in detail. The results showed that C-A-S-H nano-seeds had a greater strengthening effect than C-S-H nano-seeds especially at early ages, and also accelerated setting, with Al/Si = 0.10 being the most significant. The Al/Si = 0.10 nano-seeds advanced the second main exothermic peak by 6.7 h, promoting the production of stratlingite and causing the highest contents of CH and bound water in hydrated samples. It was also demonstrated by 27Al NMR that C-A-S-H nano-seeds enhanced the reaction degree of metakaolin by producing more C-A-S-H than AFt or AFm. The pozzolanic activity was further explored in the reaction between metakaolin and CH. This study aims to develop an efficient strengthening agent (i.e., C-A-S-H nano-seeds) for low clinker cement, and to reveal the mechanistic difference between them and common C-S-H seeds from the perspectives of hydration and pozzolanic reactions, which is of great significance for the carbon emission reduction of cement industries and solid waste utilization. © 2023 Elsevier Ltd

关键词： Hydration

Multi-Modal Cross-Domain Alignment Network for Video Moment Retrieval

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fang, Xiang Liu, Daizong Zhou, Pan Hu, Yuchong Hubei Key Laboratory of Distributed System Security Hubei Engineering Research Center on Big Data Security School of Cyber Science and Engineering Huazhong University of Science and Technology Hubei Wuhan430074 China Wangxuan Institute of Computer Technology Peking University No. 128 Zhongguancun North Street Beijing China School of Computer Science and Technology Key Laboratory of Information Storage System Ministry of Education of China Huazhong University of Science and Technology Wuhan430074 China

As an increasingly popular task in multimedia information retrieval, video moment retrieval (VMR) aims to localize the target moment from an untrimmed video according to a given language query. Most previous methods depend heavily on numerous manual annotations (i.e., moment boundaries), which are extremely expensive to acquire in practice. In addition, due to the domain gap between different datasets, directly applying these pre-trained models to an unseen domain leads to a significant performance drop. In this paper, we focus on a novel task: cross-domain VMR, where fully-annotated datasets are available in one domain ("source domain"), but the domain of interest ("target domain") only contains unannotated datasets. As far as we know, we present the first study on cross-domain VMR. To address this new task, we propose a novel Multi-Modal Cross-Domain Alignment (MMCDA) network to transfer the annotation knowledge from the source domain to the target domain. However, due to the domain discrepancy between the source and target domains and the semantic gap between videos and queries, directly applying trained models to the target domain generally leads to a performance drop. To solve this problem, we develop three novel modules: (i) a domain alignment module is designed to align the feature distributions between different domains of each modality;(ii) a cross-modal alignment module aims to map both video and query features into a joint embedding space and to align the feature distributions between different modalities in the target domain;(iii) a specific alignment module tries to obtain the fine-grained similarity between a specific frame and the given query for optimal localization. By jointly training these three modules, our MMCDA can learn domain-invariant and semantic-aligned cross-modal representations. Extensive experiments on three challenging benchmarks (ActivityNet Captions, Charades-STA and TACoS) illustrate that our cross-domain method MMCDA outperforms a

关键词： Alignment