检索结果-内蒙古大学图书馆

A news image captioning approach based on multimodal pointer-generator network

学校读者我要写书评

暂无评论

A news image captioning approach based on multimodal pointer...

作者： Chen, Jingqiang Zhuge, Hai School of Computer Science Nanjing University of Posts and Telecommunications Nanjing China School of Computer Science and Network Engineering Guangzhou University Guangzhou China Department of Computer Science School of Engineering and Applied Science Aston University Birmingham United Kingdom Key Laboratory of Intelligent Information Processing Institute of Computing Technology University of Chinese Academy of Sciences Chinese Academy of Sciences Beijing China

News image captioning aims to generate captions or descriptions for news images automatically, serving as draft captions for creating news image captions manually. News image captions are different from generic captions as news image captions contain more detailed information such as entity names and events. Therefore, both images on news and the accompanying text are the source of generating caption of news image. Pointer-generator network is a neural method defined for text summarization. This article proposes the Multimodal pointer-generation network by incorporating visual information into the original network for news image captioning. The multimodal attention mechanism is proposed by splitting attention into visual attention paid to the image and textual attention paid to the text. The multimodal pointer mechanism is proposed by using both textual attention and visual attention to compute pointer distributions, where visual attention is first transformed into textual attention via the word-image relationships. The multimodal coverage mechanism is defined to reduce repetitions of attentions or repetitions of pointer distributions. Experiments on the DailyMail test dataset and the out-of-domain BBC test dataset show that the proposed model outperforms the original pointer-generator network, the generic image captioning method, the extractive news image captioning method, and the LDA-based method according BLEU, METEOR, and ROUGL-L evaluations. Experiments also show that the proposed multimodal coverage mechanisms can improve the model, and that transforming visual attention to pointer distributions can improve the model. © 2020 John Wiley & Sons, Ltd.

关键词： Statistical tests

Tiny noise, big mistakes: adversarial perturbations induce errors in brain–computer interface spellers

学校读者我要写书评

暂无评论

National Science Review 2021年第4期8卷 78-90页

作者： Xiao Zhang Dongrui Wu Lieyun Ding Hanbin Luo Chin-Teng Lin Tzyy-Ping Jung Ricardo Chavarriaga Ministry of Education Key Laboratory of Image Processing and Intelligent Control School of Artificial Intelligence and AutomationHuazhong University of Science and Technology School of Civil Engineering and Mechanics Huazhong University of Science and Technology Centre of Artificial Intelligence Faculty of Engineering and Information Technology University of Technology Sydney Swartz Center for Computational Neuroscience Institute for Neural ComputationUniversity of California San Diego Center for Advanced Neurological Engineering Institute of Engineering in Medicine University of California San Diego ZHAW Data Lab Zürich University of Applied Sciences

An electroencephalogram(EEG)-based brain–computer interface(BCI) speller allows a user to input text to a computer by thought. It is particularly useful to severely disabled individuals, e.g. amyotrophic lateral sclerosis patients, who have no other effective means of communication with another person or a *** studies so far focused on making EEG-based BCI spellers faster and more reliable; however, few have considered their security. This study, for the first time, shows that P300 and steady-state visual evoked potential BCI spellers are very vulnerable, i.e. they can be severely attacked by adversarial perturbations,which are too tiny to be noticed when added to EEG signals, but can mislead the spellers to spell anything the attacker wants. The consequence could range from merely user frustration to severe misdiagnosis in clinical applications. We hope our research can attract more attention to the security of EEG-based BCI spellers, and more broadly, EEG-based BCIs, which has received little attention before.

关键词： electroencephalogram brain-computer interfaces BCI spellers adversarial examples

StyleGene: Crossover and Mutation of Region-level Facial Genes for Kinship Face Synthesis

学校读者我要写书评

暂无评论

StyleGene: Crossover and Mutation of Region-level Facial Gen...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Hao Li Xianxu Hou Zepeng Huang Linlin Shen Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University National Engineering Laboratory for Big Data System Computing Technology Shenzhen University School of AI and Advanced Computing Xi'an Jiaotong-Liverpool University Shenzhen Institute of Artificial Intelligence and Robotics for Society Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University

High-fidelity kinship face synthesis has many potential applications, such as kinship verification, missing child identification, and social media analysis. However, it is challenging to synthesize high-quality descendant faces with genetic relations due to the lack of large-scale, high-quality annotated kinship data. This paper proposes RFG (Region-level Facial Gene) extraction framework to address this issue. We propose to use IGE (Image-based Gene Encoder), LGE (Latent-based Gene Encoder) and Gene Decoder to learn the RFGs of a given face image, and the relationships between RFGs and the latent space of Style-GAN2. As cycle-like losses are designed to measure the $\mathcal{L}_{2}$ distances between the output of Gene Decoder and image encoder, and that between the output of LGE and IGE, only face images are required to train our framework, i.e. no paired kinship face data is required. Based upon the proposed RFGs, a crossover and mutation module is further designed to inherit the facial parts of parents. A Gene Pool has also been used to introduce the variations into the mutation of RFGs. The diversity of the faces of descendants can thus be significantly increased. Qualitative, quantitative, and subjective experiments on FIW, TSKinFace, and FF-Databases clearly show that the quality and diversity of kinship faces generated by our approach are much better than the existing state-of-the-art methods.

关键词：

Distilled Transformers with Locally Enhanced Global Representations for Face Forgery Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhang, Yaning Li, Qiufu Yu, Zitong Shen, Linlin Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University Guangdong Shenzhen518060 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Guangdong Shenzhen518060 China Shenzhen Institute of Artificial Intelligence and Robotics for Society Guangdong Shenzhen518129 China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University Guangdong Shenzhen518060 China School of Computing and Information Technology Great Bay University Guangdong Dongguan523000 China

Face forgery detection (FFD) is devoted to detecting the authenticity of face images. Although current CNN-based works achieve outstanding performance in FFD, they are susceptible to capturing local forgery patterns generated by various manipulation methods. Though transformer-based detectors exhibit improvements in modeling global dependencies, they are not good at exploring local forgery artifacts. Hybrid transformer-based networks are designed to capture local and global manipulated traces, but they tend to suffer from the attention collapse issue as the transformer block goes deeper. Besides, soft lab.ls are rarely availab.e. In this paper, we propose a distilled transformer network (DTN) to capture both rich local and global forgery traces and learn general and common representations for different forgery faces. Specifically, we design a mixture of expert (MoE) module to mine various robust forgery embeddings. Moreover, a locally-enhanced vision transformer (LEVT) module is proposed to learn locally-enhanced global representations. We design a lightweight multi-attention scaling (MAS) module to avoid attention collapse, which can be plugged and played in any transformer-based models with only a slight increase in computational costs. In addition, we propose a deepfake self-distillation (DSD) scheme to provide the model with abundant soft lab.l information. Extensive experiments show that the proposed method surpasses the state of the arts on five deepfake datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Face recognition

Taming Stable Diffusion for MRI Cross-Modality Translation

学校读者我要写书评

暂无评论

Taming Stable Diffusion for MRI Cross-Modality Translation

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Yingtai Li Shuo Yang Xiaoyan Wu Shan He S. Kevin Zhou School of Biomedical Engineering Division of Life Sciences and Medicine USTC Hefei P.R. China Center for Medical Imaging Robotics Analytic Computing & Learning (MIRACLE) Suzhou Institute for Advanced Research USTC P.R. China iFLYTEK Research iFLYTEK Co. Ltd Hefei P.R. China Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing P.R. China

ISBN: (数字)9798350386226

ISBN: (纸本)9798350386233

In this study, we explore using Stable Diffusion (SD) for unsupervised medical image-to-image translation. SD has shown remarkable performances in generating high-quality images and can be easily applied to generate custom contents by injecting standard plug-ins like LoRA, offering a promising solution to tackle the complexity caused by variations in imaging modalities, acquisition parameters, and body parts in medical imaging. However, We empirically find that existing pipelines designed for natural images fail to translate directly to medical images due to weak structural control and inappropriate color preservation. To address these issues, we propose a novel two-branch image translation pipeline. This pipeline decouples the generation of target image along the time axis and employs ControlNet to ensure precise structural preservation. Additionally, we customize SD to generate images of extreme brightness, a common feature in medical imaging. Our results on the BraTS dataset demonstrate that SD with task-specific plug-ins can generate high-quality medical images comparable to those generated by task-specific models. Since the development of these standard plug-ins can be easily done by clinicians without much knowledge of the underlying algorithm, such a mode holds the potential to significantly extend the use of medical image computing algorithms in the clinical environment.

关键词： Translation Image color analysis Computational modeling Biological system modeling Magnetic resonance imaging Pipelines Brightness Complexity theory Standards Biomedical imaging

Blockchain-Based Homomorphic Transaction Framework for Enhanced Consumer Security and Business Scalab.lity

学校读者我要写书评

暂无评论

IEEE Transactions on Consumer Electronics 2024年

作者： Guo, Tao Shang, Fengjun Dai, Xiangguang Liu, Qilie Chongqing University of Posts and Telecommunications School of Computer Science and Technology Chongqing400065 China Key Laboratory of Big Data Intelligent Computing Chongqing400065 China Chongqing Three Gorges University Key Lab. of Intelligent Info. Proc. and Contr. of Chongqing Munic. Institutions of Higher Education Chongqing Engineering Research Center of Internet of Things and Intelligent Control Technology School of Three Gorges Artificial Intelligence Chongqing400044 China Chongqing University of Posts and Telecommunications School of Communications and Information Engineering Chongqing400065 China

Energy trading in distributed microgrids represents an effective means of enhancing the utilization of renewable energy. However, the aggregation of large-scale consumption data may encounter business scalab.lity issues and potential privacy exposure concerns, necessitating the development of novel secure and scalab.e frameworks for integrating consumption demands between prosumer and microgrid entities. we research focuses on providing a user-friendly transaction model for consumers and microgrid operators. Firstly, a privacy aggregation scheme for user resource requests is designed to ensure the security and decentralization of user data. In addition, the model employs a bidirectional heterogeneous resource allocation mechanism to support the optimal allocation of multiple resources, such as renewable energy, power assets, energy storage devices, and carbon credits, among different microgrids. Finally, a microgrid transaction prototype has been constructed on the Truffle development framework. In this context, smart contracts has been designed to ensure that the transactions are carried out automatically. Experimental results indicate that the scheme provides enhanced security for multi-resource trading in a cluster context relative to existing methodologies. The number of transactions increases linearly with the resource overhead, resulting in more efficient utilization of energy trading. Furthermore, the framework's scalab.lity has been demonstrated through both temporal and spatial considerations. © 1975-2011 IEEE.

关键词： Decentralized finance

MOTION MATTERS: A NOVEL MOTION MODELING FOR CROSS-VIEW GAIT FEATURE LEARNING

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Jingqi Gao, Jiaqi Zhang, Yuzhen Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

As a unique biometric that can be perceived at a distance, gait has broad applications in person authentication, social security and so on. Existing gait recognition methods suffer from changes in viewpoint and clothing and barely consider extracting diverse motion features, a fundamental characteristic in gaits, from gait sequences. This paper proposes a novel motion modeling method to extract the discriminative and robust representation. Specifically, we first extract the motion features from the encoded motion sequences in the shallow layer. Then we continuously enhance the motion feature in deep layers. This motion modeling approach is independent of mainstream work in building network architectures. As a result, one can apply this motion modeling method to any backbone to improve gait recognition performance. In this paper, we combine motion modeling with one commonly used backbone (GaitGL) as GaitGL-M to illustrate motion modeling. Extensive experimental results on two commonly-used cross-view gait datasets demonstrate the superior performance of GaitGL-M over existing state-of-the-art methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Network architecture

DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Xiaofei Chen, Wenting Liu, Jie Lu, Qisheng Luo, Xiaoling Shen, Linlin Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University Shenzhen China Department of Electrical Engineering City University of Hong Kong Kowloon Hong Kong National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China

Medical report generation is crucial for clinical diagnosis and patient management, summarizing diagnoses and recommendations based on medical imaging. However, existing work often overlook the clinical pipeline involved in report writing, where physicians typically conduct an initial quick review followed by a detailed examination. Moreover, current alignment methods may lead to misaligned relationships. To address these issues, we propose DAMPER, a dual-stage framework for medical report generation that mimics the clinical pipeline of report writing in two stages. In the first stage, a MeSH-Guided Coarse-Grained Alignment (MCG) stage that aligns chest X-ray (CXR) image features with medical subject headings (MeSH) features to generate a rough keyphrase representation of the overall impression. In the second stage, a Hypergraph-Enhanced Fine-Grained Alignment (HFG) stage that constructs hypergraphs for image patches and report annotations, modeling high-order relationships within each modality and performing hypergraph matching to capture semantic correlations between image regions and textual phrases. Finally,the coarse-grained visual features, generated MeSH representations, and visual hypergraph features are fed into a report decoder to produce the final medical report. Extensive experiments on public datasets demonstrate the effectiveness of DAMPER in generating comprehensive and accurate medical reports, outperforming state-of-the-art methods across various evaluation metrics. Copyright © 2024, The Authors. All rights reserved.

关键词： Coarse-grained modeling

Rephrasing the Reference for Non-Autoregressive Machine Translation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Shao, Chenze Zhang, Jinchao Zhou, Jie Feng, Yang Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Pattern Recognition Center WeChat AI Tencent Inc China

Non-autoregressive neural machine translation (NAT) models suffer from the multi-modality problem that there may exist multiple possible translations of a source sentence, so the reference sentence may be inappropriate for the training when the NAT output is closer to other translations. In response to this problem, we introduce a rephraser to provide a better training target for NAT by rephrasing the reference sentence according to the NAT output. As we train NAT based on the rephraser output rather than the reference sentence, the rephraser output should fit well with the NAT output and not deviate too far from the reference, which can be quantified as reward functions and optimized by reinforcement learning. Experiments on major WMT benchmarks and NAT baselines show that our approach consistently improves the translation quality of NAT. Specifically, our best variant achieves comparable performance to the autoregressive Transformer, while being 14.7 times more efficient in inference. Copyright © 2022, The Authors. All rights reserved.

关键词： Neural machine translation