检索结果-内蒙古大学图书馆

32nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI)

作者： Wang, Menglan Yu, Yue Li, Benyuan Beijing Inst Technol Sch Comp Sci & Technol Beijing Peoples R China

ISBN: (纸本)9781728192284

Learning joint embedding between image and text is significant for text-to-image synthesis as it bridges the semantic gap between image and text. Most existing text-to-image generation methods depend on the quality of the text embedding. If the text features are not be extracted well, it is difficult for subsequent processes to generate satisfactory images. However, these methods are disturbed by the text expression form in the process of extracting text features, resulting in the ideal text features cannot be generated well. In this paper, we propose a new text encoder that learns joint embedding to capture semantic information shared by the real images and the input text, and eliminates the interference of textual expression forms. The main difference with existing works is that for different texts describing the same image, although their expressions are different, because they contain the same semantic information, the proposed text encoder extracts the similar semantic features from these different texts. Meanwhile, a special auxiliary classifier for discriminator is adopted to retain low-level features to generate fine-detailed images. We evaluate this work on the Caltech-UCSD Birds 200 (CUB) and the Oxford-102 flower dataset, experiments show that our work has better performance than the state-of-the-art works.

关键词： joint embedding special auxiliary classifier text-to-image synthesis

来源：评论

学校读者我要写书评

暂无评论

Unsupervised text-to-image synthesis

引用

PATTERN RECOGNITION 2021年 110卷 107573-107573页

作者： Dong, Yanlong Zhang, Ying Ma, Lin Wang, Zhi Luo, Jiebo Tsinghua Univ Beijing Peoples R China Tencent AI Lab Shenzhen Guangdong Peoples R China Meituan Dianping Grp Beijing Peoples R China Tsinghua Shenzhen Int Grad Sch Beijing Peoples R China Peng Cheng Lab Shenzhen Guangdong Peoples R China Univ Rochester Rochester NY 14627 USA

Recently, text-to-image synthesis has achieved great progresses with the advancement of the Genera-tive Adversarial Network (GAN). However, training the GAN models requires a large amount of pairwise image-text data, which is extremely labor-intensive to collect. In this paper, we make the first attempt to train a text-to-image synthesis model in an unsupervised manner, which does not require any human labeled image-text pair data. Specifically, we first rely on the visual concepts to bridge two independent image and sentence sets and thereby yield the pseudo image-text pair data, based on which one GAN model can thereby be initialized. One novel visual concept discrimination loss is proposed to train both generator and discriminator, which not only encourages the image expressing the true local visual concepts but also ensures the noisy visual concepts contained in the pseudo sentence being suppressed. Afterwards, one global semantic consistency regarding to the real sentence is used to adapt the pretrained GAN model to real sentences. Experimental results demonstrate that our proposed unsupervised training strategy is able to generate favorable images for given sentences, which even outperforms some existing models trained in the supervised manner. The code of this paper is available at https://***/dylls/Unsupervised_text-to-image_synthesis. (c) 2020 Elsevier Ltd. All rights reserved.

关键词： text-to-image synthesis Generative adversarial network (GAN) Unsupervised training

来源：评论

学校读者我要写书评

暂无评论

Class-Balanced text to image synthesis With Attentive Generative Adversarial Network

引用

IEEE MULTIMEDIA 2021年第3期28卷 21-31页

作者： Wang, Min Lang, Congyan Liang, Liqian Lyu, Gengyu Feng, Songhe Wang, Tao Beijing Jiaotong Univ Sch Comp & Informat Technol Beijing 100044 Peoples R China

Although the text-to-image synthesis task has shown significant progress, it still remains a challenge in generating high-quality images. In this article, we first propose an attention-driven, cycle-refinement generative adversarial network, AGAN-v1, to bridge the domain gap between visual contents and semantic concepts by constructing spatial configurations of objects. The generation of image contours is the core component, in which an attention mechanism is developed to refine local details of images by focusing on the objects that complement one subregion. Second, an advanced class-balanced generative adversarial network, AGAN-v2, is proposed to address the problem of long-tailed data distribution. Importantly, it is the first method to solve this problem in the text-to-image synthesis task. Our AGAN-v2 introduces a reweighting scheme, which adopts the effective number of samples for each class to rebalance the generative loss. Extensive quantitative and qualitative experiments on CUB and MS-COCO datasets demonstrate that the proposed AGAN-v2 significantly outperforms the state-of-the-art methods.

关键词： Generative adversarial networks Gallium nitride Training Task analysis Head Bridges image synthesis generative adversarial network text-to-image synthesis attention-driven long-tailed data distribution rebalance

来源：评论

学校读者我要写书评

暂无评论

Visual Thinking of Neural Networks: Interactive text to image synthesis

引用

IEEE ACCESS 2021年 9卷 64510-64523页

作者： Lee, Hyunhee Kim, Gyeongmin Hur, Yuna Lim, Heuiseok Korea Univ Dept Comp Sci & Engn Seoul 02841 South Korea

Reasoning, a trait of cognitive intelligence, is regarded as a crucial ability that distinguishes humans from other species. However, neural networks now pose a challenge to this human ability. text-to-image synthesis is a class of vision and linguistics, wherein the goal is to learn multimodal representations between the image and text features. Hence, it requires a high-level reasoning ability that understands the relationships between objects in the given text and generates high-quality images based on the understanding. text-to-image translation can be termed as the visual thinking of neural networks. In this study, our model infers the complicated relationships between objects in the given text and generates the final image by leveraging the previous history. We define diverse novel adversarial loss functions and finally demonstrate the best one that elevates the reasoning ability of the text-to-image synthesis. Remarkably, most of our models possess their own reasoning ability. Quantitative and qualitative comparisons with several methods demonstrate the superiority of our approach.

关键词： Cognition Visualization Neural networks Generative adversarial networks image synthesis image registration text recognition Generative adversarial networks image generation multimodal learning multimodal representation text-to-image synthesis

来源：评论

学校读者我要写书评

暂无评论

End-to-End text-to-image synthesis with Spatial Constrains

引用

ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY 2020年第4期11卷 47-47页

作者： Wang, Min Lang, Congyan Liang, Liqian Feng, Songhe Wang, Tao Gao, Yutong Beijing Jiaotong Univ Beijing Peoples R China

Although the performance of automatically generating high-resolution realistic images from text descriptions has been significantly boosted, many challenging issues in image synthesis have not been fully investigated, due to shapes variations, viewpoint changes, pose changes, and the relations of multiple objects. In this article, we propose a novel end-to-end approach for text-to-image synthesis with spatial constraints by mining object spatial location and shape information. Instead of learning a hierarchical mapping from text to image, our algorithm directly generates multi-object fine-grained images through the guidance of the generated semantic layouts. By fusing text semantic and spatial information into a synthesis module and jointly fine-tuning them with multi-scale semantic layouts generated, the proposed networks show impressive performance in text-to-image synthesis for complex scenes. We evaluate our method both on single-object CUB dataset and multiobject MS-COCO dataset. Comprehensive experimental results demonstrate that our method significantly outperforms the state-of-the-art approaches consistently across different evaluation metrics.

关键词： Computer vision text-to-image synthesis spatial constrain MS-COCO CUB

来源：评论

学校读者我要写书评

暂无评论

Cross-modal Feature Alignment based Hybrid Attentional Generative Adversarial Networks for text-to-image synthesis

引用

DIGITAL SIGNAL PROCESSING 2020年 107卷 102866-102866页

作者： Cheng, Qingrong Gu, Xiaodong Fudan Univ Dept Elect Engn Shanghai 200433 Peoples R China

With the development of the generative model, image synthesis has become a research hotspot. This paper presents a novel Cross-modal Feature Alignment based Hybrid Attentional Generative Adversarial Networks (CFA-HAGAN) for text-to-image synthesis. It mainly consists of two steps, text-image encoding and text-to-image synthesis. text-image encoding learns a Cross-modal Feature Alignment Model (CFAM), which adopts a fine-grained attentional network to learn the original multi-modalities' aligned features. The feature alignment space is viewed as the transitional space in the whole process. Then, the Hybrid Attentional Generative Adversarial Networks (HAGAN) learns the inverse mapping from the encoded text feature to the original image. Specifically, the hybrid attention block consists of text-image cross-modal attention mechanism and self-attention mechanism of an image. Cross-modal attention makes the synthesized image fine-grained by adding word-level information as additional supervision. Self-attention can solve the long-distance reliance problem of image sub-region features when synthesizes images from the hidden feature. Although excellent performance in an ocean of tasks, GANs are well-known for the difficulty of training. Adopting spectral normalization, the discriminators are satisfied with 1-Lipschitz constraint, which makes their training process more stable than original GANs. During quantitative and non-quantitative comparison with many state-of-the-art methods, the experimental results show that the proposed method achieves better performance on evaluation metric and visual effect. Besides, the experimental section presents attention visualization, ablation study, and generalization ability analysis to show the effectiveness of the proposed method. (C) 2020 Elsevier Inc. All rights reserved.

关键词： text-to-image synthesis Spectral normalization Self-attention mechanism Cross-modal attention Generative Adversarial Networks

来源：评论

学校读者我要写书评

暂无评论

DRAWGAN: text TO image synthesis WITH DRAWING GENERATIVE ADVERSARIAL NETWORKS

DRAWGAN: TEXT TO IMAGE SYNTHESIS WITH DRAWING GENERATIVE ADV...

引用

IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

作者： Zhang, Zhiqiang Zhou, Jinjia Yu, Wenxin Jiang, Ning Hosei Univ Grad Sch Sci & Engn Tokyo Japan Southwest Univ Sci & Technol Sch Comp Sci & Technol Mianyang Sichuan Peoples R China

ISBN: (纸本)9781728176055

In this paper, we propose a novel drawing generative adversarial networks (DrawGAN) for text-to-image synthesis. The whole model divides the image synthesis into three stages by imitating the process of drawing. The first stage synthesizes the simple contour image based on the text description, the second stage generates the foreground image with detailed information, and the third stage synthesizes the final result. Through the step by step synthesis process from simple to complex and easy to difficult, the model can draw the corresponding results step by step and finally achieve the higher-quality image synthesis effect. Our method is validated on the Caltech-UCSD Birds 200 (CUB) dataset and the Microsoft Common Objects in Context (MS COCO) dataset. The experimental results demonstrate the effectiveness and superiority of our method. In terms of both subjective and objective evaluation, our method's results surpass the existing state-of-the-art methods.

关键词： text-to-image synthesis Computer Vision Deep Learning Generative Adversarial Networks

来源：评论

学校读者我要写书评

暂无评论

FA-GAN: FEATURE-AWARE GAN FOR text TO image synthesis

FA-GAN: FEATURE-AWARE GAN FOR TEXT TO IMAGE SYNTHESIS

引用

IEEE International Conference on image Processing (ICIP)

作者： Jeon, Eunyeong Kim, Kunhee Kim, Daijin Pohang Univ Sci & Technol Dept Comp Sci & Engn Pohang South Korea

ISBN: (纸本)9781665441155

text-to-image synthesis aims to generate a photo-realistic image from a given natural language description. Previous works have made significant progress with Generative Adversarial Networks (GANs). Nonetheless, it is still hard to generate intact objects or clear textures (Fig 1). To address this issue, we propose Feature-Aware Generative Adversarial Network (FA-GAN) to synthesize a high-quality image by integrating two techniques: a self-supervised discriminator and a feature-aware loss. First, we design a self-supervised discriminator with an auxiliary decoder so that the discriminator can extract better representation. Secondly, we introduce a feature-aware loss to provide the generator more direct supervision by employing the feature representation from the self-supervised discriminator. Experiments on the MS-COCO dataset show that our proposed method significantly advances the state-of-the-art FID score from 28.92 to 24.58.

关键词： text-to-image synthesis Generative Adversarial Networks Feature-Aware GAN

来源：评论

学校读者我要写书评

暂无评论

text TO image synthesis WITH ERUDITE GENERATIVE ADVERSARIAL NETWORKS

TEXT TO IMAGE SYNTHESIS WITH ERUDITE GENERATIVE ADVERSARIAL ...

引用

IEEE International Conference on image Processing (ICIP)

作者： Zhang, Zhiqiang Yu, Wenxin Jiang, Ning Zhou, Jinjia Hosei Univ Tokyo Japan Southwest Univ Sci & Technol Mianyang Sichuan Peoples R China

ISBN: (纸本)9781665441155

In this paper, an Erudite Generative Adversarial Networks (EruditeGAN) is proposed for the text-to-image synthesis task. By introducing additional image distribution related to the original image into the network structure, the entire network can learn more about the image distribution and become more knowledgeable. In this case, it can be more clear about the distribution of the image that needs to be synthesized and finally synthesize high-quality results. Experiments well validate our method's effectiveness and demonstrate the different effects of different distribution situations on the final results. According to the quantitative results of Frechet Inception Distance (FID) and R-precision, our method's comprehensive score is the best, which reflects our results are closer to the real image effect.

关键词： text-to-image synthesis Computer Vision Deep Learning Generative Adversarial Networks

来源：评论

学校读者我要写书评

暂无评论

Semantically consistent text to fashion image synthesis with an enhanced attentional generative adversarial network ?

引用

PATTERN RECOGNITION LETTERS 2020年 135卷 22-29页

作者： Ak, Kenan E. Lim, Joo Hwee Tham, Jo Yew Kassim, Ashraf A. Natl Univ Singapore Elect & Comp Engn 4 Engn Dr 3Block E4 Singapore 117583 Singapore ASTAR Inst Infocomm Res I2R 1 Fusionopolis Way Singapore 138632 Singapore ESP xMedia Pte Ltd 75 Ayer Rajah Crescent02-15 Singapore 139953 Singapore

Recent advancements in Generative Adversarial Networks (GANs) have led to significant improvements in various image generation tasks including image synthesis based on text descriptions. In this paper, we present an enhanced Attentional Generative Adversarial Network (e-AttnGAN) with improved training stability for text-to-image synthesis. e-AttnGAN’s integrated attention module utilizes both sentence and word context features and performs feature-wise linear modulation (FiLM) to fuse visual and natural language representations. In addition to multimodal similarity learning for text and image features of AttnGAN [1], similarity and feature matching losses between real and generated images are included while employing classification losses for “significant attributes”. In order to improve the stability of the training and solve the mode collapse issue, spectral normalization and two-time scale update rule are used for the discriminator together with instance noise. Our experiments show that e-AttnGAN outperforms state-of-the-art methods using the FashionGen and DeepFashion-synthesis datasets in terms of inception score, R-precision and classification accuracy. A detailed ablation study has been conducted to observe the effect of each component.

关键词： Generative adversarial networks text-to-image synthesis Hierarchical image generation image generation Fashion

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：