检索结果-内蒙古大学图书馆

text-to-image generation Method Based on Object Enhancement and Attention Maps

INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS 2025年第1期16卷 961-968页

作者： Huang, Yongsen Cai, Xiaodong An, Yuefan Guilin Univ Elect Technol Sch Informat & Commun Guilin Peoples R China

In the task of text-to-image generation, common issues such as missing objects in the generated images often arise due to the model's insufficient learning of multi-object category information and the lack of consistency between the text prompts and the generated image contents. To address these challenges, this paper proposes a novel text-to-image generation approach based on object enhancement and attention maps. First, a new object enhancement strategy is introduced to improve the model's capacity to capture object-level features. The core idea is to generate difficult samples by processing the object mask maps of tokens, followed by dynamic weighting of the attention map using latent image embeddings. Second, to enhance the consistency between the text prompts and the generated image contents, we enforce similarity constraints between the cross-attention maps and the attention-weighted mask feature maps, penalizing inconsistencies through a loss function. Experimental results demonstrate that the Stable Diffusion v1.4 model, optimized using the proposed method, achieves significant improvements on the COCO instance dataset and the ADE20K instance dataset. Specifically, the MG metrics are improved by an average of 12.36% and 6.55%, respectively, compared to state-of-the-art models. Furthermore, the FID metrics show a 0.84% improvement over the state-of-the-art model on the COCO instance validation set.

关键词： Multi-object category text-to-image generation object enhancement attention maps

来源：评论

学校读者我要写书评

暂无评论

High-Quality text-to-image generation Using High-Detail Feature-Preserving Network

引用

APPLIED SCIENCES-BASEL 2025年第2期15卷 706-706页

作者： Hsu, Wei-Yen Lin, Jing-Wen Natl Chung Cheng Univ Dept Informat Management Chiayi 62102 Taiwan Natl Chung Cheng Univ Adv Inst Mfg High Tech Innovat Chiayi 62102 Taiwan Natl Chung Cheng Univ Ctr Innovat Res Aging Society CIRAS Chiayi 62102 Taiwan

Multistage text-to-image generation algorithms have shown remarkable success. However, the images produced often lack detail and suffer from feature loss. This is because these methods mainly focus on extracting features from images and text, using only conventional residual blocks for post-extraction feature processing. This results in the loss of features, greatly reducing the quality of the generated images and necessitating more resources for feature calculation, which will severely limit the use and application of optical devices such as cameras and smartphones. To address these issues, the novel High-Detail Feature-Preserving Network (HDFpNet) is proposed to effectively generate high-quality, near-realistic images from text descriptions. The initial text-to-image generation (iT2IG) module is used to generate initial feature maps to avoid feature loss. Next, the fast excitation-and-squeeze feature extraction (FESFE) module is proposed to recursively generate high-detail and feature-preserving images with lower computational costs through three steps: channel excitation (CE), fast feature extraction (FFE), and channel squeeze (CS). Finally, the channel attention (CA) mechanism further enriches the feature details. Compared with the state of the art, experimental results obtained on the CUB-Bird and MS-COCO datasets demonstrate that the proposed HDFpNet achieves better performance and visual presentation, especially regarding high-detail images and feature preservation.

关键词： generative adversarial network text-to-image generation high detail feature preservation

来源：评论

学校读者我要写书评

暂无评论

UniCanvas: Affordance-Aware Unified Real image Editing via Customized text-to-image generation

引用

INTERNATIONAL JOURNAL OF COMPUTER VISION 2025年第6期133卷 3456-3480页

作者： Jin, Jian Shen, Yang Zhao, Xinyang Fu, Zhenyong Yang, Jian Nanjing Univ Sci & Technol Sch Comp Sci & Engn Key Lab Intelligent Percept & Syst High Dimens Inf PCA LabMinist Educ Nanjing Peoples R China

The demand for assorted conditional edits on a single real image is becoming increasingly prevalent. We focus on two dominant editing tasks that respectively condition on image and text input, namely subject-driven editing and semantic editing. Previous studies typically tackle these two editing tasks separately, thereby demanding multiple editing processes to achieve versatile edits on a single image. However, fragmented and sequential editing processes not only require more user effort but also further degrade the editing quality. In this paper, we propose UniCanvas, an affordance-aware unified framework that can achieve high-quality parallel subject-driven and semantic editing on a single real image within one inference process. UniCanvas innovatively unifies the multimodal inputs of the editing task into the textual condition space using tailored customization strategies. Building upon the unified representations, we propose a novel inference pipeline that performs parallel editing by selectively blending and manipulating two collaborative text-to-image generative branches. Customization enables the editing process to harness the strong visual understanding and reasoning capability of pre-trained generative models for affordance perception, and a unified inference space further facilitates more effective affordance interaction and alignment for compelling editing. Extensive experiments on diverse real images demonstrate that UniCanvas exhibits powerful scene affordance perception in unified image editing, achieving seamless subject-driven editing and precise semantic editing for various target subjects and query prompts (https://***/unicanvas/).

关键词： Real image editing Pre-trained model customization text-to-image generation Diffusion model Affordance perception

来源：评论

学校读者我要写书评

暂无评论

PMGAN: pretrained model-based generative adversarial network for text-to-image generation

引用

VISUAL COMPUTER 2025年第1期41卷 303-314页

作者： Yu, Yue Yang, Yue Xing, Jingshuo Beijing Inst Technol Sch Comp Sci & Technol 5 Zhongguancun South St Beijing 100081 Peoples R China

text-to-image generation is a challenging task. Although diffusion models can generate high-quality images of complex scenes, they sometimes suffer from a lack of realism. Additionally, there is often a large diversity among images generated from different text with the same semantics. Furthermore, the generation of details is sometimes insufficient. Generative adversarial networks can generate realism images. These images are consistent with the text descriptions. And the networks can generate content-consistent images. In this paper, we argue that generating images that are more consistent with the text descriptions is more important than generating higher-quality images. Therefore, this paper proposes the pretrained model-based generative adversarial network (PMGAN). PMGAN utilizes multiple pre-trained models in both generator and discriminator. Specifically, in the generator, the deep attentional multimodal similarity model text encoder extracts word and sentence embeddings from the input text, and the contrastive language-image pre-training (CLIP) text encoder extracts initial image features from the input text. In the discriminator, a pre-trained CLIP image encoder extracts image features from the input image. The CLIP encoder can map text and images into a common semantic space, which is beneficial to generate high-quality images. Experimental results show that compared to the state-of-the-art methods, PMGAN achieves better scores on both inception score and Frechet inception distance and can produce higher quality images while maintaining greater consistency with text descriptions.

关键词： Pretrained model Generative adversarial network text-to-image generation Feature extraction

来源：评论

学校读者我要写书评

暂无评论

Exploring text-to-image generation models: Applications and cloud resource utilization

引用

COMPUTERS & ELECTRICAL ENGINEERING 2025年 123卷

作者： Jaiprakash, Sahani Pooja Prakash, Shyam Bennett Univ Sch Comp Sci Engn & Technol Greater Noida Uttar Pradesh India ITER Siksha O Anusandhan Dept CSE Bhubaneswar Odisha India Natl Inst Technol Dept CSE Tadepalligudem Andhra Pradesh India

Generating images from text presents a significant challenge in computer vision. Moreover, manually acquiring images from multiple perspectives for object or product generation is a resource-intensive and expensive endeavor. However, recent breakthroughs in deep learning and artificial intelligence have opened doors to creating new images from diverse data sources, and cloud resources play a pivotal role in alleviating the resource-intensive nature of this endeavor. As a result, substantial research efforts have been directed toward advancing image generation techniques, yielding impressive results. This paper aims to provide a comprehensive overview of existing image generation methods, offering insights into this evolving field of text-to-image generation. It traces the historical development of this technology. It examines the key models that have shaped its evolution, including Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), Conditional GANs (CGANs), StackGAN, Transformers, and diffusion models. The paper offers insights into the functioning of text-to-image generation within the GAN architecture, elucidating the mechanisms behind transforming textual descriptions into visual content. Additionally, the integration of text-to-image generation with cloud and edge-cloud computing highlights the synergistic potential of these technologies while addressing the challenges and considerations associated with cloud infrastructure. The paper concludes by surveying the diverse applications of text-to-image generation across various domains, such as art, ecommerce, entertainment, and education. It also discusses the evaluation metrics commonly used in assessing the quality of generated images and the challenges that exist both within the methods and in their application across different domains. This review offers a comprehensive overview of the capabilities and limitations of text-to-image generation. Also, we have introduced a new HiResGAN model u

关键词： VAE GAN StackGAN Diffusion model Cloud resouce utilization Edge-cloud computing text-to-image generation

来源：评论

学校读者我要写书评

暂无评论

Skin-Adapter: Fine-Grained Skin-Color Preservation for text-to-image generation 31st

Skin-Adapter: Fine-Grained Skin-Color Preservation for Text-...

引用

31st International Conference on Multimedia Modeling

作者： Chen, Zhuowei Huang, Mengqi Chen, Nan Mao, Zhendong Univ Sci & Technol China Hefei Peoples R China

ISBN: (纸本)9789819620708;9789819620715

With the recent advancements in diffusion-based text-to-image (T2I) models, generating high-quality, human-centric images has become increasingly easy. However, T2I models cannot preserve the fine-grained skin color information from reference images, raising potential AI ethics concerns. In this work, we propose Skin-Adapter, the first work on preserving fine-grained skin color information in text-to-image generation. To achieve this goal, we first devise the frequency-based adaptive color histogram to accurately represent the user's skin color information. Additionally, we introduce the color distribution matching reward to explicitly enhance skin color consistency between input and output images. Experimental results show that our Skin-Adapter can maintain the fine-grained skin color information of input images in the generated images. Furthermore, we validate the superiority of our approach through quantitative and qualitative comparisons against possible alternatives.

关键词： text-to-image generation Controllable image generation AI Ethics

来源：评论

学校读者我要写书评

暂无评论

MirrorDiff: Prompt redescription for zero-shot grounded text-to-image generation with attention modulation

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2025年 153卷

作者： Liu, Chang Shao, Mingwen Gong, Zhengyi Lv, Xiang Meng, Lingzhuang China Univ Petr East China Qingdao Inst Software Coll Comp Sci & Technol State Key Lab Chem Safety Qingdao 266580 Peoples R China

Large-scale layout-conditioned text-to-image diffusion models have made significant progress and achieved remarkable results in generating diverse and high-quality images, realizing objects appearing in specific regions simultaneously. However, existing methods still fail with attribute coupling, unreasonable spatial relationships expressions and missing objects when the prompt is complex with multiple objects containing multiple attributes. In addition, it is difficult for users to give precise layout conditions for complex prompts. To address the above issues, we propose MirrorDiff, a novel training-free grounded text-to-image-to-text framework by redescription to correct inaccurate content expressions of synthetic images iteratively. Specifically, we first utilize large language models as layout generator which have the ability to understand visual concepts and support plausible arrangements to generate scene layout for complex prompts to help users obtain precision layout more conveniently. Subsequently, to solve small object missing, we design a layout-guided attention modulation strategy to properly adjust attention maps during diffusion generation process, which effectively increases attention of small objects. Additionally, semantic text regeneration supervision is proposed to constrain the redescription to keep consistent with the given text semantically, which aims to mitigate attribute coupling and failures of spatial relationships expressions. We conduct extensive experiments on four benchmarks and our method achieves the best results in all categories on the Holistic, Reliable and Scalable benchmark, which shows that our proposed MirrorDiff achieves state-of-the-art results both quantitatively and qualitatively compared with current superior models.

关键词： text-to-image generation Large language model Diffusion model Attention modulation

来源：评论

学校读者我要写书评

暂无评论

text-to-image generation via Semi-Supervised Training

Text-to-Image Generation via Semi-Supervised Training

引用

IEEE International Conference on Visual Communications and image Processing (VCIP)

作者： Ji, Zhongyi Wang, Wenmin Chen, Baoyang Han, Xiao Peking Univ Sch Elect & Comp Engn Shenzhen Grad Sch Shenzhen Peoples R China Macau Univ Sci & Technol Intemat Inst Next Generat Internet Macau Peoples R China

ISBN: (纸本)9781728180687

Synthesizing images from text is an important problem and has various applications. Most of the existing studies of text-to-image generation utilize supervised methods and rely on a fully-labeled dataset, but detailed and accurate descriptions of images are onerous to obtain. In this paper, we introduce a simple but effective semi-supervised approach that considers the feature of unlabeled images as " Pseudo text Feature". Therefore, the unlabeled data can participate in the following training process. To achieve this, we design a Modality-invariant Semanticconsistent Module which aims to make the image feature and the text feature indistinguishable and maintain their semantic information. Extensive qualitative and quantitative experiments on MNIST and Oxford-102 flower datasets demonstrate the effectiveness of our semi-supervised method in comparison to supervised ones. We also show that the proposed method can be easily plugged into other visual generation models such as image translation and performs well.

关键词： image Synthesis text-to-image generation Semi-Supervised Learning Pseudo Feature

来源：评论

学校读者我要写书评

暂无评论

The effectiveness of T5, GPT-2, and BERT on text-to-image generation task

引用

PATTERN RECOGNITION LETTERS 2023年第1期173卷 57-63页

作者： Bahani, Mourad El Ouaazizi, Aziza Maalmi, Khalil Sidi Mohamed Ben Abdellah Univ Artificial Intelligence Data Sci & Emerging Syst L Fes Morocco

text-to-image (T2I) generation, which involves synthesizing an image from a textual description, has emerged as a popular research topic in computer vision. Meanwhile, transformer-based models, such as BERT, GPT-2, and T5, have demonstrated promising results in various natural language processing tasks, including text generation and translation. However, their application to T2I generation remains largely unexplored. Therefore, a comparative study investigating the performance of BERT, GPT-2, and T5 in T2I generation is of significant importance. Such a study would shed light on the strengths and weaknesses of each model, facilitating the identification of the most suitable approach for this task. In this paper, we propose three architectures to conduct a comparative study of T5, GPT-2, and BERT in T2I generation tasks. We fine-tune these models to generate text vectors and then transform the textual information into images using the affine transformation into the DF-GAN generator. Subsequently, we evaluate the quality, diversity, and the models' ability to recognize the words. Our experiments on the challenging CUB and Oxford-102 flower datasets demonstrate that T5 exhibits promising potential for T2I generation. It has the capability to generate visually appealing and semantically coherent images from textual descriptions.

关键词： Deep learning Computer vision Generative adversarial networks text-to-image generation Natural language processing Transformer-based models

来源：评论

学校读者我要写书评

暂无评论

Robustness of Generative Adversarial CLIPs Against Single-Character Adversarial Attacks in text-to-image generation

引用

IEEE ACCESS 2024年 12卷 162551-162563页

作者： Chanakya, Patibandla Harsha, Putla Pratap Singh, Krishna Indian Inst Informat Technol Dept Informat Technol Prayagraj 211012 India

Generative Adversarial Networks (GANs) have emerged as a powerful type of generative model, particularly effective at creating images from textual descriptions. Similar to diffusion models, GANs rely on text encoders to extract embeddings from these descriptions. However, this reliance introduces specific vulnerabilities to adversarial attacks. A notable example is a single-character adversarial attack, where altering a single character in the text description can lead to significant performance degradation in the generated image quality and model's performance. In this study, we systematically evaluate the susceptibility of GANs to such attacks using Generative Adversarial CLIP (GALIP), a single-stage architecture that leverages a pre-trained Contrastive Language-image Pre-training (CLIP) text encoder for text embeddings. We meticulously selected captions with single-character modifications that exhibit maximum and median-distance embeddings for the attack. Experimental results show up to 310.5% degradation in Fr & eacute;chet Inception Distance (FID) scores, underscoring the importance of developing improved defenses in text-to-image synthesis.

关键词： text to image Robustness Perturbation methods Degradation Diffusion models image synthesis image quality Training Generators Generative adversarial networks Single-character attack GALIP GAN CLIP text encoder text-to-image generation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：