检索结果-内蒙古大学图书馆

8th International Conference on Arabic Language Processing

作者： Bahani, Mourad El Ouaazizi, Aziza Maalmi, Khalil Essahlaoui, Abdelouahed Sidi Mohamed Ben Abdellah Univ Natl Sch Appl Sci Artificial Intelligence Data Sci & Emerging Syst Fes Morocco Sidi Mohamed Ben Abdellah Univ Lab Engn Sci FPT Taza Morocco

ISBN: (纸本)9783031804373;9783031804380

text-to-image generation is a cutting-edge technology that enables computers to generate images from textual descriptions. While this technology has been extensively researched and applied to English language text, applying it to Arabic language text is still in its early stages. Additionally, the Arabic language is challenging due to its right-to-left writing system and extensive vocabulary of 1.3 million words. In this paper, we explore text-to-image generation for generating images from Arabic language text descriptions. Firstly, we fine-tune a transformer-based model pre-trained on the Arabic text to transform the text information into affine transformation within the DF-GAN generator. Secondly, we present a text transformer that combines LSTM layers to address the limitation of unrecognized words. Thirdly, a mask predictor is trained into the generator using a weakly supervised method and incorporated into the affine transformation for a more effective integration of image and text features. In addition, we add the DAMSM loss function as a regularization to the loss function to achieve convergences and stability in the training phase. The experiment on two challenging datasets CUB and Oxford-flower shows that our architectures can accurately generate high-quality images faithfully representing the Arabic textual descriptions. We believe the scaling of this task could have critical applications in fields such as Arabic visual learning, e-commerce, advertising, and entertainment.

关键词： Machine Learning Deep Learning Computer Vision Generative Adversarial Networks text-to-image generation Arabic text Processing

来源：评论

学校读者我要写书评

暂无评论

Customization of the text-to-image diffusion model by fine-tuning for the generation of synthetic images of cyanobacterial blooms in lentic water bodies

引用

Expert Systems with Applications 2025年 287卷

作者： Fredy Barrientos-Espillco Gonzalo Pajares José A. López-Orozco Eva Besada-Portas Department of Computer Architecture and Automation University Complutense of Madrid 28040 Madrid Spain Institute for Knowledge Technology University Complutense of Madrid 28040 Madrid Spain

Cyanobacterial blooms emerge unpredictably on the surface of lentic water bodies, posing both ecological threats and public health risks. To effectively monitor these events, this study introduces the use of Machine Vision Systems (MVS) integrated into Autonomous Surface Vehicles (ASVs). These ASVs are capable of autonomous and safe navigation, enabling them to detect cyanobacterial blooms while avoiding obstacles. Convolutional Neural Networks (CNNs) are employed for early detection and continuous monitoring, but their effectiveness hinges on access to large, high-quality training datasets. Due to the sporadic and uncontrollable nature of bloom occurrences, acquiring sufficient real-world images for training and validating CNN models is a significant challenge. To overcome this, the Stable Diffusion XL (SDXL) text-to-image generative model is utilized to produce realistic synthetic images, ensuring a sufficient dataset for training. However, SDXL alone struggles to accurately depict cyanobacterial blooms. To address this limitation, DreamBooth is used to fine-tune SDXL with a small set of real bloom-specific image patches. To ensure the diversity of the synthetic dataset, detailed prompts for SDXL are generated using a Large Language Model (LLM). The combination of SDXL fine-tuning with LLM-driven prompts design applied to environmental monitoring and autonomous navigation in lentic environments represents the core innovation of this work. A dual-task CNN model is then trained on the synthetic dataset to simultaneously detect blooms and obstacles. Experimental results demonstrate the effectiveness and novelty of the proposed approach, showing improvements of up to 15.74% in object detection and 6.48% in semantic segmentation compared to the baseline dataset.

关键词： Artificial intelligence Machine vision systems text-to-image generation Fine-tuning Large language model Convolutional neural networks Autonomous surface vehicles Cyanobacterial blooms

来源：评论

学校读者我要写书评

暂无评论

Txt2Img-MHN: Remote Sensing image generation From text Using Modern Hopfield Networks

引用

IEEE TRANSACTIONS ON image PROCESSING 2023年 32卷 5737-5750页

作者： Xu, Yonghao Yu, Weikang Ghamisi, Pedram Kopp, Michael Hochreiter, Sepp Inst Adv Res Artificial Intelligence IARAI A-1030 Vienna Austria Linkoping Univ Dept Elect Engn Comp Vis Lab S-58183 Linkoping Sweden Helmholtz Inst Freiberg Resource Technol Helmholtz Zentrum Dresden Rossendorf Machine Learning Grp D-09599 Freiberg Germany Johannes Kepler Univ Linz Inst Machine Learning ELLIS Unit Linz LIT AI Lab A-4040 Linz Austria

The synthesis of high-resolution remote sensing images based on text descriptions has great potential in many practical application scenarios. Although deep neural networks have achieved great success in many important remote sensing tasks, generating realistic remote sensing images from text descriptions is still very difficult. To address this challenge, we propose a novel text-to-image modern Hopfield network (Txt2Img-MHN). The main idea of Txt2Img-MHN is to conduct hierarchical prototype learning on both text and image embeddings with modern Hopfield layers. Instead of directly learning concrete but highly diverse text-image joint feature representations for different semantics, Txt2Img-MHN aims to learn the most representative prototypes from text-image embeddings, achieving a coarse-to-fine learning strategy. These learned prototypes can then be utilized to represent more complex semantics in the text-to-image generation task. To better evaluate the realism and semantic consistency of the generated images, we further conduct zero-shot classification on real remote sensing data using the classification model trained on synthesized images. Despite its simplicity, we find that the overall accuracy in the zero-shot classification may serve as a good metric to evaluate the ability to generate an image from text. Extensive experiments on the benchmark remote sensing text-image dataset demonstrate that the proposed Txt2Img-MHN can generate more realistic remote sensing images than existing methods. Code and pre-trained models are available online (https://***/YonghaoXu/Txt2Img-MHN).

关键词： Deep learning image synthesis modern Hopfield networks remote sensing text-to-image generation zero-shot classification

来源：评论

学校读者我要写书评

暂无评论

Interpolating the text-to-image Correspondence Based on Phonetic and Phonological Similarities for Nonword-to-image generation

引用

IEEE ACCESS 2024年 12卷 41299-41316页

作者： Matsuhira, Chihaya Kastner, Marc A. Komamizu, Takahiro Hirayama, Takatsugu Doman, Keisuke Kawanishi, Yasutomo Ide, Ichiro Nagoya Univ Grad Sch Informat Nagoya Aichi 4648601 Japan Kyoto Univ Grad Sch Informat Kyoto 6068501 Japan Nagoya Univ Math & Data Sci Ctr Nagoya Aichi 4648601 Japan Univ Human Environm Fac Environm Sci Okazaki Aichi 4443505 Japan Chukyo Univ Sch Engn Toyota Aichi 4700393 Japan RIKEN Guardian Robot Project Seika Kyoto 6190288 Japan

text-to-image (T2I) generation is the task of synthesizing images corresponding to a given text input. The recent innovations in artificial intelligence have enhanced the capacity of conventional T2I generation, yielding more and more powerful models day by day. However, their behavior is known to become unstable in the face of text inputs containing nonwords that have no definition within a language. This behavior not only results in situations where image generation does not match human expectations but also hinders these models from being utilized in psycholinguistic applications and simulations. This paper exploits the human nature of associating nonwords with their phonetically and phonologically similar words and uses it to propose a T2I generation framework robust against nonword inputs. The framework comprises a phonetics-aware language model as well as an adjusted T2I generation model. Our evaluations confirm that the proposed nonword-to-image generation synthesizes images that depict visual concepts of phonetically similar words more stably than comparative methods. We also assess how the image generation results match human expectations, showing a better agreement than the phonetics-blind baseline.

关键词： Nonwords phonetics pronunciation psycholinguistics text-to-image generation vision and language

来源：评论

学校读者我要写书评

暂无评论

Cross-modal text and visual generation: A systematic review. Part 1: image to text

引用

INFORMATION FUSION 2023年第1期93卷 302-329页

作者： Zelaszczyk, Maciej Mandziuk, Jacek Warsaw Univ Technol Plac Politech 1 PL-00661 Warsaw Poland

We review the existing literature on generating text from visual data under the cross-modal generation umbrella, which affords us to compare and contrast various approaches taking visual data as input and producing text outputs, while not limiting the analysis to narrowly-defined areas, such as image captioning. We provide a breakdown of image-to-text generation methods into generative and non-generative image captioning and visual dialogue, with further distinctions provided for relevant areas. We provide template methods and discuss the existing research in light of such template methods, highlighting both the salient commonalities between different approach and significant departures from the templates. Where it is of interest, we also provide comparisons between templates for distinct areas. To achieve a comprehensive review, we focus on research papers published at 8 leading machine learning conferences in the years 2016-2021 as well as a number of papers, which do not conform to our search criteria, but nonetheless come from leading venues. This is the first review we know of to provide a systematic description of the current state of image-to-text generation and tie distinct research areas together looking at them through the lens of cross-modal generation.

关键词： image-to-text generation text-to-image generation image captioning Cross-modal learning Multimodal learning Representation learning

来源：评论

学校读者我要写书评

暂无评论

ControlStyle: text-Driven Stylized image generation Using Diffusion Priors 23

ControlStyle: Text-Driven Stylized Image Generation Using Di...

引用

31st ACM International Conference on Multimedia (MM)

作者： Chen, Jingwen Pan, Yingwei Yao, Ting Mei, Tao Sun Yat Sen Univ Guangzhou Peoples R China Univ Sci & Technol China Hefei Peoples R China HiDream Ai Inc Beijing Peoples R China

ISBN: (纸本)9798400701085

Recently, the multimedia community has witnessed the rise of diffusion models trained on large-scale multi-modal data for visual content creation, particularly in the field of text-to-image generation. In this paper, we propose a new task for "stylizing" text-to-image models, namely text-driven stylized image generation, that further enhances editability in content creation. Given input text prompt and style image, this task aims to produce stylized images which are both semantically relevant to input text prompt and meanwhile aligned with the style image in style. To achieve this, we present a new diffusion model (ControlStyle) via upgrading a pre-trained text-to-image model with a trainable modulation network enabling more conditions of text prompts and style images. Moreover, diffusion style and content regularizations are simultaneously introduced to facilitate the learning of this modulation network with these diffusion priors, pursuing high-quality stylized text-to-image generation. Extensive experiments demonstrate the effectiveness of our ControlStyle in producing more visually pleasing and artistic results, surpassing a simple combination of text-to-image model and conventional style transfer techniques.

关键词： diffusion models text-to-image generation style transfer

来源：评论

学校读者我要写书评

暂无评论

Leveraging the Syntactic Structure of the text Prompt to Enhance Object-Attribute Binding in image generation 2

Leveraging the Syntactic Structure of the Text Prompt to Enh...

引用

2nd Workshop on Large Generative Models Meet Multimodal Applications (LGM3A)

作者： Trusca, Maria Mihaela Nuyts, Wolf Thomm, Jonathan Honig, Robert Hofmann, Thomas Tuytelaars, Tinne Moens, Marie-Francine Katholieke Univ Leuven Dept Comp Sci Leuven Belgium Swiss Fed Inst Technol Dept Comp Sci Zurich Switzerland Katholieke Univ Leuven Dept Elect Engn Leuven Belgium

ISBN: (纸本)9798400711930

Current diffusion models can generate photorealistic images from text prompts but often struggle to correctly associate the attributes mentioned in the text with the appropriate objects in the image. To address this issue, we propose focused cross-attention (FCA), which controls visual attention maps using syntactic constraints from the input sentence. Additionally, the syntactic structure of the prompt aids in disentangling the multimodal CLIP embeddings commonly used in text-to-image (T2I) generation. The resulting DisCLIP embeddings and FCA can be easily integrated into state-of-the-art diffusion models without requiring additional training. We demonstrate significant improvements in T2I generation, particularly in the accurate binding of attributes to objects, across multiple datasets.

关键词： Focused Cross-Attention Disentangle CLIP text-to-image generation Diffusion Models

来源：评论

学校读者我要写书评

暂无评论

Chest-Diffusion: A Light-Weight text-to-image Model for Report-to-CXR generation 21

Chest-Diffusion: A Light-Weight Text-to-Image Model for Repo...

引用

21st IEEE International Symposium on Biomedical Imaging (ISBI)

作者： Huang, Peng Gao, Xue Huang, Lihong Jiao, Jing Li, Xiaokang Wang, Yuanyuan Guo, Yi Fudan Univ Dept Elect Engn Shanghai 200433 Peoples R China Key Lab Med Imaging Comp & Comp Assisted Interven Shanghai 200032 Peoples R China

ISBN: (纸本)9798350313345;9798350313338

text-to-image generation has important implications for generation of diverse and controllable images. Several attempts have been made to adapt Stable Diffusion (SD) to the medical domain. However, the large distribution difference between medical reports and natural texts, as well as high computational complexity in common stable diffusion limit the authenticity and feasibility of the generated medical images. To solve above problems, we propose a novel light-weight transformer-based diffusion model learning framework, Chest-Diffusion, for report-to-CXR generation. Chest-Diffusion employs a domainspecific text encoder to obtain accurate and expressive text features to guide image generation, improving the authenticity of the generated images. Meanwhile, we introduce a light-weight transformer architecture as the denoising model, reducing the computational complexity of the diffusion model. Experiments demonstrate that our Chest-Diffusion achieves the lowest FID score 24.456, under the computation budget of 118.918 GFLOPs, which is nearly one-third of the computational complexity of SD.

关键词： Chest X-ray Diffusion model text-to-image generation CLIP Vision Transformer

来源：评论

学校读者我要写书评

暂无评论

text-based Sequential image generation 14

Text-based Sequential Image Generation

引用

14th International Conference on Machine Vision (ICMV)

作者： Efimova, Valeria Filchenkov, Andrey ITMO Univ St Petersburg Russia Statanly Technol St Petersburg Russia

ISBN: (数字)9781510650459

ISBN: (纸本)9781510650459;9781510650442

Despite recent impressive results of generative adversarial networks on text-to-image generation, the generation of complex scenes with multiple objects in the complicated background remains challenging;moreover, end-to-end text-to-image generation still suffers from poor image quality. In this work, we propose a sequential algorithm of text-to-image generation, which allows synthesizing high-quality images (more than 1024x1024 pixels). The proposed approach consists of location inference, key objects extraction, image search, layout generation, and image harmonization stages. We compare the suggested approach with state-of-the-art image generation model DALL-E with text-to-image mapping. Our approach demonstrates the effectiveness and visual plausibility of the generated images based on golden section layouts.

关键词： text-to-image generation transformer layout generation

来源：评论

学校读者我要写书评

暂无评论

"Journey of Finding the Best Query": Understanding the User Experience of AI image generation System

引用

INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION 2025年第2期41卷 951-969页

作者： Kim, Soomin Eun, Jinsu Oh, Changhoon Lee, Joonhwan Seoul Natl Univ Seoul South Korea Yonsei Univ Seoul South Korea

With the advancement of AI, even people without professional experience can create artworks using AI-based image generation systems like DALL-E 2. However, little is known about how users interact with these new AI algorithms, much less how AI-infused systems can be designed. We explore the user experience of these new technologies and their potential to foster creativity. A user study was carried out where 13 participants executed tasks of creating artworks using DALL-E 2 alongside in-depth interviews related to their experience. The results showed that users had ambivalent opinions regarding the algorithm's performance. When users were informed of the system's capabilities, they subsequently utilized more specific prompts to generate the intended output. Users also optimized their prompts (the queries they entered to create artworks) based on how algorithms worked to achieve their desired outcome. The users wanted a two-way interaction where AI explained the outcome and accepted feedback rather than simply accepting unilateral instructions. We discuss the implications for designing interfaces that maximize creativity while providing comfort for the users.

关键词： Generative AI human-AI interaction text-to-image generation prompt engineering

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：