检索结果-内蒙古大学图书馆

NEURAL PROCESSING LETTERS 2022年第5期54卷 4609-4646页

作者： Elasri, Mohamed Elharrouss, Omar Al-Maadeed, Somaya Tairi, Hamid Sidi Mohamed Ben Abdellah Univ Dept Comp Sci Fes Morocco Qatar Univ Dept Comp Sci & Engn Doha Qatar

The creation of an image from another and from different types of data including text, scene graph, and object layout, is one of the very challenging tasks in computer vision. In addition, capturing images from different views for generating an object or a product can be exhaustive and expansive to do manually. Now, using deep learning and artificial intelligence techniques, the generation of new images from different type of data has become possible. For that, a significant effort has been devoted recently to develop image generation strategies with a great achievement. To that end, we present in this paper, to the best of the authors' knowledge, the first comprehensive overview of existing image generation methods. Accordingly, a description of each image generation technique is performed based on the nature of the adopted algorithms, type of data used, and main objective. Moreover, each image generation category is discussed by presenting the proposed approaches. In addition, a presentation of existing image generation datasets is given. The evaluation metrics that are suitable for each image generation category are discussed and a comparison of the performance of existing solutions is provided to better inform the state-of-the-art and identify their limitations and strengths. Lastly, the current challenges that are facing this subject are presented.

关键词： image generation text-to-image generation Sketch-to-image generation Layout-to-image generation image-to-image translation Panoramic image generation

来源：评论

学校读者我要写书评

暂无评论

GOYA: Leveraging Generative Art for Content-Style Disentanglement

引用

JOURNAL OF IMAGING 2024年第7期10卷 156-156页

作者： Wu, Yankun Nakashima, Yuta Garcia, Noa Osaka Univ Intelligence & Sensing Lab Suita Osaka 5650871 Japan

The content-style duality is a fundamental element in art. These two dimensions can be easily differentiated by humans: content refers to the objects and concepts in an artwork, and style to the way it looks. Yet, we have not found a way to fully capture this duality with visual representations. While style transfer captures the visual appearance of a single artwork, it fails to generalize to larger sets. Similarly, supervised classification-based methods are impractical since the perception of style lies on a spectrum and not on categorical labels. We thus present GOYA, which captures the artistic knowledge of a cutting-edge generative model for disentangling content and style in art. Experiments show that GOYA explicitly learns to represent the two artistic dimensions (content and style) of the original artistic image, paving the way for leveraging generative models in art analysis.

关键词： art analysis representation disentanglement text-to-image generation

来源：评论

学校读者我要写书评

暂无评论

FHS-adapter: fine-grained hierarchical semantic adapter for Chinese landscape paintings generation

引用

HERITAGE SCIENCE 2024年第1期12卷 265页

作者： Peng, Xianlin Hu, Qiyao Fan, Fangqing Xie, Penglin Zhang, Yihan Cao, Rui Northwest Univ Sch Art Xian 710127 Peoples R China Northwest Univ Sch Informat Sci & Technol Xian 710127 Peoples R China Northwest Univ State Prov Joint Engn & Res Ctr Adv Networking & I Xian 710127 Peoples R China Shaanxi Key Lab Higher Educ Inst Generat Artificia Xian 710127 Peoples R China Northwest Univ Shaanxi Silk Rd Cultural Heritage Digital Protect Xian 710127 Peoples R China Northwest Univ Network & Data Ctr Xian 710127 Peoples R China

How to migrate text-to-image models based on pre-trained diffusion models to adapt them to domain generation tasks is a common problem. In particular, the generation task for Chinese landscape paintings with unique characteristics suffers from a scarcity of fine-grained contextual details specific to such artwork. Moreover, the use of substantial amounts of non-landscape painting data during pre-training predisposes the model to be swayed by alternative visual styles, thereby leading to generated images that inadvertently lack the distinctive traits inherent to Chinese paintings. In this paper, we propose a Fine-grained Hierarchical Semantic Adapter for Chinese landscape paintings generation, namely FHS-adapter. The method orchestrates the diffusion process in a batch-wise manner, leveraging external fine-grained multi-perspective information to guide it. It gradually diminishes the influence of other style images embedded in the pre-trained diffusion model, ultimately preserving a greater number of landscape painting elements. The encoder was also replaced with the Taiyi-CLIP encoder, which is adapted for Chinese. We propose T2ICLP, a multimodal dataset containing 10,000 high-quality image-text pairs of Chinese landscape paintings. Unlike previous datasets, this dataset extracts fine-grained textual information from four perspectives, including Meta, Description, Sentiment, Poem. We compared the proposed model with the mainstream diffusion-based T2I models. Through an anonymous user study, our FHS-adapter method performs well in simulating various aspects such as brushwork, e.g.'Gou, Cun, Dian, Ran' means hooking, texturing, dotting, and dyeing, compositional space, elemental proportions, and color usage of different painting genres and artists. Our dataset is available at https://***/T2ICLP/t2iclp.

关键词： Traditional Chinese landscape paintings Fine-grained text-to-image generation

来源：评论

学校读者我要写书评

暂无评论

Towards Geographic Inclusion in the Evaluation of text-to-image Models 24

Towards Geographic Inclusion in the Evaluation of Text-to-Im...

引用

6th ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)

作者： Hall, Melissa Bell, Samuel J. Ross, Candace Williams, Adina Drozdzal, Michal Soriano, Adriana Romero Meta AI Seattle WA 98164 USA Meta AI Paris France Meta AI Toronto ON Canada McGill Univ Mila Meta AI Montreal PQ Canada CIFAR AI Chair Toronto ON Canada

ISBN: (纸本)9798400704505

Rapid progress in text-to-image generative models coupled with their deployment for visual content creation has magnified the importance of thoroughly evaluating their performance and identifying potential biases. In pursuit of models that generate images that are realistic, diverse, visually appealing, and consistent with the given prompt, researchers and practitioners often turn to automated metrics to facilitate scalable and cost-effective performance profiling. However, commonly-used metrics often fail to account for the full diversity of human preference;often even in-depth human evaluations face challenges with subjectivity, especially as interpretations of evaluation criteria vary across regions and cultures. In this work, we conduct a large, cross-cultural study to study how much annotators in Africa, Europe, and Southeast Asia vary in their perception of geographic representation, visual appeal, and consistency in real and generated images from state-of-the art public APIs. We collect over 65, 000 image annotations and 20 survey responses. We contrast human annotations with common automated metrics, finding that human preferences vary notably across geographic location and that current metrics do not fully account for this diversity. For example, annotators in different locations often disagree on whether exaggerated, stereotypical depictions of a region are considered geographically representative. In addition, the utility of automatic evaluations is dependent on assumptions about their set-up, such as the alignment of feature extractors with human perception of object similarity or the definition of "appeal" captured in reference datasets used to ground evaluations. We recommend steps for improved automatic and human evaluations. This includes collecting annotations from people located inside and outside the region of interest, instructing annotators on whether they should follow specific definitions of evaluation criteria or utilize their own interpretat

关键词： text-to-image generation geography evaluation

来源：评论

学校读者我要写书评

暂无评论

CAN GENERATIVE AI MODELS COUNT? Finetuning Stable Diffusion for Architecture image generation with Designated Floor Numbers Using a Small Dataset 29th

CAN GENERATIVE AI MODELS COUNT? Finetuning Stable Diffusion ...

引用

29th International Conference of the Association-for-Computer-Aided-Architectural-Design-Research-in-Asia (CAADRIA)

作者： Xu, Weishun Li, Mingming Yang, Xuyou Hangzhou Coll Engn & Architecture Hangzhou Peoples R China Zhejiang Univ Coll Comp Sci & Technol Hangzhou Peoples R China 1 DesignLab Hangzhou Peoples R China

ISBN: (纸本)9789887891819

Despite the increasing popularity of off-the-shelf text-to-image generative artificial intelligence models in early-stage architectural design practices, general-purpose models are challenged in domain-specific tasks such as generating buildings with the correct number of floors. We hypothesise that this problem is mainly caused by the lack of floor number information in standard training sets. To overcome the often-dodged problem in creating a text-image pair dataset large enough for finetuning the original model in design research, we propose to use BLIP method for both understanding and generation based automated labelling and captioning with online images. A small dataset of 25,172 text-image pairs created with this method is used to finetune an off-the-shelf Stable Diffusion model for 10 epochs with affordable computing power. Compared to the base model with a less than 20% chance to generate the correct number of floors, the finetuned model has an over 50% overall chance for correct floor number and 87.3% change to control the floor count discrepancy within 1 storey.

关键词： text-to-image generation Model Finetuning Stable Diffusion Automated Labelling

来源：评论

学校读者我要写书评

暂无评论

Examining the text-to-image Community of Practice: Why and How do People Prompt Generative AIs? 23

Examining the Text-to-Image Community of Practice: Why and H...

引用

15th Conference on Creativity and Cognition (C and C)

作者： Sanchez, Teo Selas Studio Paris France

ISBN: (纸本)9798400701801

image generation gained popularity with machine learning (ML) models generating images from text, fuelling new online communities of practices. This work explores the sociology, motivations, and usages of AI art hobbyists. We analyzed an online questionnaire answered by 64 practitioners and a dataset of user prompts sent to the Stable Diffusion generative model. Our findings suggest that TTI generation is a recreational activity mainly conducted by narrow socio-demographic groups who use auxiliary techniques across platforms and beyond request-response interactions. Inherent model limitations and finding suitable prompt formulation are the main obstacles practitioners face. A taxonomy and a corresponding ML model capable of recognizing the semantic content of unseen prompts were created to conduct the user prompt analysis. The prompt analysis revealed that artist names are the main specifier used beside the main subject, often in sequences. We finally discuss the design and socio-technical implications of our work for creativity support.

关键词： text-to-image generation community of practice

来源：评论

学校读者我要写书评

暂无评论

ENTWINED INVERSION: TUNE-FREE INVERSION FOR REAL image FAITHFUL RECONSTRUCTION AND EDITING 49

ENTWINED INVERSION: TUNE-FREE INVERSION FOR REAL IMAGE FAITH...

引用

49th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Huang, Jiancheng Liu, Yifan Lv, Jiaxi Chen, Shifeng Chinese Acad Sci Shenzhen Inst Adv Technol Beijing Peoples R China Univ Chinese Acad Sci Beijing Peoples R China Southern Univ Sci & Technol Shenzhen Peoples R China

ISBN: (纸本)9798350344868;9798350344851

text-conditional image editing is a very practical AIGC task that has recently emerged with great commercial and academic research value. For real image editing, most diffusion model-based methods use DDIM Inversion as the first stage before editing, but DDIM Inversion often results in reconstruction failure, leading to unsatisfactory performance for all downstream edits. In order to solve this problem, we first mathematically analyze the reason for the reconstruction failure of DDIM Inversion, and then propose a new inversion and sampling method named Entwined Inversion that can achieve satisfactory reconstruction and editing performance, which can solve two major problems: 1) the object can retain the main content of the original image;2) the edited object can conform to the semantics of the text prompt. In addition, our method does not require training the diffusion model itself on a large dataset, nor does it require any fine-tuning for some particular images.

关键词： Real image editing Diffusion model text-to-image generation AIGC

来源：评论

学校读者我要写书评

暂无评论

SBCR: Stochasticity Beats Content Restriction Problem in Training and Tuning Free image Editing 24

SBCR: Stochasticity Beats Content Restriction Problem in Tra...

引用

4th Annual International Conference on Multimedia Retrieval (ICMR)

作者： Huang, Jiancheng Yan, Mingfu Liu, Yifan Chen, Shifeng Chinese Acad Sci Shenzhen Inst Adv Technol Shenzhen Peoples R China Southern Univ Sci & Technol Shenzhen Peoples R China

ISBN: (纸本)9798400706028

text-conditional image editing is a practical AIGC task that has recently emerged with great commercial and academic value. For real image editing, most diffusion model-based methods use DDIM Inversion as a first stage before editing. However, DDIM Inversion often results in reconstruction failure, leading to unsatisfactory performance for downstream editing. Many inversion-based works modify the formula to address this problem but this leads to another content restriction problem. To solve the content restriction problem, we first analyze why the reconstruction via DDIM Inversion fails and then propose Reconstruction-and-generation Balancing Noises (R&G-B noises) that can achieve superior reconstruction and editing performance with the following advantages: 1) It can perfectly reconstruct real images without fine-tuning. 2) It can overcome the content restriction problem and generate diverse content.

关键词： Real image editing Diffusion model text-to-image generation AIGC Generative model

来源：评论

学校读者我要写书评

暂无评论

Predicated Diffusion: Predicate Logic-Based Attention Guidance for text-to-image Diffusion Models

Predicated Diffusion: Predicate Logic-Based Attention Guidan...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Sueyoshi, Kota Matsubara, Takashi Osaka Univ 1-3 Machikaneyama Toyonaka Osaka 5608531 Japan

ISBN: (纸本)9798350353013;9798350353006

Diffusion models have achieved remarkable success in generating high-quality, diverse, and creative images. However, in text- based image generation, they often struggle to accurately capture the intended meaning of the text. For instance, a specified object might not be generated, or an adjective might incorrectly alter unintended objects. Moreover, we found that relationships indicating possession between objects are frequently overlooked. Despite the diversity of users' intentions in text, existing methods often focus on only some aspects of these intentions. In this paper, we propose Predicated Diffusion, a unified framework designed to more effectively express users' intentions. It represents the intended meaning as propositions using predicate logic and treats the pixels in attention maps as fuzzy predicates. This approach provides a differentiable loss function that offers guidance for the image generation process to better fulfill the propositions. Comparative evaluations with existing methods demonstrated that Predicated Diffusion excels in generating images faithful to various text prompts, while maintaining high image quality, as validated by human evaluators and pretrained image-text models.

关键词： attention guidance diffusion models predicate logic text-to-image generation

来源：评论

学校读者我要写书评

暂无评论

GenTron: Diffusion Transformers for image and Video generation

GenTron: Diffusion Transformers for Image and Video Generati...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Chen, Shoufa Xu, Mengmeng Ren, Jiawei Cong, Yuren He, Sen Xie, Yanping Sinha, Animesh Luo, Ping Xiang, Tao Perez-Rua, Juan-Manuel Univ Hong Kong Hong Kong Peoples R China Meta Menlo Pk CA USA

ISBN: (纸本)9798350353013;9798350353006

In this study, we explore Transformer-based diffusion models for image and video generation. Despite the dominance of Transformer architectures in various fields due to their flexibility and scalability, the visual generative domain primarily utilizes CNN-based U-Net architectures, particularly in diffusion-based models. We introduce GenTron, a family of Generative models employing Transformer-based diffusion, to address this gap. Our initial step was to adapt Diffusion Transformers (DiTs) from class to text conditioning, a process involving thorough empirical exploration of the conditioning mechanism. We then scale GenTron from approximately 900M to over 3B parameters, observing improvements in visual quality. Furthermore, we extend GenTron to text-to-video generation, incorporating novel motion-free guidance to enhance video quality. In human evaluations against SDXL, GenTron achieves a 51.1% win rate in visual quality (with a 19.8% draw rate), and a 42.3% win rate in text alignment (with a 42.9% draw rate). GenTron notably performs well in T2I-CompBench, highlighting its compositional generation ability. We hope GenTron could provide meaningful insights and serve as a valuable reference for future research. Please refer to the website 1 1 https://***/gentron_website/ and the arXiv version for the most up-to-date results: https://***/abs/2312.04557.

关键词： Diffusion Transformers text-to-image generation text-to-Video generation

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：