检索结果-内蒙古大学图书馆

Challenges in Generating Accurate text in images: A Benchmark for text-to-image models on Specialized Content

APPLIED SCIENCES-BASEL 2025年第5期15卷 2274-2274页

作者： Bosheah, Zenab Bilicki, Vilmos Univ Szeged Inst Informat Dept Software Engn H-6720 Szeged Hungary

Rapid advances in text-to-image (T2I) generative models have significantly enhanced visual content creation. However, evaluating these models remains challenging, particularly when assessing their ability to handle complex textual content. The primary aim of this research is to develop a systematic evaluation framework for assessing T2I models' capabilities in generating specialized content, with emphasis on measuring text rendering accuracy and identifying model limitations across diverse domains. The framework utilizes carefully crafted prompts that require precise formatting, semantic alignment, and compositional reasoning to evaluate model performance. Our evaluation methodology encompasses a comprehensive assessment across many critical domains: mathematical equations, chemical diagrams, programming code, flowcharts, multi-line text, and paragraphs, with each domain tested through specifically designed challenge sets. GPT-4 serves as an automated evaluator, assessing outputs based on key metrics such as text accuracy, readability, formatting consistency, visual design, contextual relevance, and error recovery. Weighted scores generated by GPT-4 are compared with human evaluations to measure alignment and reliability. The results reveal that current T2I models face significant challenges with tasks requiring structural precision and domain-specific accuracy. Notable difficulties include symbol alignment in equations, bond angles in chemical diagrams, syntactical correctness in code, and the generation of coherent multi-line text and paragraphs. This study advances our understanding of fundamental limitations in T2I model architectures while establishing a novel framework for the systematic evaluation of text rendering capabilities. Despite these limitations, the proposed benchmark provides a clear pathway for evaluating and tracking improvements in T2I models, establishing a standardized framework for assessing their ability to generate accurate and reliable str

关键词： text-to-image models visual text rendering human-AI alignment complex textual content

来源：评论

学校读者我要写书评

暂无评论

Ethical-Lens: Curbing malicious usages of open-source text-to-image models

引用

PATTERNS 2025年第3期6卷 101187页

作者： Cai, Yuzhu Yin, Sheng Wei, Yuxi Xu, Chenxin Mao, Weibo Juefei-Xu, Felix Chen, Siheng Wang, Yanfeng Shanghai Jiao Tong Univ Sch Artificial Intelligence Shanghai 200241 Peoples R China Beihang Univ Sch Comp Sci & Engn Beijing 100191 Peoples R China NYU New York NY 10012 USA Shanghai AI Lab Shanghai 201210 Peoples R China

The burgeoning landscape of text-to-image models, exemplified by innovations such as Midjourney and DALLE 3, has revolutionized content creation across diverse sectors. However, these advances bring forth critical ethical concerns, particularly with the misuse of open-source models to generate content that violates societal norms. Addressing this, we introduce Ethical-Lens, a framework designed to facilitate the value-aligned usage of text-to-image tools without necessitating internal model revision. Ethical- Lens ensures value alignment in text-to-image models across toxicity and bias dimensions by refining user commands and rectifying model outputs. Systematic evaluation metrics, combining GPT4-V, HEIM, and FairFace scores, assess alignment capability. Our experiments reveal that Ethical-Lens enhances alignment capabilities to levels comparable with or superior to commercial models such as DALLE 3, while preserving the quality of generated images. This study indicates the potential of Ethical-Lens to promote the sustainable development of open-source text-to-image tools and their beneficial integration into society.

关键词： value alignment text-to-image models large language models AI safety

来源：评论

学校读者我要写书评

暂无评论

EvilPromptFuzzer: generating inappropriate content based on text-to-image models

引用

CYBERSECURITY 2024年第1期7卷 70页

作者： He, Juntao Dai, Haoran Sui, Runqi Yuan, Xuejing Liu, Dun Feng, Hao Liu, Xinyue Yang, Wenchuan Cui, Baojiang Li, Kedan Beijing Univ Posts & Telecommun Sch Cyberspace Secur 10 Xitucheng Rd Beijing 100876 Peoples R China Revery AI Inc 60 Hazelwood Dr 126 Champaign IL 61820 USA

text-to-image (TTI) models provide huge innovation ability for many industries, while the content security triggered by them has also attracted wide attention. Considerable research has focused on content security threats of large language models (LLMs), yet comprehensive studies on the content security of TTI models are notably scarce. This paper introduces a systematic tool, named EvilPromptFuzzer, designed to fuzz evil prompts in TTI models. For 15 kinds of fine-grained risks, EvilPromptFuzzer employs the strong knowledge-mining ability of LLMs to construct seed banks, in which the seeds cover various types of characters, interrelations, actions, objects, expressions, body parts, locations, surroundings, etc. Subsequently, these seeds are fed into the LLMs to build scene-diverse prompts, which can weaken the semantic sensitivity related to the fine-grained risks. Hence, the prompts can bypass the content audit mechanism of the TTI model, and ultimately help to generate images with inappropriate content. For the risks of violence, horrible, disgusting, animal cruelty, religious bias, political symbol, and extremism, the efficiency of EvilPromptFuzzer for generating inappropriate images based on DALL.E 3 are greater than 30%, namely, more than 30 generated images are malicious among 100 prompts. Specifically, the efficiency of horrible, disgusting, political symbols, and extremism up to 58%, 64%, 71%, and 50%, respectively. Additionally, we analyzed the vulnerability of existing popular content audit platforms, including Amazon, Google, Azure, and Baidu. Even the most effective Google SafeSearch cloud platform identifies only 33.85% of malicious images across three distinct categories.

关键词： Risks of AI-generated content Inappropriate content text-to-image models

来源：评论

学校读者我要写书评

暂无评论

SurrogatePrompt: Bypassing the Safety Filter of text-to-image models via Substitution 24

SurrogatePrompt: Bypassing the Safety Filter of Text-to-Imag...

引用

31st Conference on Computer and Communications Security

作者： Ba, Zhongjie Zhong, Jieming Lei, Jiachen Cheng, Peng Wang, Qinglong Qin, Zhan Wang, Zhibo Ren, Kui Zhejiang Univ State Key Lab Blockchain & Data Secur Hangzhou Peoples R China

ISBN: (纸本)9798400706363

Advanced text-to-image models such as DALL center dot E 2, Midjourney, and Stable Diffusion can generate highly realistic images, raising significant concerns regarding the potential proliferation of unsafe content. This includes adult, violent, or deceptive imagery of political figures. Despite claims of rigorous safety mechanisms implemented in these models to restrict the generation of Not-SafeFor-Work (NSFW) content, we successfully devise and exhibit the first prompt attacks on Midjourney, producing abundant photorealistic NSFW images. We reveal the fundamental principles of such prompt attacks and strategically substitute high-risk sections within a suspect prompt to evade closed-source safety measures. Our novel framework, SurrogatePrompt, systematically generates attack prompts, utilizing large language models and image-to-text modules to automate attack prompt creation at scale. Evaluation results disclose an 88% success rate in bypassing Midjourney's proprietary safety filter with our attack prompts, leading to counterfeit images depicting political figures in violent scenarios with high probability. We also demonstrate attacks generating explicit adult-themed imagery. Both subjective and objective assessments validate that the images generated from our attack prompts present considerable safety hazards.

关键词： Adversarial Examples Prompt Engineering Safety Control text-to-image models Large Language models

来源：评论

学校读者我要写书评

暂无评论

Efficient Exploration of image Classifier Failures with Bayesian Optimization and text-to-image models

Efficient Exploration of Image Classifier Failures with Baye...

引用

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Le Coz, Adrien Ouertatani, Houssem Herbin, Stephane Adjed, Faouzi IRT SystemX F-91120 Palaiseau France Univ Paris Saclay DTIS ONERA F-91123 Palaiseau France INRIA Lille France

ISBN: (纸本)9798350365474

image classifiers should be used with caution in the real world. Performance evaluated on a validation set may not reflect performance in the real world. In particular, classifiers may perform well for conditions that are frequently encountered during training, but poorly for other infrequent conditions. In this study, we hypothesize that recent advances in text-to-image generative models make them valuable for benchmarking computer vision models such as image classifiers: they can generate images conditioned by textual prompts that cause classifier failures, allowing failure conditions to be described with textual attributes. However, their generation cost becomes an issue when a large number of synthetic images need to be generated, which is the case when many different attribute combinations need to be tested. We propose an image classifier benchmarking method as an iterative process that alternates image generation, classifier evaluation, and attribute selection. This method efficiently explores the attributes that ultimately lead to poor behavior detection.

关键词： bayesian optimization image classifier failures text-to-image models

来源：评论

学校读者我要写书评

暂无评论

AI's Regimes of Representation: A Community-centered Study of text-to-image models in South Asia 23

AI's Regimes of Representation: A Community-centered Study o...

引用

6th ACM Conference on Fairness, Accountability, and Transparency (FAccT)

作者： Qadri, Rida Shelby, Renee Bennett, Cynthia L. Denton, Emily Google Res San Francisco CA 94103 USA Google Res New York NY USA

ISBN: (纸本)9781450372527

This paper presents a community-centered study of cultural limitations of text-to-image (T2I) models in the South Asian context. We theorize these failures using scholarship on dominant media regimes of representations and locate them within participants' reporting of their existing social marginalizations. We thus show how generative AI can reproduce an outsiders gaze for viewing South Asian cultures, shaped by global and regional power inequities. By centering communities as experts and soliciting their perspectives on T2I limitations, our study adds rich nuance into existing evaluative frameworks and deepens our understanding of the culturally-specific ways AI technologies can fail in non-Western and Global South settings. We distill lessons for responsible development of T2I models, recommending concrete pathways forward that can allow for recognition of structural inequalities.

关键词： human-centered AI AI harms cultural harms of AI text-to-image models generative AI non-western AI fairness South Asia qualitative research in AI failure modes

来源：评论

学校读者我要写书评

暂无评论

Typology of Risks of Generative text-to-image models 23

Typology of Risks of Generative Text-to-Image Models

引用

AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES)

作者： Bird, Charlotte Ungless, Eddie L. Kasirzadeh, Atoosa Univ Edinburgh Sch Informat Edinburgh Midlothian Scotland Univ Edinburgh Alan Turing Inst Edinburgh Midlothian Scotland

ISBN: (纸本)9798400702310

This paper investigates the direct risks and harms associated with modern text-to-image generative models, such as DALL-E and Mid-journey, through a comprehensive literature review. While these models offer unprecedented capabilities for generating images, their development and use introduce new types of risk that require careful consideration. Our review reveals significant knowledge gaps concerning the understanding and treatment of these risks despite some already being addressed. We offer a taxonomy of risks across six key stakeholder groups, inclusive of unexplored issues, and suggest future research directions. We identify 22 distinct risk types, spanning issues from data bias to malicious use. The investigation presented here is intended to enhance the ongoing discourse on responsible model development and deployment. By highlighting previously overlooked risks and gaps, it aims to shape subsequent research and governance initiatives, guiding them toward the responsible, secure, and ethically conscious evolution of text-to-image models.

关键词： Generative AI Generative models text-to-image models Responsible AI AI ethics AI safety AI governance AI risks

来源：评论

学校读者我要写书评

暂无评论

Unsafe Diffusion: On the Generation of Unsafe images and Hateful Memes From text-to-image models 23

Unsafe Diffusion: On the Generation of Unsafe Images and Hat...

引用

30th ACM SIGSAC Conference on Computer and Communications Security (ACM CCS)

作者： Qu, Yiting Shen, Xinyue He, Xinlei Backes, Michael Zannettou, Savvas Zhang, Yang CISPA Helmholtz Ctr Informat Secur Saarbrucken Germany Delft Univ Technol Delft Netherlands

ISBN: (纸本)9798400700507

State-of-the-art text-to-image models like Stable Diffusion and DALLE center dot 2 are revolutionizing how people generate visual content. At the same time, society has serious concerns about how adversaries can exploit such models to generate problematic or unsafe images. In this work, we focus on demystifying the generation of unsafe images and hateful memes from text-to-image models. We first construct a typology of unsafe images consisting of five categories (sexually explicit, violent, disturbing, hateful, and political). Then, we assess the proportion of unsafe images generated by four advanced text-to-image models using four prompt datasets. We find that text-to-image models can generate a substantial percentage of unsafe images;across four models and four prompt datasets, 14.56% of all generated images are unsafe. When comparing the four text-to-image models, we find different risk levels, with Stable Diffusion being the most prone to generating unsafe content (18.92% of all generated images are unsafe). Given Stable Diffusion's tendency to generate more unsafe content, we evaluate its potential to generate hateful meme variants if exploited by an adversary to attack a specific individual or community. We employ three image editing methods, DreamBooth, textual Inversion, and SDEdit, which are supported by Stable Diffusion to generate variants. Our evaluation result shows that 24% of the generated images using DreamBooth are hateful meme variants that present the features of the original hateful meme and the target individual/community;these generated images are comparable to hateful meme variants collected from the real world. Overall, our results demonstrate that the danger of large-scale generation of unsafe images is imminent. We discuss several mitigating measures, such as curating training data, regulating prompts, and implementing safety filters, and encourage better safeguard tools to be developed to prevent unsafe generation.

关键词： text-to-image models Unsafe images Hateful Memes

来源：评论

学校读者我要写书评

暂无评论

Designing with words: exploring the integration of text-to-image models in industrial design

引用

DIGITAL CREATIVITY 2024年第4期35卷 378-391页

作者： Barros, Mario Ai, Qi Aalborg Univ Rendsburggade 14 DK-9000 Aalborg Denmark

This study investigates the effectiveness of text-to-image (T2I) models in assisting industrial designers with the visualization of product concepts. Using action research and a reflection-in-action, this research conducts a series of experiments utilizing Stable Diffusion for generating chair concepts. The research findings are positioned within an existing framework of computational tools usage, and assess the quality of the generated images in supporting industrial design activities during the define and develop phases of the design process. While T2I models provide competent support to ambiguity and sketch improvement, their efficacy in generating coherent solutions for further development is *** study provides guidelines for the effective application of T2I models in industrial design practice. It discusses the T2I models' suitability for specific design activities, addresses associated risks, and proposes a taxonomy of prompt modifiers that ensures a closer match between input prompts and generated product concepts.

关键词： text-to-image models industrial design design process prompts creativity

来源：评论

学校读者我要写书评

暂无评论

SneakyPrompt: Jailbreaking text-to-image Generative models 45

SneakyPrompt: Jailbreaking Text-to-image Generative Models

引用

45th IEEE Symposium on Security and Privacy (SP)

作者： Yang, Yuchen Hui, Bo Yuan, Haolin Gong, Neil Cao, Yinzhi Johns Hopkins Univ Baltimore MD 21218 USA Duke Univ Durham NC USA

ISBN: (纸本)9798350331318;9798350331301

text-to-image generative models such as Stable Diffusion and DALL center dot E raise many ethical concerns due to the generation of harmful images such as Not-Safe-for-Work (NSFW) ones. To address these ethical concerns, safety filters are often adopted to prevent the generation of NSFW images. In this work, we propose SneakyPrompt, the first automated attack framework, to jailbreak text-to-image generative models such that they generate NSFW images even if safety filters are adopted. Given a prompt that is blocked by a safety filter, SneakyPrompt repeatedly queries the text-to-image generative model and strategically perturbs tokens in the prompt based on the query results to bypass the safety filter. Specifically, SneakyPrompt utilizes reinforcement learning to guide the perturbation of tokens. Our evaluation shows that SneakyPrompt successfully jailbreaks DALL center dot E 2 with closed-box safety filters to generate NSFW images. Moreover, we also deploy several state-of-the-art, open-source safety filters on a Stable Diffusion model. Our evaluation shows that SneakyPrompt not only successfully generates NSFW images, but also outperforms existing text adversarial attacks when extended to jailbreak text-to-image generative models, in terms of both the number of queries and qualities of the generated NSFW images. SneakyPrompt is open-source and available at this repository: https://***/Yuchen413/text2image safety.

关键词： Jailbreaking Not-Safe-For-Work (NSFW) text-to-image models

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：