SLayR: Scene Layout Generation with Rectified Flow

作     者：Braunstein, Cameron Petekkaya, Hevra Lenssen, Jan Eric Toneva, Mariya Ilg, Eddy 

作者机构：Saarland University Saarland Informatics Campus Germany Max Planck Institute for Informatics Saarland Informatics Campus Germany Max Planck Institute for Software Systems Saarland Informatics Campus Germany Computer Vision and Machine Perception Lab University of Technology Nuremberg Germany 

出 版 物：《arXiv》 (arXiv)

年 卷 期：2024年

核心收录：

主　　题：Flow simulation 

摘      要：We introduce SLayR, Scene Layout Generation with Rectified flow, a novel transformer-based model for text-to-layout generation which can then be paired with existing layout-to-image models to produce images. SLayR addresses a domain in which current text-to-image pipelines struggle: generating scene layouts that are of significant variety and plausibility, when the given prompt is ambiguous and does not provide constraints on the scene. SLayR surpasses existing baselines including LLMs in unconstrained generation, and can generate layouts from an open caption set. To accurately evaluate the layout generation, we introduce a new benchmark suite, including numerical metrics and a carefully designed repeatable human-evaluation procedure that assesses the plausibility and variety of generated images. We show that our method sets a new state of the art for achieving both at the same time, while being at least 3× times smaller in the number of parameters. Copyright © 2024, The Authors. All rights reserved.