Advancing Sequential Manga Colorization for AR Through Data Synthesis

作     者：Golyadkin, Maksim Saraev, Sergey Makarov, Ilya 

作者机构：Artificial Intelligence Res Inst AIRI Moscow 105064 Russia HSE Univ Int Lab Intelligent Syst & Struct Anal Moscow 101000 Russia ITMO Univ AI Talent Hub St Petersburg 191101 Russia ISP RAS Res Ctr Trusted Artificial Intelligence Moscow 109004 Russia 

出 版 物：《IEEE ACCESS》 (IEEE Access)

年 卷 期：2025年第13卷

页      面：7526-7537页

核心收录：

基　　金：Computational Resources of HPC Facilities at HSE University Research Centers in the field of Artificial Intelligence Analytical Center (ACRF)in Accordance with the Subsidy Agreement [000000D730321P5Q0002] Ivannikov Institute for System Programming of Russian Academy of Sciences [70-2021-00142] 

主　　题：Diffusion models Adaptation models Image color analysis Synthetic data Data models Visualization Training Image synthesis Benchmark testing Translation Manga colorization augmented reality dataset benchmark 

摘      要：Manga colorization in augmented reality (AR) environments presents unique challenges, particularly when colorizing manga pages captured in photos under various real-world conditions. Testing models in AR settings for manga colorization has been a significant challenge, primarily because of the absence of suitable datasets tailored for this task. To address this, we propose a benchmark for evaluating existing colorization models. We first collected a relatively small dataset of manga book photos taken in settings suitable for AR applications. Then, we developed a method that leverages a pretrained diffusion model to generate synthetic photos from scans of manga pages. Using large datasets of manga scans, we created an extensive synthetic dataset. Combining both real and synthetic data, we established a comprehensive benchmark for manga colorization in AR scenarios. We tested existing models for natural image and manga colorization on this benchmark. As a result, our evaluation showed that current models are not well-suited for AR-based colorization tasks, indicating a need for further improvement.