检索结果-内蒙古大学图书馆

On the Robustness of Normalizing Flows for Inverse Problems in Imaging

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hong, Seongmin Park, Inbum Chun, Se Young Dept. of Electrical and Computer Engineering INMC Interdisciplinary Program in AI Seoul National University Korea Republic of

Conditional normalizing flows can generate diverse image samples for solving inverse problems. Most normalizing flows for inverse problems in imaging employ the conditional affine coupling layer that can generate diverse images quickly. However, unintended severe artifacts are occasionally observed in the output of them. In this work, we address this critical issue by investigating the origins of these artifacts and proposing the conditions to avoid them. First of all, we empirically and theoretically reveal that these problems are caused by "exploding inverse" in the conditional affine coupling layer for certain out-of-distribution (OOD) conditional inputs. Then, we further validated that the probability of causing erroneous artifacts in pixels is highly correlated with a Mahalanobis distance-based OOD score for inverse problems in imaging. Lastly, based on our investigations, we propose a remark to avoid exploding inverse and then based on it, we suggest a simple remedy that substitutes the affine coupling layers with the modified rational quadratic spline coupling layers in normalizing flows, to encourage the robustness of generated image samples. Our experimental results demonstrated that our suggested methods effectively suppressed critical artifacts occurring in normalizing flows for super-resolution space generation and low-light image enhancement. Copyright © 2022, The Authors. All rights reserved.

关键词： Inverse problems

Text2Scene: Text-driven Indoor Scene Stylization with Part-Aware Details

学校读者我要写书评

暂无评论

Text2Scene: Text-driven Indoor Scene Stylization with Part-A...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Inwoo Hwang Hyeonwoo Kim Young Min Kim Department of Electrical and Computer Engineering Seoul National University Interdisciplinary Program in Artificial Intelligence and INMC Seoul National University

We propose Text2Scene, a method to automatically create realistic textures for virtual scenes composed of multiple objects. Guided by a reference image and text descriptions, our pipeline adds detailed texture on labeled 3D geometries in the room such that the generated colors respect the hierarchical structure or semantic parts that are often composed of similar materials. Instead of applying flat stylization on the entire scene at a single step, we obtain weak semantic cues from geometric segmentation, which are further clarified by assigning initial colors to segmented parts. Then we add texture details for individual objects such that their projections on image space exhibit feature embedding aligned with the embedding of the input. The decomposition makes the entire pipeline tractable to a moderate amount of computation resources and memory. As our framework utilizes the existing resources of image and text embedding, it does not require dedicated datasets with high-quality textures designed by skillful artists. To the best of our knowledge, it is the first practical and scalable approach that can create detailed and realistic textures of the desired style that maintain structural context for scenes with multiple objects.

关键词：

Superpixel Tokenization for Vision Transformers: Preserving Semantic Integrity in Visual Tokens

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lew, Jaihyun Jang, Soohyuk Lee, Jaehoon Yoo, Seungryong Kim, Eunji Lee, Saehyung Mok, Jisoo Kim, Siwon Yoon, Sungroh Interdisciplinary Program in AI Seoul National University Korea Republic of Department of Electrical and Computer Engineering Seoul National University Korea Republic of Amazon United States AIIS ASRI INMC ISRC Seoul National University Korea Republic of

Transformers, a groundbreaking architecture proposed for natural language processing (NLP), have also achieved remarkable success in computer vision. A cornerstone of their success lies in the attention mechanism, which models relationships among tokens. While the tokenization process in NLP inherently ensures that each token maintains semantic integrity without containing multiple meanings, the grid-based tokenization of Vision Transformer (ViT) relies on uniformly partitioned square image patches, which may result in an arbitrary mixing of visual concepts within a token. In this work, we propose a novel tokenization pipeline that replaces the grid-based tokenization with superpixels, encouraging each token to capture a distinct visual concept. Unlike square image patches, superpixels are formed in varying shapes, sizes, and locations, making direct substitution challenging. To address this, our pipeline first generates pixel-level embeddings and efficiently aggregates them within superpixel clusters, producing superpixel tokens that seamlessly replace patch tokens in ViT. Extensive experiments demonstrate that our approach enhances the performance of ViT on various downstream tasks and introduces intriguing properties such as adaptive inference and semantic integrity in tokens. © 2024, CC BY.

关键词： Semantics

Adversarial Environment Design via Regret-Guided Diffusion Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chung, Hojun Lee, Junseo Kim, Minsoo Kim, Dohyeong Oh, Songhwai Interdisciplinary Program in Artificial Intelligence and ASRI Seoul National University Korea Republic of Department of Electrical and Computer Engineering and ASRI Seoul National University Korea Republic of

Training agents that are robust to environmental changes remains a significant challenge in deep reinforcement learning (RL). Unsupervised environment design (UED) has recently emerged to address this issue by generating a set of training environments tailored to the agent’s capabilities. While prior works demonstrate that UED has the potential to learn a robust policy, their performance is constrained by the capabilities of the environment generation. To this end, we propose a novel UED algorithm, adversarial environment design via regret-guided diffusion models (ADD). The proposed method guides the diffusion-based environment generator with the regret of the agent to produce environments that the agent finds challenging but conducive to further improvement. By exploiting the representation power of diffusion models, ADD can directly generate adversarial environments while maintaining the diversity of training environments, enabling the agent to effectively learn a robust policy. Our experimental results demonstrate that the proposed method successfully generates an instructive curriculum of environments, outperforming UED baselines in zero-shot generalization across novel, out-of-distribution environments. Project page: https://***/projects/ADD © 2024, CC BY-NC-SA.

关键词： Adversarial machine learning

Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lee, Saehyung Yoon, Seunghyun Bui, Trung Shi, Jing Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Adobe Research Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of

Multimodal large language models (MLLMs) excel at generating highly detailed captions but often produce hallucinations. Our analysis reveals that existing hallucination detection methods struggle with detailed captions. We attribute this to the increasing reliance of MLLMs on their generated text, rather than the input image, as the sequence length grows. To address this issue, we propose a multiagent approach that leverages LLM-MLLM collaboration to correct given captions. Additionally, we introduce an evaluation framework and a benchmark dataset to facilitate the systematic analysis of detailed captions. Our experiments demonstrate that our proposed evaluation method better aligns with human judgments of factuality than existing metrics and that existing approaches to improve the MLLM factuality may fall short in hyper-detailed image captioning tasks. In contrast, our proposed method significantly enhances the factual accuracy of captions, even improving those generated by GPT-4V. Finally, we highlight a limitation of VQA-centric benchmarking by demonstrating that an MLLM’s performance on VQA benchmarks may not correlate with its ability to generate detailed image captions. © 2024, CC BY.

关键词： Image enhancement

Safe CoR: A Dual-Expert Approach to Integrating Imitation Learning and Safe Reinforcement Learning Using Constraint Rewards

学校读者我要写书评

暂无评论

Safe CoR: A Dual-Expert Approach to Integrating Imitation Le...

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Hyeokjin Kwon Gunmin Lee Junseo Lee Songhwai Oh Interdisciplinary Program in Artificial Intelligence and ASRI Seoul National University Seoul Korea Department of Electrical and Computer Engineering and ASRI Seoul National University Seoul Korea

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

In the realm of autonomous agents, ensuring safety and reliability in complex and dynamic environments remains a paramount challenge. Safe reinforcement learning addresses these concerns by introducing safety constraints, but still faces challenges in navigating intricate environments such as complex driving situations. To overcome these challenges, we present the safe constraint reward (Safe CoR) framework, a novel method that utilizes two types of expert demonstrations—reward expert demonstrations focusing on performance optimization and safe expert demonstrations prioritizing safety. By exploiting a constraint reward (CoR), our framework guides the agent to balance performance goals of reward sum with safety constraints. We test the proposed framework in diverse environments, including the safety gym, metadrive, and the real-world Jackal platform. Our proposed framework improves algorithm performance by 39% and reduces constraint violations by 88% on the real-world Jackal platform, highlighting its effectiveness. Through this innovative approach, we expect significant advancements in real-world performance, leading to transformative effects in the realm of safe and reliable autonomous agents.

关键词： Navigation Imitation learning Focusing Reinforcement learning Autonomous agents Safety Reliability Optimization Intelligent robots Faces

Guest Editorial: Special Issue on Technology Trends in Control, Robotics and Systems - For the 30th Anniversary of the Institute of Control, Robotics and Systems -

学校读者我要写书评

暂无评论

Journal of Institute of Control, Robotics and Systems 2024年第4期30卷 295-295页

作者： Ko, Nak Yong Kang, Taesam Song, Jin Woo Lim, Myo Taeg Jo, Kang-Hyun Department of Electronic Engineering Interdisciplinary Program in IT-Bio Convergence System Chosun University Korea Republic of Department of Mechanical and Aerospace Engineering Konkuk University Korea Republic of Department of Intelligent Mechatronics Engineering Sejong University Korea Republic of School of Electrical Engineering Korea University Korea Republic of Department of Electrical Electronic and Computer Engineering University of Ulsan Korea Republic of

Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kim, Eunji Kim, Siwon Park, Minjun Entezari, Rahim Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Stability AI United Kingdom AIIS ASRI INMC ISRC Seoul National University Korea Republic of

Recent advancements in text-to-image models, such as Stable Diffusion, show significant demographic biases. Existing debiasing techniques rely heavily on additional training, which imposes high computational costs and risks of compromising core image generation functionality. This hinders them from being widely adopted to real-world applications. In this paper, we explore Stable Diffusion’s overlooked potential to reduce bias without requiring additional training. Through our analysis, we uncover that initial noises associated with minority attributes form "minority regions" rather than scattered. We view these "minority regions" as opportunities in SD to reduce bias. To unlock the potential, we propose a novel de-biasing method called ‘weak guidance,’ carefully designed to guide a random noise to the minority regions without compromising semantic integrity. Through analysis and experiments on various versions of SD, we demonstrate that our proposed approach effectively reduces bias without additional training, achieving both efficiency and preservation of core image generation functionality. © 2024, CC BY-NC-SA.

关键词： Digital elevation model

DEAL: Decoupled Classifier with Adaptive Linear Modulation for Group Robust Early Diagnosis of MCI to AD Conversion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lee, Donggyu Park, Juhyeon Moon, Taesup The Department of Electrical and Computer Engineering Sungkyunkwan University Suwon16419 Korea Republic of The Interdisciplinary Program in Artificial Intelligence Seoul National University Seoul08826 Korea Republic of The Department of Electrical and Computer Engineering Automation and Systems Research Institute Institute of New Media and Communications Interdisciplinary Program in Artificial Intelligence Artificial Intelligent Institute Seoul National University Seoul08826 Korea Republic of

While deep learning-based Alzheimer's disease (AD) diagnosis has recently made significant advancements, particularly in predicting the conversion of mild cognitive impairment (MCI) to AD based on MRI images, there remains a critical gap in research regarding the group robustness of the diagnosis. Although numerous studies pointed out that deep learning-based classifiers may exhibit poor performance in certain groups by relying on unimportant attributes, this issue has been largely overlooked in the early diagnosis of MCI to AD conversion. In this paper, we present the first comprehensive investigation of the group robustness in the early diagnosis of MCI to AD conversion using MRI images, focusing on disparities in accuracy between groups, specifically sMCI and pMCI individuals divided by age. Our experiments reveal that standard classifiers consistently underperform for certain groups across different architectures, highlighting the need for more tailored approaches. To address this, we propose a novel method, dubbed DEAL (DEcoupled classifier with Adaptive Linear modulation), comprising two key components: (1) a linear modulation of features from the penultimate layer, incorporating easily obtainable age and cognitive indicative tabular features, and (2) a decoupled classifier that provides more tailored decision boundaries for each group, further improving performance. Through extensive experiments and evaluations across different architectures, we demonstrate the efficacy of DEAL in improving the group robustness of the MCI to AD conversion prediction. © 2024, CC BY-NC-SA.

关键词：