检索结果-内蒙古大学图书馆

Superpixel Tokenization for Vision Transformers: Preserving Semantic Integrity in Visual Tokens

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lew, Jaihyun Jang, Soohyuk Lee, Jaehoon Yoo, Seungryong Kim, Eunji Lee, Saehyung Mok, Jisoo Kim, Siwon Yoon, Sungroh Interdisciplinary Program in AI Seoul National University Korea Republic of Department of Electrical and Computer Engineering Seoul National University Korea Republic of Amazon United States AIIS ASRI INMC ISRC Seoul National University Korea Republic of

Transformers, a groundbreaking architecture proposed for natural language processing (NLP), have also achieved remarkable success in computer vision. A cornerstone of their success lies in the attention mechanism, which models relationships among tokens. While the tokenization process in NLP inherently ensures that each token maintains semantic integrity without containing multiple meanings, the grid-based tokenization of Vision Transformer (ViT) relies on uniformly partitioned square image patches, which may result in an arbitrary mixing of visual concepts within a token. In this work, we propose a novel tokenization pipeline that replaces the grid-based tokenization with superpixels, encouraging each token to capture a distinct visual concept. Unlike square image patches, superpixels are formed in varying shapes, sizes, and locations, making direct substitution challenging. To address this, our pipeline first generates pixel-level embeddings and efficiently aggregates them within superpixel clusters, producing superpixel tokens that seamlessly replace patch tokens in ViT. Extensive experiments demonstrate that our approach enhances the performance of ViT on various downstream tasks and introduces intriguing properties such as adaptive inference and semantic integrity in tokens. © 2024, CC BY.

关键词： Semantics

A Design Method of Distributed Algorithms via Discrete-time Blended Dynamics Theorem

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Kim, Jeong Woo Lee, Jin Gyu Lee, Donggil Shim, Hyungbo ASRI Department of Electrical and Computer Engineering Seoul National University Seoul Korea Republic of CAP Research Group Department of Electrical and Electronic Engineering Imperial College London United Kingdom

We develop a discrete-time version of the blended dynamics theorem for the use of designing distributed computation algorithms. The blended dynamics theorem enables to predict the behavior of heterogeneous multi-agent systems. Therefore, once we get a blended dynamics for a particular computational task, design idea of node dynamics for individual heterogeneous agents can easily occur. In the continuous-time case, prediction by blended dynamics was enabled by high coupling gain among neighboring agents. In the discrete-time case, we propose an equivalent action, which we call multi-step coupling in this paper. Compared to the continuous-time case, the blended dynamics can have more variety depending on the coupling matrix. This benefit is demonstrated with three applications;distributed estimation of network size, distributed computation of the PageRank, and distributed computation of the degree sequence of a graph, which correspond to the coupling by doubly-stochastic, column-stochastic, and row-stochastic matrices, respectively. © 2022, CC BY-NC-ND.

关键词： Dynamics

Model Reference Gaussian Process Regression: Data-Driven Output Feedback Controller

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Kim, Hyuntae Chang, Hamin Shim, Hyungbo ASRI Department of Electrical and Computer Engineering Seoul National University 1 Gwanak-ro Gwanak-gu Seoul08826 Korea Republic of

Data-driven controls using Gaussian process regression have recently gained much attention. In such approaches, system identification by Gaussian process regression is mostly followed by model-based controller designs. However, the outcomes of Gaussian process regression are often too complicated to apply conventional control designs, which makes the numerical design such as model predictive control employed in many cases. To overcome the restriction, our idea is to perform Gaussian process regression to the inverse of the plant with the same input/output data for the conventional regression. With the inverse, one can design a model reference controller without resorting to numerical control methods. This paper considers single-input single-output (SISO) discrete-time nonlinear systems of minimum phase with relative degree one. It is highlighted that the model reference Gaussian process regression controller is designed directly from pre-collected input/output data without system identification. Copyright © 2022, The Authors. All rights reserved.

关键词： Controllers

Quantitative Manipulation of Custom Attributes on 3D-Aware Image Synthesis

学校读者我要写书评

暂无评论

Quantitative Manipulation of Custom Attributes on 3D-Aware I...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Hoseok Do EunKyung Yoo Taehyeong Kim Chul Lee Jin young Choi AI Lab CTO Division LG Electronics South Korea Dept. of Electrical and Computer Engineering ASRI Seoul National University South Korea Dept. of Biosystems Engineering Seoul National University South Korea

While 3D-based GAN techniques have been successfully applied to render photo-realistic 3D images with a variety of attributes while preserving view consistency, there has been little research on how to fine-control 3D images without limiting to a specific category of objects of their properties. To fill such research gap, we propose a novel image manipulation model of 3D-based GAN representations for a fine-grained control of specific custom attributes. By extending the latest 3D-based GAN models (e.g., EG3D), our user-friendly quantitative manipulation model enables a fine yet normalized control of 3D manipulation of multi-attribute quantities while achieving view consistency. We validate the effectiveness of our proposed technique both qualitatively and quantitatively through various experiments.

关键词：

Towards More Robust Interpretation via Local Gradient Alignment

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Joo, Sunghwan Jeong, Seokhyeon Heo, Juyeon Weller, Adrian Moon, Taesup Department of Electrical and Computer Engineering Sungkyunkwan University Korea Republic of Department of Electrical and Computer Engineering ASRI INMC IPAI Seoul National University Korea Republic of University of Cambridge United Kingdom The Alan Turing Institute United Kingdom

Neural network interpretation methods, particularly feature attribution methods, are known to be fragile with respect to adversarial input perturbations. To address this, several methods for enhancing the local smoothness of the gradient while training have been proposed for attaining robust feature attributions. However, the lack of considering the normalization of the attributions, which is essential in their visualizations, has been an obstacle to understanding and improving the robustness of feature attribution methods. In this paper, we provide new insights by taking such normalization into account. First, we show that for every non-negative homogeneous neural network, a naive 2-robust criterion for gradients is not normalization invariant, which means that two functions with the same normalized gradient can have different values. Second, we formulate a normalization invariant cosine distance-based criterion and derive its upper bound, which gives insight for why simply minimizing the Hessian norm at the input, as has been done in previous work, is not sufficient for attaining robust feature attribution. Finally, we propose to combine both 2 and cosine distance-based criteria as regularization terms to leverage the advantages of both in aligning the local gradient. As a result, we experimentally show that models trained with our method produce much more robust interpretations on CIFAR-10 and ImageNet-100 without significantly hurting the accuracy, compared to the recent baselines. To the best of our knowledge, this is the first work to verify the robustness of interpretation on a larger-scale dataset beyond CIFAR-10, thanks to the computational efficiency of our method. © 2022, CC BY-NC-SA.

关键词： Computational efficiency

Stability of linear systems with slow and fast time variation and switching

学校读者我要写书评

暂无评论

Stability of linear systems with slow and fast time variatio...

IEEE Conference on Decision and Control

作者： Daniel Liberzon Hyungbo Shim Coordinated Science Laboratory University of Illinois Urbana-Champaign Urbana USA Department of Electrical and Computer Engineering ASRI Seoul National University Korea

ISBN: (数字)9781665467612

ISBN: (纸本)9781665467629

We establish exponential stability for a class of linear systems with slow and fast time variation and switching. We use the averaging method to approximate the original system by the average system which only exhibits slow time variation and switching. We then apply a stability criterion recently developed for such systems to prove stability of the average system and, consequently, of the original system.

关键词： Linear systems Stability Stability criteria Switches Control systems Control theory Nonlinear systems

Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusion

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kim, Eunji Kim, Siwon Park, Minjun Entezari, Rahim Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Stability AI United Kingdom AIIS ASRI INMC ISRC Seoul National University Korea Republic of

Recent advancements in text-to-image models, such as Stable Diffusion, show significant demographic biases. Existing debiasing techniques rely heavily on additional training, which imposes high computational costs and risks of compromising core image generation functionality. This hinders them from being widely adopted to real-world applications. In this paper, we explore Stable Diffusion’s overlooked potential to reduce bias without requiring additional training. Through our analysis, we uncover that initial noises associated with minority attributes form "minority regions" rather than scattered. We view these "minority regions" as opportunities in SD to reduce bias. To unlock the potential, we propose a novel de-biasing method called ‘weak guidance,’ carefully designed to guide a random noise to the minority regions without compromising semantic integrity. Through analysis and experiments on various versions of SD, we demonstrate that our proposed approach effectively reduces bias without additional training, achieving both efficiency and preservation of core image generation functionality. © 2024, CC BY-NC-SA.

关键词： Digital elevation model

ENTROPY IS NOT ENOUGH FOR TEST-TIME ADAPTATION: FROM THE PERSPECTIVE OF DISENTANGLED FACTORS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lee, Jonghyun Jung, Dahuin Lee, Saehyung Park, Junsung Shin, Juhyeon Hwang, Uiwon Yoon, Sungroh Department of Electrical and Computer Engineering Seoul National University Korea Republic of School of Computer Science and Engineering Soongsil University Korea Republic of Interdisciplinary Program in Artificial Intelligence Seoul National University Korea Republic of Division of Digital Healthcare Yonsei University Korea Republic of AIIS ASRI INMC and ISRC Seoul National University Korea Republic of

Test-time adaptation (TTA) fine-tunes pre-trained deep neural networks for unseen test data. The primary challenge of TTA is limited access to the entire test dataset during online updates, causing error accumulation. To mitigate it, TTA methods have utilized the model output’s entropy as a confidence metric that aims to determine which samples have a lower likelihood of causing error. Through experimental studies, however, we observed the unreliability of entropy as a confidence metric for TTA under biased scenarios and theoretically revealed that it stems from the neglect of the influence of latent disentangled factors of data on predictions. Building upon these findings, we introduce a novel TTA method named Destroy Your Object (DeYO), which leverages a newly proposed confidence metric named Pseudo-Label Probability Difference (PLPD). PLPD quantifies the influence of the shape of an object on prediction by measuring the difference between predictions before and after applying an object-destructive transformation. DeYO consists of sample selection and sample weighting, which employ entropy and PLPD concurrently. For robust adaptation, DeYO prioritizes samples that dominantly incorporate shape information when making predictions. Our extensive experiments demonstrate the consistent superiority of DeYO over baseline methods across various scenarios, including biased and wild. Project page is publicly available at https://***/DeYO/. Copyright © 2024, The Authors. All rights reserved.

关键词： Entropy

Multi-Level Branched Regularization for Federated Learning

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Kim, Jinkyu Kim, Geeho Han, Bohyung Computer Vision Laboratory Department of Electrical and Computer Engineering & ASRI Seoul National University Korea Republic of Interdisciplinary Program of Artificial Intelligence Seoul National University Korea Republic of

A critical challenge of federated learning is data heterogeneity and imbalance across clients, which leads to inconsistency between local networks and unstable convergence of global models. To alleviate the limitations, we propose a novel architectural regularization technique that constructs multiple auxiliary branches in each local model by grafting local and global subnetworks at several different levels and that learns the representations of the main pathway in the local model congruent to the auxiliary hybrid pathways via online knowledge distillation. The proposed technique is effective to robustify the global model even in the non-iid setting and is applicable to various federated learning frameworks conveniently without incurring extra communication costs. We perform comprehensive empirical studies and demonstrate remarkable performance gains in terms of accuracy and efficiency compared to existing methods. The source code is available in our project page1 Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation