检索结果-内蒙古大学图书馆

MMC: Multi-Modal Colorization of Images using Textual Descriptions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ghosh, Subhankar Bhattacharya, Saumik Roy, Prasun Pal, Umapada Blumenstein, Michael Faculty of Engineering and IT University of Technology SydneyNSW Australia The Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India Indian Institute of Technology Kharagpur India

Handling various objects with different colors is a significant challenge for image colorization techniques. Thus, for complex real-world scenes, the existing image colorization algorithms often fail to maintain color consistency. In this work, we attempt to integrate textual descriptions as an auxiliary condition, along with the grayscale image that is to be colorized, to improve the fidelity of the colorization process. To do so, we have proposed a deep network that takes two inputs (grayscale image and the respective encoded text description) and tries to predict the relevant color components. Also, we have predicted each object in the image and have colorized them with their individual description to incorporate their specific attributes in the colorization process. After that, a fusion model fuses all the image objects (segments) to generate the final colorized image. As the respective textual descriptions contain color information of the objects present in the image, text encoding helps to improve the overall quality of predicted colors. In terms of performance, the proposed method outperforms existing colorization techniques in terms of LPIPS, PSNR and SSIM metrics. © 2023, CC BY.

关键词： Color

Multi-scale Attention Guided Pose Transfer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Roy, Prasun Bhattacharya, Saumik Ghosh, Subhankar Pal, Umapada The Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India Indian Institute of Technology Kharagpur India

Pose transfer refers to the probabilistic image generation of a person with a previously unseen novel pose from another image of that person having a different pose. Due to potential academic and commercial applications, this problem is extensively studied in recent years. Among the various approaches to the problem, attention guided progressive generation is shown to produce state-of-the-art results in most cases. In this paper, we present an improved network architecture for pose transfer by introducing attention links at every resolution level of the encoder and decoder. By utilizing such dense multi-scale attention guided approach, we are able to achieve significant improvement over the existing methods both visually and analytically. We conclude our findings with extensive qualitative and quantitative comparisons against several existing methods on the DeepFashion dataset. © 2022, CC BY.

关键词： Network architecture

DocSegTr: An Instance-Level End-to-End Document Image Segmentation Transformer

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Biswas, Sanket Banerjee, Ayan Lladós, Josep Pal, Umapada Computer Vision Center Computer Science Department Universitat Autònoma de Barcelona Spain Computer Vision and Pattern Recognition Unit Indian Statistical Institute India

Understanding documents with rich layouts is an essential step towards information extraction. Business intelligence processes often require the extraction of useful semantic content from documents at a large scale for subsequent decision-making tasks. In this context, instance-level segmentation of different document objects (title, sections, figures etc.) has emerged as an interesting problem for the document analysis and understanding community. To advance the research in this direction, we present a transformer-based model called DocSegTr for end-to-end instance segmentation of complex layouts in document images. The method adapts a twin attention module, for semantic reasoning, which helps to become highly computationally efficient compared with the state-of-the-art. To the best of our knowledge, this is the first work on transformer-based document segmentation. Extensive experimentation on competitive benchmarks like PubLayNet, PRIMA, Historical Japanese (HJ) and TableBank demonstrate that our model achieved comparable or better segmentation performance than the existing state-of-the-art approaches with the average precision of 89.4, 40.3, 83.4 and 93.3. This simple and flexible framework could serve as a promising baseline for instance-level recognition tasks in document images. © 2022, CC BY-SA.

关键词： Decision making

A Survey of Artificial Hummingbird Algorithm and Its Variants: Statistical Analysis, Performance Evaluation, and Structural Reviewing

学校读者我要写书评

暂无评论

Archives of Computational Methods in Engineering 2025年第1期32卷 269-310页

作者： Hosseinzadeh, Mehdi Rahmani, Amir Masoud Husari, Fatimatelbatoul Mahmoud Alsalami, Omar Mutab Marzougui, Mehrez Nguyen, Gia Nhu Lee, Sang-Woong Institute of Research and Development Duy Tan University Da Nang Viet Nam School of Medicine and Pharmacy Duy Tan University Da Nang Viet Nam Future Technology Research Center National Yunlin University of Science and Technology Yunlin Taiwan Department of Communication and Computer Engineering Faculty of Engineering Cihan University-Erbil Kurdistan Region Iraq Department of Electrical Engineering College of Engineering Taif University Taif21944 Saudi Arabia Department of Computer Engineering College of Computer Science King Khalid University Abha61413 Saudi Arabia School of Computer Science Duy Tan University Da Nang Viet Nam Pattern Recognition and Machine Learning Lab Gachon University 1342 Seongnamdaero Sujeonggu Seongnam13120 Korea Republic of

In the last few decades, metaheuristic algorithms that use the laws of nature have been used dramatically in numerous and complex optimization problems. The artificial hummingbird algorithm (AHA) is one of the metaheuristic algorithms that was invented in 2022 based on the foraging and migration behavior of the hummingbird for modeling and solving optimization problems. The algorithm initially starts with an initial random population of solutions. It then uses iterative processes and hummingbird position updates to balance exploration and exploitation toward the most optimal solutions. This paper has a detailed and extensive review of the AHA algorithm considering the aspects of hybrid, improved, binary, multi-objective, and optimization problems. In addition, a wide range of applications of AHA in various fields such as feature selection, image processing, scheduling, Internet of Things, classification, clustering, financial and economic issues, forecasting, wireless sensor networks, and many engineering challenges are explored. The statistical and numerical results showed that the AHA algorithm with deep learning methods, Levy flight, and opposition-based learning had the best performance. Also, the AHA algorithm is most widely used in solving multimodal optimization problems and continuous functions. © The Author(s) under exclusive licence to International Center for Numerical Methods in Engineering (CIMNE) 2024.

关键词： Optimization

UNIFORMER: UNIFIED TRANSFORMER FOR EFFICIENT SPATIOTEMPORAL REPRESENTATION LEARNING 10

学校读者我要写书评

暂无评论

UNIFORMER: UNIFIED TRANSFORMER FOR EFFICIENT SPATIOTEMPORAL ...

10th International Conference on Learning Representations, ICLR 2022

作者： Li, Kunchang Wang, Yali Gao, Peng Song, Guanglu Liu, Yu Li, Hongsheng Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Shanghai AI Laboratory Shanghai China SenseTime Research The Chinese University of Hong Kong Hong Kong

It is a challenging task to learn rich and multi-scale spatiotemporal semantics from high-dimensional videos, due to large local redundancy and complex global dependency between video frames. The recent advances in this research have been mainly driven by 3D convolutional neural networks and vision transformers. Although 3D convolution can efficiently aggregate local context to suppress local redundancy from a small 3D neighborhood, it lacks the capability to capture global dependency because of the limited receptive field. Alternatively, vision transformers can effectively capture long-range dependency by self-attention mechanism, while having the limitation on reducing local redundancy with blind similarity comparison among all the tokens in each layer. Based on these observations, we propose a novel Unified transFormer (UniFormer) which seamlessly integrates merits of 3D convolution and spatiotemporal self-attention in a concise transformer format, and achieves a preferable balance between computation and accuracy. Different from traditional transformers, our relation aggregator can tackle both spatiotemporal redundancy and dependency, by learning local and global token affinity respectively in shallow and deep layers. We conduct extensive experiments on the popular video benchmarks, e.g., Kinetics-400, Kinetics-600, and Something-Something V1&V2. With only ImageNet-1K pretraining, our UniFormer achieves 82.9%/84.8% top-1 accuracy on Kinetics-400/Kinetics-600, while requiring 10× fewer GFLOPs than other state-of-the-art methods. For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60.9% and 71.2% top-1 accuracy respectively. Code is available at https://***/Sense-X/UniFormer. © 2022 ICLR 2022 - 10th International Conference on Learning Representationss. All rights reserved.

关键词： Redundancy

KV Inversion: KV Embeddings Learning for Text-Conditioned Real Image Action Editing

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Jiancheng Liu, Yifan Qin, Jin Chen, Shifeng ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China University of Chinese Academy of Sciences Beijing China

Text-conditioned image editing is a recently emerged and highly practical task, and its potential is immeasurable. However, most of the concurrent methods are unable to perform action editing, i.e. they can not produce results that conform to the action semantics of the editing prompt and preserve the content of the original image. To solve the problem of action editing, we propose KV Inversion, a method that can achieve satisfactory reconstruction performance and action editing, which can solve two major problems: 1) the edited result can match the corresponding action, and 2) the edited object can retain the texture and identity of the original real image. In addition, our method does not require training the Stable Diffusion model itself, nor does it require scanning a large-scale dataset to perform time-consuming training. Copyright © 2023, The Authors. All rights reserved.

关键词： Textures

Bootstrap Diffusion Model Curve Estimation for High Resolution Low-Light Image Enhancement

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Huang, Jiancheng Liu, Yifan Chen, Shifeng ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen China University of Chinese Academy of Sciences Beijing China

Learning-based methods have attracted a lot of research attention and led to significant improvements in low-light image enhancement. However, most of them still suffer from two main problems: expensive computational cost in high resolution images and unsatisfactory performance in simultaneous enhancement and denoising. To address these problems, we propose BDCE, a bootstrap diffusion model that exploits the learning of the distribution of the curve parameters instead of the normal-light image itself. Specifically, we adopt the curve estimation method to handle the high-resolution images, where the curve parameters are estimated by our bootstrap diffusion model. In addition, a denoise module is applied in each iteration of curve adjustment to denoise the intermediate enhanced result of each iteration. We evaluate BDCE on commonly used benchmark datasets, and extensive experiments show that it achieves state-of-the-art qualitative and quantitative performance. Copyright © 2023, The Authors. All rights reserved.

关键词： Image enhancement

A New U-Net Based System for Multi-Cultural Wedding Image Classification

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Shivakumara, Palaiahnakote Kumar, C. Pavan Nemade, Jagrut J. Michael, Kshitiz Kumar, Akash Anami, Basavaraj S. Pal, Umapada Faculty of Computer Science and Information Technology University of Malaya Kuala Lumpur Malaysia Indian Institute of Information Technology Dharwad India KLE Institute of Technology India Computer Vision and Pattern Recognition Unit Indian Statistical Institute Kolkata India

Use of social media for communication, sharing expressing views, broadcasting news, threatening and blackmailing has become an integral part of society. One such activity is understanding multi-cultural wedding images uploaded on social media. This paper presents a novel method based on the combination of U-Net, Convolutional Neural Network and Random Forest for classification of multicultural wedding images. In the case of wedding images, bride and bridegroom draw the attention of the viewers. This observation led to propose U-Net for segmenting the region of bride and bridegroom in a novel way. Similarly, it is noted that the costumes of bride and bridegroom are vital information for differentiating different cultures. This cue motivated us to extract features using CNN for classification. Since the extracted features using CNN are capable of discriminating images of different classes, we propose a simple and effective Random-Forest for Multicultural Wedding Image Classification. The efficiency of the proposed model is demonstrated by testing it on our own dataset of six multi-cultural wedding classes and standard dataset of wedding and non-wedding images classes. Experimental results on both the datasets show that the proposed model outperforms the state-of-the-art models in terms of average classification rate © 2023, The Authors. All rights reserved.

关键词： Convolutional neural networks

Med-DANet V2: A Flexible Dynamic Architecture for Efficient Medical Volumetric Segmentation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Shen, Haoran Zhang, Yifu Wang, Wenxuan Chen, Chen Liu, Jing Song, Shanshan Li, Jiangyun School of Automation and Electrical Engineering University of Science and Technology Beijing China Center for Research in Computer Vision University of Central Florida United States National Lab of Pattern Recognition Institute of Automation Chinese Academy of Sciences China

Recent works have shown that the computational efficiency of 3D medical image (e.g. CT and MRI) segmentation can be impressively improved by dynamic inference based on slice-wise complexity. As a pioneering work, a dynamic architecture network for medical volumetric segmentation (i.e. Med-DANet [44]) has achieved a favorable accuracy and efficiency trade-off by dynamically selecting a suitable 2D candidate model from the pre-defined model bank for different slices. However, the issues of incomplete data analysis, high training costs, and the two-stage pipeline in Med-DANet require further improvement. To this end, this paper further explores a unified formulation of the dynamic inference framework from the perspective of both the data itself and the model structure. For each slice of the input volume, our proposed method dynamically selects an important foreground region for segmentation based on the policy generated by our Decision Network and Crop Position Network. Besides, we propose to insert a stage-wise quantization selector to the employed segmentation model (e.g. U-Net) for dynamic architecture adapting. Extensive experiments on BraTS 2019 and 2020 show that our method achieves comparable or better performance than previous state-of-the-art methods with much less model complexity. Compared with previous methods Med-DANet and TransBTS with dynamic and static architecture respectively, our framework improves the model efficiency by up to nearly 4.1 and 17.3 times with comparable segmentation results on BraTS 2019. Copyright © 2023, The Authors. All rights reserved.

关键词： Computational efficiency