检索结果-内蒙古大学图书馆

vision Transformer Hashing for Image Retrieval

学校读者我要写书评

暂无评论

Vision Transformer Hashing for Image Retrieval

2022 IEEE International Conference on Multimedia and Expo, ICME 2022

作者： Dubey, Shiv Ram Singh, Satish Kumar Chu, Wei-Ta Indian Institute of Information Technology Computer Vision and Biometrics Lab Allahabad India National Cheng Kung University Dept. of Computer Science and Information Engineering Taiwan

ISBN: (数字)9781665485630

ISBN: (纸本)9781665485630

Recently, Transformer has emerged as a new architecture in deep learning by utilizing self-attention without convolution. Transformer is also extended to vision Transformer (ViT) for the visual recognition with a promising performance on ImageNet. In this paper, we propose a vision Transformer Hashing (VTS) for image retrieval. We utilize the pre-trained ViT on ImageNet as the backbone network and add the hashing head. The proposed VTS model is fine tuned for hashing under six different image retrieval frameworks with their objective functions. We perform the extensive experiments on CIFAR10, ImageNet, NUS-Wide, and COCO datasets. The proposed VTS based image retrieval outperforms the recent state-of-the-art hashing techniques with a significant margin. We also find the proposed VTS model as the backbone network is better than the existing networks, such as AlexNet and ResNet. The code is released at https://***/shivram1987/visionTransformerHashing. © 2022 IEEE.

关键词： Image retrieval

ETLNet: An Efficient TCN-BiLSTM Network for Road Anomaly Detection Using Smartphone Sensors

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ansari, Mohd Faiz Sandilya, Rakshit Javed, Mohammed Doermann, David Computer Vision and Biometrics Lab Department of IT IIIT Allahabad India Computer Science and Engineering University at Buffalo BuffaloNY United States

Road anomalies can be defined as irregularities on the road surface or in the surface itself. Some may be intentional (such as speed-bumps), accidental (such as materials falling off a truck), or the result of roads’ excessive use or low or no maintenance, such as potholes. Despite their varying origins, these irregularities often harm vehicles substantially. Speed bumps are intentionally placed for safety but are dangerous due to their non-standard shape, size, and lack of proper markings. Potholes are unintentional and can also cause severe damage. To address the detection of these anomalies, we need an automated road monitoring system. Today, various systems exist that use visual information to track these anomalies. Still, due to poor lighting conditions and improper or missing markings, they may go undetected and have severe consequences for public transport, automated vehicles, etc. In this paper, the Enhanced Temporal-BiLSTM Network (ETLNet) is introduced as a novel approach that integrates two Temporal Convolutional Network (TCN) layers with a Bidirectional Long Short-Term Memory (BiLSTM) layer. This combination is tailored to detect anomalies effectively irrespective of lighting conditions, as it depends not on visuals but smartphone inertial sensor data. Our methodology employs accelerometer and gyroscope sensors, typically in smartphones, to gather data on road conditions. Empirical evaluations demonstrate that the ETLNet model maintains an F1-score for detecting speed bumps of 99.3%. The ETLNet model’s robustness and efficiency significantly advance automated road surface monitoring technologies. © 2024, CC BY.

关键词： Long short-term memory

TRANSFORMER-BASED CLIPPED CONTRASTIVE QUANTIZATION LEARNING FOR UNSUPERVISED IMAGE RETRIEVAL

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Dubey, Ayush Dubey, Shiv Ram Singh, Satish Kumar Chu, Wei-Ta Computer Vision and Biometrics Lab Indian Institute of Information Technology Allahabad India Department of Computer Science and Information Engineering National Cheng Kung University Taiwan

Unsupervised image retrieval aims to learn the important visual characteristics without any given level to retrieve the similar images for a given query image. The Convolutional Neural Network (CNN)-based approaches have been extensively exploited with self-supervised contrastive learning for image hashing. However, the existing approaches suffer due to lack of effective utilization of global features by CNNs and biased-ness created by false negative pairs in the contrastive learning. In this paper, we propose a TransClippedCLR model by encoding the global context of an image using Transformer having local context through patch based processing, by generating the hash codes through product quantization and by avoiding the potential false negative pairs through clipped contrastive learning. The proposed model is tested with superior performance for unsupervised image retrieval on benchmark datasets, including CIFAR10, NUS-Wide and Flickr25K, as compared to the recent state-of-the-art deep models. The results using the proposed clipped contrastive learning are greatly improved on all datasets as compared to same backbone network with vanilla contrastive learning. Copyright © 2024, The Authors. All rights reserved.

关键词： Benchmarking

DIFEM: Key-points Interaction based Feature Extraction Module for Violence Recognition in Videos

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Mittal, Himanshu Basak, Suvramalya Gautam, Anjali Computer Vision and Biometrics Lab Department of Information Technology Indian Institute of Information Technology Allahabad U.P. Prayagraj India

Violence detection in surveillance videos is a critical task for ensuring public safety. As a result, there is increasing need for efficient and lightweight systems for automatic detection of violent behaviours. In this work, we propose an effective method which leverages human skeleton key-points to capture inherent properties of violence, such as rapid movement of specific joints and their close proximity. At the heart of our method is our novel Dynamic Interaction Feature Extraction Module (DIFEM) which captures features such as velocity, and joint intersections, effectively capturing the dynamics of violent behavior. With the features extracted by our DIFEM, we use various classification algorithms such as Random Forest, Decision tree, AdaBoost and k-Nearest Neighbor. Our approach has substantially lesser amount of parameter expense than the existing state-of-the-art (SOTA) methods employing deep learning techniques. We perform extensive experiments on three standard violence recognition datasets, showing promising performance in all three datasets. Our proposed method surpasses several SOTA violence recognition methods. © 2024, CC BY.

关键词： Random forests

Transformer-Based Clipped Contrastive Quantization Learning For Unsupervised Image Retrieval

学校读者我要写书评

暂无评论

Transformer-Based Clipped Contrastive Quantization Learning ...

IEEE International Conference on Image Processing

作者： Ayush Dubey Shiv Ram Dubey Satish Kumar Singh Wei-Ta Chu Computer Vision and Biometrics Lab Indian Institute of Information Technology Allahabad India Department of Computer Science and Information Engineering National Cheng Kung University Taiwan

ISBN: (数字)9798350349399

ISBN: (纸本)9798350349405

关键词： computer vision Visualization Quantization (signal) Image coding Codes Image retrieval Contrastive learning

Non-Uniform Illumination Attack for Fooling Convolutional Neural Networks

学校读者我要写书评

暂无评论

IEEE Transactions on Artificial Intelligence

IEEE Transactions on Artificial Intelligence 2025年

作者： Jain, Akshay Dubey, Shiv Ram Singh, Satish Kumar Santosh, K.C. Chaudhuri, Bidyut Baran Indian Institute of Information Technology Allahabad Computer Vision and Biometrics Lab Department of Information Technology Uttar Pradesh Prayagraj211015 India University of South Dakota AI Research Lab Department of Computer Science VermillionSD57069 United States Indian Statistical Institute Computer Vision and Pattern Recognition Unit Kolkata700108 India

Convolutional Neural Networks (CNNs) have made remarkable strides;however, they remain susceptible to vulnerabilities, particularly to image perturbations that humans can easily recognize. This weakness, often termed as 'attacks,' underscores the limited robustness of CNNs and the need for research into fortifying their resistance against such manipulations. This study introduces a novel Non-Uniform Illumination (NUI) attack technique, where images are subtly altered using varying NUI masks. Extensive experiments are conducted on widely-accepted datasets including CIFAR10, TinyImageNet, CalTech256 and NWPU-RESISC45 focusing on image classification with 12 different NUI masks. The resilience of VGG, ResNet, MobilenetV3-small, InceptionV3 and efficientNet b0 models against NUI attacks are evaluated. Our results show a substantial decline in the CNN models' classification accuracy when subjected to NUI attacks, due to changes in the image pixel value distribution, indicating their vulnerability under non-uniform illumination. To mitigate this, a defense strategy is proposed, including NUI-attacked images, generated through the new NUI transformation, into the training set. The results demonstrate a significant enhancement in CNN model performance when confronted with perturbed images affected by NUI attacks. This strategy seeks to bolster CNN models' resilience against NUI attacks. A comparative study with other attack techniques shows the effectiveness of the NUI attack and defense technique. © 2020 IEEE.

关键词： Convolutional neural networks

PTSR: Patch Translator for Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Baghel, Neeraj Dubey, Shiv Ram Singh, Satish Kumar The Computer Vision and Biometrics Lab Department of Information Technology Indian Institute of Information Technology Allahabad Prayagraj India

Image super-resolution generation aims to generate a high-resolution image from its low-resolution image. However, more complex neural networks bring high computational costs and memory storage. It is still an active area for offering the promise of overcoming resolution limitations in many applications. In recent years, transformers have made significant progress in computer vision tasks as their robust self-attention mechanism. However, recent works on the transformer for image super-resolution also contain convolution operations. We propose a patch translator for image super-resolution (PTSR) to address this problem. The proposed PTSR is a transformer-based GAN network with no convolution operation. We introduce a novel patch translator module for regenerating the improved patches utilising multi-head attention, which is further utilised by the generator to generate the 2× and 4× super-resolution images. The experiments are performed using benchmark datasets, including DIV2K, Set5, Set14, and BSD100. The results of the proposed model is improved on an average for 4× super-resolution by 21.66% in PNSR score and 11.59% in SSIM score, as compared to the best competitive models. We also analyse the proposed loss and saliency map to show the effectiveness of the proposed method. Copyright © 2023, The Authors. All rights reserved.

关键词： Convolution

SRTransGAN: Image Super-Resolution using Transformer based Generative Adversarial Network

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Baghel, Neeraj Dubey, Shiv Ram Singh, Satish Kumar Computer Vision and Biometrics Lab Department of Information Technology Indian Institute of Information Technology Allahabad211015 India

Image super-resolution aims to synthesize high-resolution image from a low-resolution image. It is an active area to overcome the resolution limitations in several applications like low-resolution object-recognition, medical image enhancement, etc. The generative adversarial network (GAN) based methods have been the state-of-the-art for image super-resolution by utilizing the convolutional neural networks (CNNs) based generator and discriminator networks. However, the CNNs are not able to exploit the global information very effectively in contrast to the transformers, which are the recent breakthrough in deep learning by exploiting the self-attention mechanism. Motivated from the success of transformers in language and vision applications, we propose a SRTransGAN for image super-resolution using transformer based GAN. Specifically, we propose a novel transformer-based encoder-decoder network as a generator to generate 2× images and 4× images. We design the discriminator network using vision transformer which uses the image as sequence of patches and hence useful for binary classification between synthesized and real high-resolution images. The proposed SRTransGAN outperforms the existing methods by 4.38% on an average of PSNR and SSIM scores. We also analyze the saliency map to understand the learning ability of the proposed method. Copyright © 2023, The Authors. All rights reserved.

关键词： Generative adversarial networks

Fire and Smoke Detection using an Enhanced YOLOv8 Model

学校读者我要写书评

暂无评论

Fire and Smoke Detection using an Enhanced YOLOv8 Model

computer vision and Machine Intelligence (CVMI), International Conference on

作者： Bhavesh Kumar Bohra Bulla Rajesh Mohammed Javed David Doermann Department of IT Computer Vision and Biometrics Lab IIIT Allahabad India Department of Computer Science and Engineering IIIT Sricity India Department of Computer Science and Engineering University at Buffalo Buffalo NY USA

ISBN: (数字)9798350376876

ISBN: (纸本)9798350376883

Due in part to global warming, the frequency of wildfires has increased in recent years, significantly impacting humans, wildlife, and forest ecosystems. To address this challenge, using technology for early fire detection through smoke can help firefighters and other stakeholders limit their spread and reduce the impact. In this research, we adopt the latest object detection model - YOLOv8 and use a fine-tuned version for detecting smoke and fire using the latest datasets. YOLOv8 backbone begins with two convolutional (Conv) layers, followed by a series of alternating Conv layers and C2F modules, and ends with a Spatial Pyramid Pooling - Fast (SPPF) module. Our improvement involves adding two extra Conv layers at the beginning of the backbone, between the alternating sequence of Conv and C2F layers. As the smoke and fire detection problem involves searching for smaller objects in the image, increasing the feature extraction layers improves the performance of smoke and fire detection. The fine-tuned model is tested with the FASDD v4 dataset, which depicts fire, smoke, non-fire, and non-smoke scenarios.

关键词： Climate change Fire safety Smoke detectors YOLO Global warming Wildfires Fires