检索结果-内蒙古大学图书馆

Zero-Shot image Harmonization with Generative Model Prior

学校读者我要写书评

暂无评论

IEEE Transactions on Multimedia 2025年

作者： Chen, Jianqi Zhang, Yilan Zou, Zhengxia Chen, Keyan Shi, Zhenwei Beihang University Image Processing Center School of Astronautics Beijing100191 China Beihang University State Key Laboratory of Virtual Reality Technology and Systems School of Astronautics Beijing100191 China Beihang University Department of Guidance Navigation and Control School of Astronautics Beijing100191 China

We propose a zero-shot approach to image harmonization, aiming to overcome the reliance on large amounts of synthetic composite images in existing methods. These methods, while showing promising results, involve significant training expenses and often struggle with generalization to unseen images. To this end, we introduce a fully modularized framework inspired by human behavior. Leveraging the reasoning capabilities of recent foundation models in language and vision, our approach comprises three main stages. Initially, we employ a pretrained vision-language model (VLM) to generate descriptions for the composite image. Subsequently, these descriptions guide the foreground harmonization direction of a text-to-image generative model (T2I). We refine text embeddings for enhanced representation of imaging conditions and employ self-attention and edge maps for structure preservation. Following each harmonization iteration, an evaluator determines whether to conclude or modify the harmonization direction. The resulting framework, mirroring human behavior, achieves harmonious results without the need for extensive training. We present compelling visual results across diverse scenes and objects, along with quantitative comparisons validating the effectiveness of our approach. Our code is available at https://***/WindVChen/Diff-Harmonization. © 1999-2012 IEEE.

关键词： Visual languages

Zero-Shot Audio Captioning Using Soft and Hard Prompts

学校读者我要写书评

暂无评论

IEEE Transactions on Audio, Speech and Language Processing

IEEE Transactions on Audio, Speech and Language processing 2025年 33卷 2045-2058页

作者： Yiming Zhang Xuenan Xu Ruoyi Du Haohe Liu Yuan Dong Zheng-Hua Tan Wenwu Wang Zhanyu Ma Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing China Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Centre for Vision Speech and Signal Processing University of Surrey Guildford U.K. Department of Electronic Systems Aalborg University Aalborg Denmark

In traditional audio captioning methods, a model is usually trained in a fully supervised manner using a human-annotated dataset containing audio-text pairs and then evaluated on the test set from the same dataset. Such methods have two limitations. First, these methods are often data-hungry and require time-consuming and expensive human annotations to obtain audio-text pairs. Second, these models often suffer from performance degradation in cross-domain scenarios, i.e., when the input audio comes from a different domain than the training set, and this issue has received little attention. To address these issues, we propose a new zero-shot method for audio captioning. Our method is built on the contrastive language-audio pre-training (CLAP) model. During training, the model reconstructs the ground-truth caption using the CLAP text encoder. In the inference stage, the model generates text descriptions from the CLAP audio embeddings of given audio inputs. To enhance the ability of the model in transitioning from text-to-text generation to audio-to-text generation, we propose to use the mixed-augmentations-based soft prompt to learn more robust latent representations, leveraging instance replacement and embedding augmentation. Additionally, we introduce the retrieval-based acoustic-aware hard prompt to improve the cross-domain performance of the model by employing the domain-agnostic label information of sound events. Extensive experiments on AudioCaps and Clotho benchmarks show the effectiveness of our proposed method, which outperforms other zero-shot audio captioning approaches for in-domain scenarios and outperforms the compared methods for cross-domain scenarios, underscoring the generalization ability of our method.

关键词： Training Decoding Semantics Data models Acoustics Electronic mail Benchmark testing Transformers Robustness Perturbation methods

COVIDx CT-3: A Large-scale, Multinational, Open-Source Benchmark Dataset for Computer-aided COVID-19 Screening from Chest CT images

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Gunraj, Hayden Tuinstra, Tia Wong, Alexander Vision and Image Processing Lab University of Waterloo Canada Department of Systems Design Engineering University of Waterloo Canada Waterloo AI Institute University of Waterloo Canada DarwinAI Corp Canada

Computed tomography (CT) has been widely explored as a COVID-19 screening and assessment tool to complement RT-PCR testing. To assist radiologists with CT-based COVID-19 screening, a number of computer-aided systems have been proposed. However, many proposed systems are built using CT data which is limited in both quantity and diversity. Motivated to support efforts in the development of machine learning-driven screening systems, we introduce COVIDx CT-3, a large-scale multinational benchmark dataset for detection of COVID-19 cases from chest CT images. COVIDx CT-3 includes 431,205 CT slices from 6,068 patients across at least 17 countries, which to the best of our knowledge represents the largest, most diverse dataset of COVID-19 CT images in open-access form. Additionally, we examine the data diversity and potential biases of the COVIDx CT-3 dataset, finding that significant geographic and class imbalances remain despite efforts to curate data from a wide variety of sources. Copyright © 2022, The Authors. All rights reserved.

关键词： Large dataset

Hybrid Data-Free Knowledge Distillation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Jialiang Chen, Shuo Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education China Jiangsu Key Laboratory of Image and Video Understanding for Social Security China Center for Advanced Intelligence Project RIKEN Japan Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

Data-free knowledge distillation aims to learn a compact student network from a pre-trained large teacher network without using the original training data of the teacher network. Existing collection-based and generation-based methods train student networks by collecting massive real examples and generating synthetic examples, respectively. However, they inevitably become weak in practical scenarios due to the difficulties in gathering or emulating sufficient real-world data. To solve this problem, we propose a novel method called Hybrid Data-Free Distillation (HiDFD), which leverages only a small amount of collected data as well as generates sufficient examples for training student networks. Our HiDFD comprises two primary modules, i.e., the teacher-guided generation and student distillation. The teacher-guided generation module guides a Generative Adversarial Network (GAN) by the teacher network to produce high-quality synthetic examples from very few real-world collected examples. Specifically, we design a feature integration mechanism to prevent the GAN from overfitting and facilitate the reliable representation learning from the teacher network. Meanwhile, we drive a category frequency smoothing technique via the teacher network to balance the generative training of each category. In the student distillation module, we explore a data inflation strategy to properly utilize a blend of real and synthetic data to train the student network via a classifier-sharing-based feature alignment technique. Intensive experiments across multiple benchmarks demonstrate that our HiDFD can achieve state-of-the-art performance using 120 times less collected data than existing methods. Code is available at https://***/tangjialiang97/HiDFD. Copyright © 2024, The Authors. All rights reserved.

关键词： Students

Deep Learning Methods for Ship Classification: From Visible to Infrared images 5

学校读者我要写书评

暂无评论

Deep Learning Methods for Ship Classification: From Visible ...

5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023

作者： Liu, Tianci Qin, Hengjia Zhan, Zhuo Liu, Yunpeng Shenyang Institute of Automation Chinese Academy of Sciences Shenyang110016 China Institutes for Robotics and Intelligent Manufacturing Chinese Academy of Sciences Shenyang110169 China University of Chinese Academy of Sciences Beijing100049 China Chinese Academy of Sciences Key Laboratory of Opto-Electronic Information Processing Shenyang110016 China Key Laboratory of Image Understanding and Computer Vision Liaoning Province Shenyang110016 China The Third Military Representative Office of the Air Force Equipment Department in Shenyang Region Liaoning Province Shenyang110027 China

ISBN: (纸本)9798350357950

Deep learning methods have achieved excellent performances on visual tasks of target recognition and classification. The rapid development of autonomous seafaring vessels comes up with the requirement to recognize other maritime ships day and night. However, the recognition of ships based on the deep neural networks may not always access the results as expectation when the ships are under the nighttime environment. To this issue, we consider the ship recognition task under the deep learning framework with paired visible images and infrared images. In this article, we propose an end-to-end convolutional network based on visible images and infrared images of the autonomous seafaring vessels. To demonstrate the effectiveness of our model, we choose the V AIS dataset to test the performance of classifying the maritime ships. Experimental results show that the proposed network outperforms the state-of-the-art methods based on the V AIS database. © 2023 IEEE.

关键词： deep learning infrared images neural network Ship recognition

GEM: Context-Aware Gaze EstiMation with Visual Search Behavior Matching for Chest Radiograph

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Liu, Shaonan Chen, Wenting Liu, Jie Luo, Xiaoling Shen, Linlin Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University China Department of Electrical Engineering City University of Hong Kong Hong Kong AI Research Center for Medical Image Analysis and Diagnosis Shenzhen University China Guangdong Provincial Key Laboratory of Intelligent Information Processing China

Gaze estimation is pivotal in human scene comprehension tasks, particularly in medical diagnostic analysis. Eye-tracking technology facilitates the recording of physicians’ ocular movements during image interpretation, thereby elucidating their visual attention patterns and information-processing strategies. In this paper, we initially define the context-aware gaze estimation problem in medical radiology report settings. To understand the attention allocation and cognitive behavior of radiologists during the medical image interpretation process, we propose a context-aware Gaze EstiMation (GEM) network that utilizes eye gaze data collected from radiologists to simulate their visual search behavior patterns throughout the image interpretation process. It consists of a context-awareness module, visual behavior graph construction, and visual behavior matching. Within the context-awareness module, we achieve intricate multimodal registration by establishing connections between medical reports and images. Subsequently, for a more accurate simulation of genuine visual search behavior patterns, we introduce a visual behavior graph structure, capturing such behavior through high-order relationships (edges) between gaze points (nodes). To maintain the authenticity of visual behavior, we devise a visual behavior-matching approach, adjusting the high-order relationships between them by matching the graph constructed from real and estimated gaze points. Extensive experiments on four publicly available datasets demonstrate the superiority of GEM over existing methods and its strong generalizability, which also provides a new direction for the effective utilization of diverse modalities in medical image interpretation and enhances the interpretability of models in the field of medical imaging. https://***/Tiger-SN/GEM. Copyright © 2024, The Authors. All rights reserved.

关键词： Eye movements

Difference in Uptake Pattern Heterogeneity and Reproducibility of Radiomics Features Between FAPI and FDG Pet/Ct Imaging of Tumors

学校读者我要写书评

暂无评论

Difference in Uptake Pattern Heterogeneity and Reproducibili...

IEEE International Symposium on Biomedical Imaging

作者： Tianmeng Xu Yanchao Huang Tingting Ang Shuilin Chu Peiyin Luo Shunfang Wang Yufeng Zhang Lijun Lu Ganghua Tang Wenbing Lv School of Information and Yunnan Key Laboratory of Intelligent Systems and Computing Yunnan University Kunming China Department of Nuclear Medicine Laboratory for Quality Control and Evaluation of Radiopharmaceuticals Nanfang Hospital Southern Medical University Guangzhou China School of Biomedical Engineering and Guangdong Provincial Key Laboratory of Medical Image Processing Southern Medical University Guangzhou China

ISBN: (数字)9798350313338

ISBN: (纸本)9798350313345

To investigate the difference in uptake pattern heterogeneity between fluorodeoxyglucose (FDG) and fibroblast activation protein inhibitor (FAPI) PET/CT and the reproducibility of radiomics features with respect to different calculation methods of standardized uptake value (SUV). 210 patients with various tumors were involved. Among the extracted 497 radiomics features, there are only 20 features that exhibited an ICC difference of more than 0.25 between the two tracers when using different SUV calculation methods, revealing the similar influence of SUV calculation methods. In contrast, the ICC values for most texture features were generally lower (0.22-0.39) between FDG and FAPI PET images, indicating a large difference in uptake pattern heterogeneity between the two tracers. Our results demonstrated that the influence of SUV calculation methods on the reproducibility of FDG and FAPI PET features is similar. The uptake pattern heterogeneity characterized by radiomics features between FDG and FAPI PET images is significantly different.

关键词： Proteins Protein engineering Inhibitors Fibroblasts Feature extraction Reproducibility of results Radiomics

Memristor-Based Circuit of Sequential Associative Memory with Memory Interactions and Stimulus Similarity Effect

学校读者我要写书评

暂无评论

IEEE Transactions on Cognitive and Developmental systems 2025年

作者： Xiong, Dongdong Wang, Xiaoping Jiang, Yiming Yang, Chao Jiang, Man Lai, Jingang Zeng, Zhigang Huazhong University of Science and Technology School of Artificial Intelligence and Automation Wuhan430074 China Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Wuhan430074 China Hubei Key Laboratory of Braininspired Intelligent Systems Wuhan430074 China Huazhong University of Science and Technology Institute of Artificial Intelligence School of Artificial Intelligence and Automation Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Wuhan430074 China Guangxi University of Science and Technology School of Computer Science and Technology Liuzhou545006 China Huazhong University of Science and Technology Department of Physiology School of Basic Medicine and Tongji Medical College Wuhan430030 China

Most proposed memristor-based circuits of associative memory consider various mechanisms in only one associative memory. Few works on circuit design of sequential associative memory have been reported. In this paper, a memristor-based circuit of sequential associative memory with memory interactions and stimulus similarity effect is designed. Two associative memories, the prior associative memory (PAM) and the later associative memory (LAM), are formed successively. The PAM modifies the rate of the formation of the LAM, and the LAM, in turn, affects the strength of the PAM, which are two interactions in sequential associative memory, called transfer and retroaction. In addition, the magnitudes of transfer and retroaction are determined by the similarity of the conditioned stimuli. The above functions are realized by the ring input processing module, memory module, transfer module and retroaction module. It has been identified in circuit analysis that the proposed circuit is power efficient and has good robustness. Furthermore, the proposed circuit has promising applications for fire rescue robots. © 2016 IEEE.

关键词： Associative storage

A Late-Stage Bitemporal Feature Fusion Network for Semantic Change Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Chenyao Zhang, Haotian Guo, Han Zou, Zhengxia Shi, Zhenwei The Image Processing Center School of Astronautics Beihang University Beijing100191 China The State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China The Shanghai Artificial Intelligence Laboratory Shanghai200232 China The Department of Guidance Navigation and Control School of Astronautics Beihang University Beijing100191 China

Semantic change detection is an important task in geoscience and earth observation. By producing a semantic change map for each temporal phase, both the land use land cover categories and change information can be interpreted. Recently some multi-task learning based semantic change detection methods have been proposed to decompose the task into semantic segmentation and binary change detection subtasks. However, previous works comprise triple branches in an entangled manner, which may not be optimal and hard to adopt foundation models. Besides, lacking explicit refinement of bitemporal features during fusion may cause low accuracy. In this letter, we propose a novel late-stage bitemporal feature fusion network to address the issue. Specifically, we propose local global attentional aggregation module to strengthen feature fusion, and propose local global context enhancement module to highlight pivotal semantics. Comprehensive experiments are conducted on two public datasets, including SECOND and Landsat-SCD. Quantitative and qualitative results show that our proposed model achieves new state-of-the-art performance on both datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics