检索结果-内蒙古大学图书馆

Iterative Sound Source Localization for Unknown Number of Sources

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Fu, Yanjie Ge, Meng Yin, Haoran Qian, Xinyuan Wang, Longbiao Zhang, Gaoyan Dang, Jianwu Tianjin Key Laboratory of Cognitive Computing and Application College of Intelligence and Computing Tianjin University Tianjin China Department of Electrical and Computer Engineering National University of Singapore Singapore Japan Advanced Institute of Science and Technology Ishikawa Japan

Sound source localization aims to seek the direction of arrival (DOA) of all sound sources from the observed multichannel audio. For the practical problem of unknown number of sources, existing localization algorithms attempt to predict a likelihood-based coding (i.e., spatial spectrum) and employ a pre-determined threshold to detect the source number and corresponding DOA value. However, these threshold-based algorithms are not stable since they are limited by the careful choice of threshold. To address this problem, we propose an iterative sound source localization approach called ISSL, which can iteratively extract each source's DOA without threshold until the termination criterion is met. Unlike threshold-based algorithms, ISSL designs an active source detector network based on binary classifier to accept residual spatial spectrum and decide whether to stop the iteration. By doing so, our ISSL can deal with an arbitrary number of sources, even more than the number of sources seen during the training stage. The experimental results show that our ISSL achieves significant performance improvements in both DOA estimation and source number detection compared with the existing threshold-based algorithms. Copyright © 2022, The Authors. All rights reserved.

关键词： Direction of arrival

Cross-camera Trajectories Help Person Retrieval in a Camera Network

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Xin Xie, Xiaohua Lai, Jianhuang Zheng, Wei-Shi The School of Computer Science and Engineering The Guangdong Province Key Laboratory of Information Security Technology The Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education Sun Yat-sen University Guangzhou510006 China

We are concerned with retrieving a query person from multiple videos captured by a non-overlapping camera network. Existing methods often rely on purely visual matching or consider temporal constraints but ignore the spatial information of the camera network. To address this issue, we propose a pedestrian retrieval framework based on cross-camera trajectory generation, which integrates both temporal and spatial information. To obtain pedestrian trajectories, we propose a novel cross-camera spatio-temporal model that integrates pedestrians’ walking habits and the path layout between cameras to form a joint probability distribution. Such a spatio-temporal model among a camera network can be specified using sparsely sampled pedestrian data. Based on the spatio-temporal model, cross-camera trajectories can be extracted by the conditional random field model and further optimized by restricted non-negative matrix factorization. Finally, a trajectory re-ranking technique is proposed to improve the pedestrian retrieval results. To verify the effectiveness of our method, we construct the first cross-camera pedestrian trajectory dataset, the Person Trajectory Dataset, in real surveillance scenarios. Extensive experiments verify the effectiveness and robustness of the proposed method. Copyright © 2022, The Authors. All rights reserved.

关键词： Trajectories

Adversarial Feature Augmentation for Cross-domain Few-shot Classification

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Hu, Yanxu Ma, Andy J. School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Few-shot classification is a promising approach to solving the problem of classifying novel classes with only limited annotated data for training. Existing methods based on meta-learning predict novel-class labels for (target domain) testing tasks via meta knowledge learned from (source domain) training tasks of base classes. However, most existing works may fail to generalize to novel classes due to the probably large domain discrepancy across domains. To address this issue, we propose a novel adversarial feature augmentation (AFA) method to bridge the domain gap in few-shot learning. The feature augmentation is designed to simulate distribution variations by maximizing the domain discrepancy. During adversarial training, the domain discriminator is learned by distinguishing the augmented features (unseen domain) from the original ones (seen domain), while the domain discrepancy is minimized to obtain the optimal feature encoder. The proposed method is a plug-and-play module that can be easily integrated into existing few-shot learning methods based on meta-learning. Extensive experiments on nine datasets demonstrate the superiority of our method for cross-domain few-shot classification compared with the state of the art. Code is available at https : //***/youthhoo/AFA For Few shot learning. Copyright © 2022, The Authors. All rights reserved.

关键词： Learning systems

Multimodal Action Quality Assessment

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zeng, Ling-An Zheng, Wei-Shi School of Artificial Intelligence Sun Yat-sen University Guangdong Zhuhai519082 China School of Computer Science and Engineering Sun Yat-sen University Guangdong Guangzhou510275 China Guangdong Key Laboratory of Information Security Technology Sun Yat-sen University Guangdong Guangzhou510275 China Key Laboratory of Machine Intelligence and Advanced Computing Sun Yat-sen University Ministry of Education Guangdong Guangzhou510275 China

Action quality assessment (AQA) is to assess how well an action is performed. Previous works perform modelling by only the use of visual information, ignoring audio information. We argue that although AQA is highly dependent on visual information, the audio is useful complementary information for improving the score regression accuracy, especially for sports with background music, such as figure skating and rhythmic gymnastics. To leverage multimodal information for AQA, i.e., RGB, optical flow and audio information, we propose a Progressive Adaptive Multimodal Fusion Network (PAMFN) that separately models modality-specific information and mixed-modality information. Our model consists of with three modality-specific branches that independently explore modality-specific information and a mixed-modality branch that progressively aggregates the modality-specific information from the modality-specific branches. To build the bridge between modality-specific branches and the mixed-modality branch, three novel modules are proposed. First, a Modality-specific Feature Decoder module is designed to selectively transfer modality-specific information to the mixed-modality branch. Second, when exploring the interaction between modality-specific information, we argue that using an invariant multimodal fusion policy may lead to suboptimal results, so as to take the potential diversity in different parts of an action into consideration. Therefore, an Adaptive Fusion Module is proposed to learn adaptive multimodal fusion policies in different parts of an action. This module consists of several FusionNets for exploring different multimodal fusion strategies and a PolicyNet for deciding which FusionNets are enabled. Third, a module called Cross-modal Feature Decoder is designed to transfer cross-modal features generated by Adaptive Fusion Module to the mixed-modality branch. Our extensive experiments validate the efficacy of the proposed method, and our method achieves state-of-the-a

关键词： Damage detection

Exploring Dual-task Correlation for Pose Guided Person Image Generation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhang, Pengze Yang, Lingxiao Lai, Jianhuang Xie, Xiaohua School of Computer Science and Engineering Sun Yat-Sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Pose Guided Person Image Generation (PGPIG) is the task of transforming a person image from the source pose to a given target pose. Most of the existing methods only focus on the ill-posed source-to-target task and fail to capture reasonable texture mapping. To address this problem, we propose a novel Dual-task Pose Transformer Network (DPTN), which introduces an auxiliary task (i.e., source-to-source task) and exploits the dual-task correlation to promote the performance of PGPIG. The DPTN is of a Siamese structure, containing a source-to-source self-reconstruction branch, and a transformation branch for source-to-target generation. By sharing partial weights between them, the knowledge learned by the source-to-source task can effectively assist the source-to-target learning. Furthermore, we bridge the two branches with a proposed Pose Transformer Module (PTM) to adaptively explore the correlation between features from dual tasks. Such correlation can establish the fine-grained mapping of all the pixels between the sources and the targets, and promote the source texture transmission to enhance the details of the generated target images. Extensive experiments show that our DPTN outperforms state-of-the-arts in terms of both PSNR and LPIPS. In addition, our DPTN only contains 9.79 million parameters, which is significantly smaller than other approaches. Our code is available at: https://***/PangzeCheung/Dual-task-Pose-Transformer-Network. © 2022, CC BY.

关键词： Textures

Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chen, Qi Yang, Lingxiao Lai, Jianhuang Xie, Xiaohua School of Computer Science and Engineering Sun Yat-Sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Weakly Supervised Semantic Segmentation (WSSS) based on image-level labels has attracted much attention due to low annotation costs. Existing methods often rely on Class Activation Mapping (CAM) that measures the correlation between image pixels and classifier weight. However, the classifier focuses only on the discriminative regions while ignoring other useful information in each image, resulting in incomplete localization maps. To address this issue, we propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Specifically, IPE tailors prototypes for every image to capture complete regions, formed our Image-Specific CAM (IS-CAM), which is realized by two sequential steps. In addition, GSC is proposed to construct the consistency of general CAM and our specific IS-CAM, which further optimizes the feature representation and empowers a self-correction ability of prototype exploration. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014 segmentation benchmark and results show our SIPE achieves new state-of-the-art performance using only image-level labels. The code is available at https://***/chenqi1126/SIPE. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantic Segmentation

Visual Class Incremental Learning with Textual Priors Guidance based on an Adapted Vision-Language Model

学校读者我要写书评

暂无评论

IEEE Transactions on Multimedia 2025年

作者： Zhang, Wentao Yu, Tong Wang, Ruixuan Xie, Jianhui Trucco, Emanuele Zheng, Wei-Shi Yang, Xiaobo Sun Yat-sen Univerisity School of Computer Science and Engineering Guangzhou510275 China Peng Cheng Laboratory Shenzhen518066 China MOE Key Laboratory of Machine Intelligence and Advanced Computing Guangzhou510275 China Second Affiliated Hospital Guangzhou University of Chinese Medicine State Key Laboratory of Dampness Syndrome of Chinese Medicine Guangzhou510260 China Guangdong Provincial Key Laboratory of Clinical Research on Traditional Chinese Medicine Syndrome Guangzhou510120 China University of Dundee School of Science and Engineering DundeeDD1 4HN United Kingdom

An ideal artificial intelligence (AI) system should have the capability to continually learn like humans. However, when learning new knowledge, AI systems often suffer from catastrophic forgetting of old knowledge. Although many continual learning methods have been proposed, they often ignore the issue of misclassifying similar classes and make insufficient use of textual priors of visual classes to improve continual learning performance. In this study, we propose a continual learning framework based on a pre-trained vision-language model (VLM) that does not require storing old class data. This framework utilizes parameter-efficient fine-tuning of the VLM's text encoder for constructing a shared and consistent semantic textual space throughout the continual learning process. The textual priors of visual classes are encoded by the adapted VLM's text encoder to generate discriminative semantic representations, which are then used to guide the learning of visual classes. Additionally, fake out-of-distribution (OOD) images constructed from each training image further assist in the learning of visual classes. Extensive empirical evaluations on three natural datasets and one medical dataset demonstrate the superiority of the proposed framework. The source code is available at https://***/OpenMedIA/CIL_Adapterd_VLM. © 1999-2012 IEEE.

关键词： Contrastive Learning

Inspection of EEG Signals for Noninvasive Blood Glucose Monitoring in Prediabetes Diagnosis

学校读者我要写书评

暂无评论

Inspection of EEG Signals for Noninvasive Blood Glucose Moni...

2023 IEEE International Symposium on Medical Measurements and Applications, MeMeA 2023

作者： Igbe, Tobore Samuel, Oluwarotimi Williams Li, Jingzhen Kulwa, Frank Kandwal, Abhishek Nie, Zedong University of Virginia Center for Diabetes Technology Department of Psychiatry and Neurobehavioural Sciences School of Medicine CharlottesvilleVA United States University of Derby School of Computing and Engineering DerbyDE22 1GB United Kingdom Center for Minimally Invasive and Surgical Devices Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Guandong Shenzhen China Cas Key Laboratory of Human-Machine Intelligence-Synergy Systems and Shenzhen College of Advanced Technology Ucas Guangdong Shenzhen518055 China Shoolini University Department of Physics and Materials Science Himachal Pradesh India

ISBN: (纸本)9781665493840

Prediabetes is a metabolic disorder where the blood glucose (BG) level is higher than normal but not high as diabetes;early diagnosis can prevent health complications and death. However, to determine the BG level, it is required to prick the finger, which causes pain and discomfort. To eliminate this problem, there is a need to investigate noninvasive techniques to estimate BG values and continuous BG monitoring. In this paper, we investigated the changes in electroencephalogram (EEG) frequency parameters that have been scarcely considered for prediabetes diagnosis. We analyzed EEG signals after carrying out an oral glucose tolerance test on 25 participants. Five frequency bands of EEG signals were obtained continuously in 3 positions;frontal (F), occipital (O), and parental (P). The analysis is performed using a boxplot to examine the pattern of the recorded signals. The result shows that the EEG signal from the O position has a sensitivity of 95.3% in the left hemisphere, the P location has 90.3% in the right hemisphere, and F has 93% in the left hemisphere. This observation shows the appropriate location and the combination of EEG frequency parameters, such as the alpha and beta mean power from O and P, which can be integrated into a wearable device to provide a promising clinical solution for noninvasive blood glucose monitoring and prediabetes diagnosis. © 2023 IEEE.

关键词： Electroencephalography

Texture-guided Saliency Distilling for Unsupervised Salient Object Detection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhou, Huajun Qiao, Bo Yang, Lingxiao Lai, Jianhuang Xie, Xiaohua School of Computer Science and Engineering Sun Yat-sen University China Guangdong Province Key Laboratory of Information Security Technology China Key Laboratory of Machine Intelligence and Advanced Computing Ministry of Education China

Deep Learning-based Unsupervised Salient Object Detection (USOD) mainly relies on the noisy saliency pseudo labels that have been generated from traditional handcraft methods or pre-trained networks. To cope with the noisy labels problem, a class of methods focus on only easy samples with reliable labels but ignore valuable knowledge in hard samples. In this paper, we propose a novel USOD method to mine rich and accurate saliency knowledge from both easy and hard samples. First, we propose a Confidence-aware Saliency Distilling (CSD) strategy that scores samples conditioned on samples’ confidences, which guides the model to distill saliency knowledge from easy samples to hard samples progressively. Second, we propose a Boundary-aware Texture Matching (BTM) strategy to refine the boundaries of noisy labels by matching the textures around the predicted boundaries. Extensive experiments on RGB, RGBD, RGB-T, and video SOD benchmarks prove that our method achieves state-of-the-art USOD performance. Code is available at ***/moothes/A2S-v2. Copyright © 2022, The Authors. All rights reserved.

关键词： Textures