检索结果-内蒙古大学图书馆

Dual encoding for video retrieval by text

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Dong, Jianfeng Li, Xirong Xu, Chaoxi Yang, Xun Yang, Gang Wang, Xun Wang, Meng College of Computer and Information Engineering Zhejiang Gongshang University Hangzhou310035 China Key Lab of Data Engineering and Knowledge Engineering Renmin University of China The AI and Media Computing Lab School of Information Renmin University of China Beijing100872 China School of Computing National University of Singapore Singapore37580 Singapore School of Computer Science and Information Engineering Hefei University of Technology Hefei230009 China

This paper attacks the challenging problem of video retrieval by text. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described exclusively in the form of a natural-language sentence, with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is crucial. To that end, the two modalities need to be first encoded into real-valued vectors and then projected into a common space. In this paper we achieve this by proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Our novelty is two-fold. First, different from prior art that resorts to a specific single-level encoder, the proposed network performs multi-level encoding that represents the rich content of both modalities in a coarse-to-fine fashion. Second, different from a conventional common space learning algorithm which is either concept based or latent space based, we introduce hybrid space learning which combines the high performance of the latent space and the good interpretability of the concept space. Dual encoding is conceptually simple, practically effective and end-to-end trained with hybrid space learning. Extensive experiments on four challenging video datasets show the viability of the new method. Code is available at https://***/danieljf24/hybrid_space. Copyright © 2020, The Authors. All rights reserved.

关键词： Signal encoding

Combining Self-Supervised Learning and Adversarial Training Based Domain Adaptation for Speaker Verification

学校读者我要写书评

暂无评论

Combining Self-Supervised Learning and Adversarial Training ...

International Symposium on Chinese Spoken Language Processing

作者： Zhengyang Chen Shuai Wang Bing Han Yanmin Qian Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai Shenzhen Research Institute of Big Data Shenzhen Chinese University of Hong Kong Shenzhen

ISBN: (数字)9798331516826

ISBN: (纸本)9798331516833

Adapting an existing well-trained system to a new domain using only unlabeled data is a highly sought-after yet challenging task for speaker verification in real-world scenarios. In this paper, we study two different domain adaptation methods, the adversarial domain adaptation (ADA) and the self-supervised learning-based domain adaptation (SSDA). To facilitate the deployment of unsupervised adaptation methods in applications, we conduct a detailed analysis of the characteristics of both the ADA and SSDA adaptation strategies. Our findings indicate that the SSDA strategy's performance is highly influenced by the amount of target domain data, whereas the ADA strategy is relatively insensitive to data quantity. Furthermore, augmenting target domain data enhances SSDA system performance but diminishes ADA performance. To further enhance system performance, we explore the complementarity between ADA and SSDA. Our results demonstrate that ADA and SSDA complement each other. When both strategies are applied jointly, the best system achieves over 20.0% relative Equal Error Rate (EER) improvement on the Cnceleb evaluation set and over 35.0% relative average EER improvement on the SRE16 Cantonese and Tagalog evaluation set under domain mismatched conditions.

关键词： Training Error analysis System performance Self-supervised learning Speech processing

Conceptual Modeling for Advanced Application Domains 1

学校读者我要写书评

暂无评论

丛书名： Lecture Notes in Computer Science

1000年

作者： Shan Wang Katsumi Tanaka Shuigeng Zhou Tok-Wang Ling Jihong Guan Dong-qing Yang Fabio Grandi Eleni E. Mangina Il-Yeol Song Heinrich C. Mayr

Meteorology-Assisted Spatio-Temporal Graph Network for Uncivilized Urban Event Prediction

学校读者我要写书评

暂无评论

Meteorology-Assisted Spatio-Temporal Graph Network for Unciv...

IEEE International Conference on data Mining (ICDM)

作者： Yang Luo Zehao Gu Shiyang Zhou Yun Xiong Xiaofeng Gao Shanghai Jiao Tong University Shanghai China Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Fudan University Shanghai China Shanghai Key Laboratory of Data Science School of Computer Science Fudan University

Uncivilized urban events disrupt urban order and have a detrimental impact on daily life. Recognizing the significant implications of these events, urban managers strive to proactively prevent them by accurately predicting their future occurrence. However, existing methods overlook crucial contextual information within urban scenarios while mining spatio-temporal dependencies in single event series. Fortunately, we discovered a connection between meteorological conditions and uncivilized events. To leverage this relationship, we propose a novel approach named the Meteorology-Assisted Spatio-Temporal Graph Neural Network (MAST) which integrates meteorological information into the spatio-temporal dependency modeling for predicting urban uncivilized events. Additionally, our approach captures latent regularities in human behavior by explicitly modeling individuals’ psychological states based on meteorological information. We also adopt cross-view contrastive learning between urban regions to dynamically capture the informative components of meteorological information for precise prediction of urban uncivilized events. Experimental evaluations on a real-world dataset demonstrate the superiority of MAST over state-of-theart baselines in terms of predictive performance.

关键词：

Metric based on multi-order spaces for cross-modal retrieval

学校读者我要写书评

暂无评论

Metric based on multi-order spaces for cross-modal retrieval

IEEE International Conference on Multimedia and Expo (ICME)

作者： Liang Zhang Bingpeng Ma Guorong Li Qingming Huang Key Laboratory of Big Data Mining and Knowledge Management CAS China School of Computer and Control Engineering University of Chinese Academy of Sciences China Key Lab of Intell. Info. Process. Inst. of Comput. Tech. CAS China

ISBN: (纸本)9781509060689

This paper proposes a novel method for cross-modal retrieval. Different from vector (text)-to-vector (image) framework of the traditional cross-modal methods, we adopt a vector (text)-to-matrix (image) framework. We assume that compared with vectors, matrices can directly represent images and characterize the structure of feature space. Furthermore, we propose a Metric based on Multi-order spaces (MMs). Multi-order statistic features are used to represent images for enriching the semantic information, and metrics among the multi-spaces are jointly learned to measure the similarity between two different modalities. Specifically, there are three steps for MMs. First, we jointly use the bags of visual features (zero-order), mean (first-order) and covariance (second-order) to characterize each image. Second, considering that covariance matrices and vectors lie on a Riemannian manifold and an Euclidean space respectively, we embed multi-order spaces into their corresponding Hilbert spaces to reduce the heterogeneity among the original spaces. Finally, the similarity between two different modalities can be measured by learning multiple transformations from the different Hilbert spaces to a common subspace. The performance of the proposed method over the state-of-the-art has been demonstrated through the experiments on two public datasets.

关键词： Hilbert space Manifolds Correlation Covariance matrices Feature extraction Linear programming

Robust Cross-Domain Speaker Verification with Multi-Level Domain Adapters

学校读者我要写书评

暂无评论

Robust Cross-Domain Speaker Verification with Multi-Level Do...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Wen Huang Bing Han Shuai Wang Zhengyang Chen Yanmin Qian AI Institute Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Shanghai China Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China

Speaker verification encounters significant challenges when confronted with diverse domain data, often resulting in performance degradation due to domain mismatch. To enhance performance in cross-domain scenarios, we introduce the Domain Adapter, an adaptable module designed for specific domains. This module learns and integrates domain-specific information with speaker-related data, mitigating domain-related variations and promoting convergence of utterance embeddings from the same speaker across diverse domains. It offers configurability across multiple levels and is adaptable to various backbone architectures. Our proposed module substantially enhances cross-domain performance with minimal parameter increments while effectively generalizing to previously unseen domains. In our experiments, we present results on the 3D-Speaker dataset, which provides acoustically-relevant attributes crucial for domain categorization and the subsequent learning of domain information. The top-performing system integrated with domain adapters achieved 10.8%, 14.8%, and 21.1% EER improvements over the baseline across three 3D-Speaker dataset trials.

关键词：

LadRa-Net: Locally-aware dynamic re-read attention net for sentence semantic matching

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, Kun Lv, Guangyi Wu, Le Chen, Enhong Liu, Qi Wang, Meng Key Laboratory of Knowledge Engineering with Big Data School of Computer and Information Hefei University of Technology Anhui Hefei230029 China AI Lab at Lenovo Research Beijing100094 China University of Science and Technology of China Hefei230026 China

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, which is widely used in various natural language tasks, such as Natural Language Inference (NLI), Paraphrase Identification (PI), and so on. Much recent progress has been made in this area, especially attention-based methods and pre-trained language model based methods. However, most of these methods focus on all the important parts in sentences in a static way and only emphasize how important the words are to the query, inhibiting the ability of attention mechanism. In order to overcome this problem and boost the performance of attention mechanism, we propose a novel dynamic re-read attention, which can pay close attention to one small region of sentences at each step and re-read the important parts for better sentence representations. Based on this attention variation, we develop a novel Dynamic Re-read Network (DRr-Net) for sentence semantic matching. Moreover, selecting one small region in dynamic re-read attention seems insufficient for sentence semantics, and employing pre-trained language models as input encoders will introduce incomplete and fragile representation problems. To this end, we extend DRr-Net to Locally-Aware Dynamic Re-read Attention Net (LadRa-Net), in which local structure of sentences is employed to alleviate the shortcoming of Byte-Pair Encoding (BPE) in pre-trained language models and boost the performance of dynamic reread attention. Extensive experiments on two popular sentence semantic matching tasks demonstrate that DRr-Net can significantly improve the performance of sentence semantic matching. Meanwhile, LadRa-Net is able to achieve better performance by considering the local structures of sentences. In addition, it is exceedingly interesting that some discoveries in our experiments are consistent with some findings of psychological research. © 2021, CC BY.

关键词： Computational linguistics

Prototype and Instance Contrastive Learning for Unsupervised Domain Adaptation in Speaker Verification

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Huang, Wen Han, Bing Chen, Zhengyang Wang, Shuai Qian, Yanmin Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence AI Institute Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China

Speaker verification system trained on one domain usually suffers performance degradation when applied to another domain. To address this challenge, researchers commonly use feature distribution matching-based methods in unsupervised domain adaptation scenarios where some unlabeled target domain data is available. However, these methods often have limited performance improvement and lack generalization in various mismatch situations. In this paper, we propose Prototype and Instance Contrastive Learning (PICL), a novel method for unsupervised domain adaptation in speaker verification through dual-level contrastive learning. For prototype contrastive learning, we generate pseudo labels via clustering to create dynamically updated prototype representations, aligning instances with their corresponding class or cluster prototypes. For instance contrastive learning, we minimize the distance between different views or augmentations of the same instance, ensuring robust and invariant representations resilient to variations like noise. This dual-level approach provides both high-level and low-level supervision, leading to improved generalization and robustness of the speaker verification model. Unlike previous studies that only evaluated mismatches in one situation, we have conducted relevant explorations on various datasets and achieved state-of-the-art performance currently, which also proves the generalization of our method. Copyright © 2024, The Authors. All rights reserved.

关键词： Contrastive Learning

Instance credibility inference for few-shot learning

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Wang, Yikai Xu, Chengming Liu, Chen Zhang, Li Fu, Yanwei School of Data Science Fudan University China Department of Engineering Science University of Oxford United Kingdom MOE Frontiers Center for Brain Science Fudan University China Shanghai Key Lab of Intelligent Information Processing Fudan University China

Few-shot learning (FSL) aims to recognize new objects with extremely limited training data for each category. Previous efforts are made by either leveraging meta-learning paradigm or novel principles in data augmentation to alleviate this extremely data-scarce problem. In contrast, this paper presents a simple statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the distribution support of unlabeled instances for few-shot learning. Specifically, we first train a linear classifier with the labeled few-shot examples and use it to infer the pseudo-labels for the unlabeled data. To measure the credibility of each pseudo-labeled instance, we then propose to solve another linear regression hypothesis by increasing the sparsity of the incidental parameters and rank the pseudo-labeled instances with their sparsity degree. We select the most trustworthy pseudo-labeled instances alongside the labeled examples to re-train the linear classifier. This process is iterated until all the unlabeled samples are included in the expanded training set, i.e. the pseudo-label is converged for unlabeled data pool. Extensive experiments under two few-shot settings show that our simple approach can establish new state-of-the-arts on four widely used few-shot learning benchmark datasets including miniImageNet, tieredImageNet, CIFAR-FS, and CUB. Our code is available at: https://***/Yikai-Wang/ICI-FSL Copyright © 2020, The Authors. All rights reserved.

关键词： Machine learning