检索结果-内蒙古大学图书馆

Long-Term Invariant Local Features via Implicit Cross-Domain Correspondences

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Pataki, Zador Altillawi, Mohammad Kanakis, Menelaos Pautrat, Rémi Shen, Fengyi Liu, Ziyuan Van Gool, Luc Pollefeys, Marc The Computer Vision and Geometry Lab Department of Computer Science ETH Zurich Switzerland The Computer Vision Center CVC-Barcelona The Intelligent Robotics Cloud Technology lab of Huawei-Munich Germany The Computer Vision Lab Department Electrical Engineering ETH Zurich Switzerland The Intelligent Robotics Cloud Technology lab of Huawei-Munich Germany The Intelligent Robotics Cloud Technology lab of Huawei-Munich Germany The Center for Processing Speech and Images KU Leuven The Computer Vision Lab ETH Zurich Switzerland

Modern learning-based visual feature extraction networks perform well in intra-domain localization, however, their performance significantly declines when image pairs are captured across long-term visual domain variations, such as different seasonal and daytime variations. In this paper, our first contribution is a benchmark to investigate the performance impact of long-term variations on visual localization. We conduct a thorough analysis of the performance of current state-of-the-art feature extraction networks under various domain changes and find a significant performance gap between intra- and cross-domain localization. We investigate different methods to close this gap by improving the supervision of modern feature extractor networks. We propose a novel data-centric method, Implicit Cross-Domain Correspondences (iCDC). iCDC represents the same environment with multiple Neural Radiance Fields, each fitting the scene under individual visual domains. It utilizes the underlying 3D representations to generate accurate correspondences across different long-term visual conditions. Our proposed method enhances cross-domain localization performance, significantly reducing the performance gap. When evaluated on popular long-term localization benchmarks, our trained networks consistently outperform existing methods. This work serves as a substantial stride toward more robust visual localization pipelines for long-term deployments, and opens up research avenues in the development of long-term invariant descriptors. Copyright © 2023, The Authors. All rights reserved.

关键词： Feature extraction

SAV-SE: Scene-aware Audio-Visual speech Enhancement with Selective State Space Model

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qian, Xinyuan Gao, Jiaran Zhang, Yaodan Zhang, Qiquan Liu, Hexin Garcia, Leibny Paola Li, Haizhou The School of Computer and Communication Engineering University of Science and Technology Beijing Beijing100083 China The School of Electrical Engineering and Telecommunications The University of New South Wales Sydney2052 Australia The College of Computing and Data Science Nanyang Technological University Singapore The Center for Language and Speech Processing Johns Hopkins University United States The Guangdong Provincial Key Laboratory of Big Data Computing The Chinese University of Hong Kong Shenzhen518172 China Shenzhen Research Institute of Big data Shenzhen51872 China

speech enhancement plays an essential role in various applications, and the integration of visual information has been demonstrated to bring substantial advantages. However, the majority of current research concentrates on the examination of facial and lip movements, which can be compromised or entirely inaccessible in scenarios where occlusions occur or when the camera view is distant. Whereas contextual visual cues from the surrounding environment have been overlooked: for example, when we see a dog bark, our brain has the innate ability to discern and filter out the barking noise. To this end, in this paper, we introduce a novel task, i.e. Scene-aware AudioVisual speech Enhancement (SAV-SE). To our best knowledge, this is the first proposal to use rich contextual information from synchronized video as auxiliary cues to indicate the type of noise, which eventually improves the speech enhancement performance. Specifically, we propose the VC-S2E method, which incorporates the Conformer and Mamba modules for their complementary strengths. Extensive experiments are conducted on public MUSIC, AVspeech and AudioSet datasets, where the results demonstrate the superiority of VC-S2E over other competitive methods. We will make the source code publicly available. Project demo page: https://***/ © 2024, CC BY.

关键词： speech enhancement

LocalViT: Analyzing Locality in Vision Transformers

学校读者我要写书评

暂无评论

LocalViT: Analyzing Locality in Vision Transformers

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Yawei Li Kai Zhang Jiezhang Cao Radu Timofte Michele Magno Luca Benini Luc Van Goo Computer Vision Lab D-ITET ETH Zurich Switzerland Center for Artificial Intelligence and Data Science (CAIDAS) University of Wurzburg Germany Center for Project-Based Learning D-ITET ETH Zurich Switzerland Integrated Systems Laboratory D-ITET ETH Zurich Switzerland Department of Electrical Electronic and Information Engineering University of Bologna Italy Processing Speech and Images (PSI) KU Leuven Belgium

The aim of this paper is to study the influence of locality mechanisms in vision transformers. Transformers originated from machine translation and are particularly good at modelling long-range dependencies within a long sequence. Although the global interaction between the token embeddings could be well modelled by the self-attention mechanism of transformers, what is lacking is a locality mechanism for infor-mation exchange within a local region. In this paper, locality mechanism is systematically investigated by carefully designed controlled experiments. We add locality to vision transformers into the feed-forward network. This seemingly simple solution is inspired by the comparison between feed-forward networks and inverted residual blocks. The importance of locality mechanisms is validated in two ways: 1) A wide range of design choices (activation function, layer placement, expansion ratio) are available for incorporating locality mechanisms and proper choices can lead to a performance gain over the baseline, and 2) The same locality mechanism is successfully applied to vision transformers with different architecture designs, which shows the generalization of the locality concept. For ImageNet2012 classification, the locality-enhanced transformers outperform the baselines Swin-T [1], DeiT-T [2] and PVT-T [3] by 1.0%, 2.6 % and 3.1 % with a negligible increase in the number of parameters and computational effort. Code is available at https://***/ofsoundof/LocalViT.

关键词：

ADMM-DAD NET: A DEEP UNFOLDING NETWORK FOR ANALYSIS COMPRESSED SENSING

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Kouni, Vasiliki Paraskevopoulos, Georgios Rauhut, Holger Alexandropoulos, George C. Kouni, Vicky Dep. of Informatics and Telecommunications National & Kapodistrian University of Athens Greece Mathematics of Information Processing RWTH Aachen University Germany School of Electrical & Computer Engineering National Technical University of Athens Greece Institute for Language & Speech Processing Athena Research Center Athens Greece

In this paper, we propose a new deep unfolding neural network based on the ADMM algorithm for analysis Compressed Sensing. The proposed network jointly learns a redundant analysis operator for sparsification and reconstructs the signal of interest. We compare our proposed network with a state-of-the-art unfolded ISTA decoder, that also learns an orthogonal sparsifier. Moreover, we consider not only image, but also speech datasets as test examples. Computational experiments demonstrate that our proposed network outperforms the state-of-the-art deep unfolding network, consistently for both real-world image and speech datasets. © 2021, CC BY.

关键词： Compressed sensing

Revisiting multi-Task learning in the deep learning era

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Vandenhende, Simon Georgoulis, Stamatios Proesmans, Marc Dai, Dengxin Gool, Luc Van Center for Processing Speech and Images Department Electrical Engineering Ku Leuven Computer Vision Lab Department Electrical Engineering Eth Zurich Center for Processing Speech and Images Ku Leuven and the Computer Vision Lab Ethz

Despite the recent progress in deep learning, most approaches still go for a silo-like solution, focusing on learning each task in isolation: Training a separate neural network for each individual task. Many real-world problems, however, call for a multi-modal approach and, therefore, for multi-Tasking models. Multi-Task learning (MTL) aims to leverage useful information across tasks to improve the generalization capability of a model. In this survey, we provide a well-rounded view on state-of-The-Art MTL techniques within the context of deep neural networks. Our contributions concern the following. First, we consider MTL from a network architecture point-of-view. We include an extensive overview and discuss the advantages/disadvantages of recent popular MTL models. Second, we examine various optimization methods to tackle the joint learning of multiple tasks. We summarize the qualitative elements of these works and explore their commonalities and differences. Finally, we provide an extensive experimental evaluation across a variety of datasets to examine the pros and cons of different methods, including both architectural and optimization based strategies. Copyright © 2020, The Authors. All rights reserved.

关键词： Deep neural networks

Discovering Phonetic Inventories with Crosslingual Automatic speech Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Żelasko, Piotr Feng, Siyuan Velázquez, Laureano Moro Abavisani, Ali Bhati, Saurabhchand Scharenborg, Odette Hasegawa-Johnson, Mark Dehak, Najim Center of Language and Speech Processing The Johns Hopkins University 3400 North Charles Street BaltimoreMD21218 United States Human Language Technology Center of Excellence The Johns Hopkins University 810 Wyman Park Drive BaltimoreMD21218 United States Multimedia Computing Group Delft University of Technology Van Mourik Broekmanweg 6 Delft2628 XE Netherlands Department of Electrical and Computer Engineering University of Illinois 405 N Mathews UrbanaIL61801 United States

The high cost of data acquisition makes Automatic speech Recognition (ASR) model training problematic for most existing languages, including languages that do not even have a written script, or for which the phone inventories remain unknown. Past works explored multilingual training, transfer learning, as well as zero-shot learning in order to build ASR systems for these low-resource languages. While it has been shown that the pooling of resources from multiple languages is helpful, we have not yet seen a successful application of an ASR model to a language unseen during training. A crucial step in the adaptation of ASR from seen to unseen languages is the creation of the phone inventory of the unseen language. The ultimate goal of our work is to build the phone inventory of a language unseen during training in an unsupervised way without any knowledge about the language. In this paper, we 1) investigate the influence of different factors (i.e., model architecture, phonotactic model, type of speech representation) on phone recognition in an unknown language;2) provide an analysis of which phones transfer well across languages and which do not in order to understand the limitations of and areas for further improvement for automatic phone inventory creation;and 3) present different methods to build a phone inventory of an unseen language in an unsupervised way. To that end, we conducted mono-, multi-, and crosslingual experiments on a set of 13 phonetically diverse languages and several in-depth analyses. We found a number of universal phone tokens (IPA symbols) that are well-recognized cross-linguistically. Through a detailed analysis of results, we conclude that unique sounds, similar sounds, and tone languages remain a major challenge for phonetic inventory discovery. © 2022, CC BY-NC-ND.

关键词： Telephone sets

An asynchronous wfst-based decoder for automatic speech recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Lv, Hang Chen, Zhehuai Xu, Hainan Povey, Daniel Xie, Lei Khudanpur, Sanjeev School of Computer Science Northwestern Polytechnical University Xi'an China Center of Language and Speech Processing United States Human Language Technology Center of Excellence Johns Hopkins University BaltimoreMD United States Xiaomi Corporation Beijing China SpeechLab Department of Computer Science and Engineering Shanghai Jiao Tong University China

We introduce asynchronous dynamic decoder, which adopts an efficient A∗ algorithm to incorporate big language models in the onepass decoding for large vocabulary continuous speech recognition. Unlike standard one-pass decoding with on-the-fly composition decoder which might induce a significant computation overhead, the asynchronous dynamic decoder has a novel design where it has two fronts, with one performing "exploration" and the other "backfill". The computation of the two fronts alternates in the decoding process, resulting in more effective pruning than the standard one-pass decoding with an on-the-fly composition decoder. Experiments show that the proposed decoder works notably faster than the standard onepass decoding with on-the-fly composition decoder, while the acceleration will be more obvious with the increment of data complexity. Copyright © 2021, The Authors. All rights reserved.

关键词： Decoding

δ-Calculus:A New Approach to Quantifying Location Privacy

学校读者我要写书评

暂无评论

computers, Materials & Continua 2020年第6期63卷 1323-1342页

作者： Lihua Yin Ran Li Jingquan Ding Xiao Li Yunchuan Guo Huibing Zhang Ang Li Cyberspace Institute of Advanced Technology Guangzhou UniversityGuangzhou510006China Institute of Information Engineering CASBeijing100093China Xinjiang Technical Institute of Physics&Chemistry Chinese Academy ScienceUrumqi830011China Key laboratory of Speech Language Information Processing of Xinjiang Urumqi830046China University of Chinese Academy of Sciences Beijing100049China Guilin University of Electronic Technology Guilin541004China Department of Electrical and Computer Engineering Duke UniversityDurham27708USA

With the rapid development of mobile wireless Internet and high-precision localization devices,location-based services(LBS)bring more convenience for people over recent *** LBS,if the original location data are directly provided,serious privacy problems *** a response to these problems,a large number of location-privacy protection mechanisms(LPPMs)(including formal LPPMs,FLPPMs,etc.)and their evaluation metrics have been proposed to prevent personal location information from being leakage and quantify privacy ***,existing schemes independently consider FLPPMs and evaluation metrics,without synergizing them into a unifying *** this paper,a unified model is proposed to synergize FLPPMs and evaluation *** detail,the probabilistic process calculus(calledδ-calculus)is proposed to characterize obfuscation schemes(which is a LPPM)and integrateα-entropy toδ-calculus to evaluate its privacy ***,we use two calculus moving and probabilistic choice to model nodes’mobility and compute its probability distribution of nodes’locations,and a renaming function to model privacy *** formally defining the attacker’s ability and extending relative entropy,an evaluation algorithm is proposed to quantify the leakage of location ***,a series of examples are designed to demonstrate the efficiency of our proposed approach.

关键词： Location privacy δ-calculus relative entropy

Zero Resource Speaking Rate Estimation from Change Point Detection of Syllable-like Units 44

学校读者我要写书评

暂无评论

Zero Resource Speaking Rate Estimation from Change Point Det...

44th IEEE International Conference on Acoustics, speech, and Signal processing, ICASSP 2019

作者： Nayak, Shekhar Bhati, Saurabhchand Rama Murty, K. Sri Department of Electrical Engineering Indian Institute of Technology Hyderabad India Center for Language and Speech Processing Johns Hopkins University United States

ISBN: (纸本)9781479981311

Speaking rate is an important attribute of the speech signal which plays a crucial role in the performance of automatic speech processing systems. In this paper, we propose to estimate the speaking rate by segmenting the speech into syllable-like units using end point detection algorithms which do not require any training and fine-tuning. Also, there are no predefined constraints on the expected number of syllabic segments. The acoustic subword units are obtained only from speech signal to estimate the speaking rate without any requirement of transcriptions or phonetic knowledge of the speech data. A recent theta-rate oscillator based syllabification algorithm is also employed for speaking rate estimation. The performance is evaluated on TIMIT corpus and spontaneous speech from Switchboard corpus. The correlation results are comparable to recent algorithms which are trained with specific training set and/or make use of the available transcriptions. © 2019 IEEE.

关键词： Point contacts