检索结果-内蒙古大学图书馆

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Leying Zhang Wangyou Zhang Zhengyang Chen Yanmin Qian AI Institute Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Shanghai China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

The acoustic background plays a crucial role in natural conversation. It provides context and helps listeners understand the environment, but a strong background makes it difficult for listeners to understand spoken words. The appropriate handling of these backgrounds is situation-dep.ndent: Although it may be necessary to remove background to ensure speech clarity, preserving the background is sometimes crucial to maintaining the contextual integrity of the speech. Despite recent advancements in zero-shot Text-to-Speech technologies, current systems often struggle with speech prompts containing backgrounds. To address these challenges, we propose a Controllable Masked Speech Prediction strategy coupled with a dual-speaker encoder, utilizing a task-related control signal to guide the prediction of dual background removal and preservation targets. Experimental results demonstrate that our approach enables precise control over the removal or preservation of background across various acoustic conditions and exhibits strong generalization capabilities in unseen scenarios.

关键词： Oral communication Predictive models Signal processing Acoustics Text to speech Speech processing

来源：评论

学校读者我要写书评

暂无评论

Improving Speech Enhancement Using Audio Tagging Knowledge From Pre-Trained Representations and Multi-Task Learning

Improving Speech Enhancement Using Audio Tagging Knowledge F...

引用

2023 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2023

作者： Lin, Shaoxiong Zhang, Chao Qian, Yanmin Tsinghua University Department of Electronic Engineering Beijing China Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence Ai Institute Department of Computer Science and Engineering Shanghai China Suzhou Institute of Artificial Intelligence Shanghai Jiao Tong University Suzhou215000 China

ISBN: (纸本)9798350306897

In deep-learning-based speech enhancement (SE), an audio-knowledge-ignorant approach is often used, which estimates a denoising model to transform the noisy input speech into clean output speech without understanding the audio events that constitute the background noises. In this paper, an audio-knowledge-aware approach is proposed to improve SE, which explicitly leverages the knowledge from audio taggings to understand the background noises. Based on the recent progress in audio pattern analysis, the audio tagging knowledge is obtained using either additional input representations extracted by pre-trained audio tagging models, or from multi-task learning with extra audio event classification or regression tasks. Experimental results based on the DNS-2020 dataset and the pre-trained Wavegram-Logmel-CNN audio tagging model show that the proposed approach leads to considerable improvements in the STOI, SDR, and SI-SNR metrics. © 2023 IEEE.

关键词： Speech enhancement

来源：评论

学校读者我要写书评

暂无评论

Exploiting Persistent CPU Cache for Scalable Persistent Hash Index

Exploiting Persistent CPU Cache for Scalable Persistent Hash...

引用

International Conference on Data engineering

作者： Bowen Zhang Shengan Zheng Liangxu Nie Zhenlin Qi Linpeng Huang Hong Mei Department of Computer Science and Engineering Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

ISBN: (数字)9798350317152

ISBN: (纸本)9798350317169

Byte-addressable persistent memory (PM) has been widely studied in the past few years. Recently, the emerging eADR technology further incorporates CPU cache into the persistence domain. The persistent CPU cache is promising to optimize the write performance of PM-based storage systems and facilitate the design of concurrent crash-consistent data structures. In this paper, we propose Spash, a highly scalable persistent hash index for PM systems with persistent CPU cache. Spash fully exploits the benefits of persistent CPU cache to implement a durable linearizable index with low PM access overhead and high concurrency. Spash employs a fine-grained extendible hash architecture and a metadata-free segment design to minimize the number of PM accesses. Moreover, Spash adopts adaptive in-place updates and compacted-flush insertions, which dramatically conserve scarce PM write bandwidth by absorbing a large amount of PM write in the persistent CPU cache. Furthermore, Spash proposes a two-phase concurrency protocol and a collaborative staged doubling mechanism, which leverage the persistent CPU cache and hardware transactional memory to achieve lock-free concurrency and durable linearizability. Spash outperforms the other state-of-the-art persistent hash indexes in YCSB workloads by up to 19.6×.

关键词： Concurrent computing Protocols Collaboration Bandwidth Data structures Data engineering Hardware

来源：评论

学校读者我要写书评

暂无评论

Decoupling Classification and Localization of CLIP

Decoupling Classification and Localization of CLIP

引用

IEEE International Conference on Multimedia and Expo Workshops (ICMEW)

作者： Muyang Yi Zhaozhi Xie Yuwen Yang Chang Liu Yue Ding Hongtao Lu Department of Computer Science and Engineering Shanghai Jiao Tong University MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

ISBN: (数字)9798350379815

ISBN: (纸本)9798350379822

Contrastive language-image pre-training, known for its effectiveness in zero-shot retrieval and classification, faces limitations in localization ability, hindering its broader application in diverse vision tasks such as object detection and segmentation. Classifying and locating objects involves gathering global features and local features, respectively. A conflict exists between these abilities when the image encoder updates the weights in pre-training. To address this issue, we propose CLIP-WIRE, which decouples the global and local representation learning by incor-porating registers into the image encoder. CLIP-WIRE allows the model to store global features in the register tokens and focus on local features through patch tokens, thereby jointly improving the classification and localization abilities. Furthermore, a segmentor is designed to validate the enhancement in semantic segmentation tasks. Comprehensive experiments and visualizations prove the significance of the proposed approach.

关键词： Location awareness Representation learning Visualization Semantic segmentation Conferences Object detection Registers

来源：评论

学校读者我要写书评

暂无评论

Towards Practical Edge Inference Attacks Against Graph Neural Networks

Towards Practical Edge Inference Attacks Against Graph Neura...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Kailai Li Jiawei Sun Ruoxin Chen Wei Ding Kexue Yu Jie Li Chentao Wu Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

Graph Neural Networks (GNNs) have demonstrated superior performance in numerous real-world applications. Despite their success, recent studies have shown that GNNs are vulnerable under edge inference attacks aimed to infer the connectivity of a given pair of nodes. However, existing methods primarily focus on the scenario when properties of target nodes are revealed. In this paper, we propose an edge inference attack in a more realistic and practical setting. In our threat model, the adversary cannot obtain properties of target nodes but can inject a single probing node and query the target GNN for its prediction. By connecting the probing and target nodes, the adversary can infer the connectivity of the target node pair based on the prediction of the probing node. Extensive experiments show that our attack performs comparably to ones that require properties of target nodes. And when given such auxiliary knowledge, our attack outperforms state-of-the-art methods.

关键词： Threat modeling Closed box Signal processing Graph neural networks Acoustics Speech processing

来源：评论

学校读者我要写书评

暂无评论

Band-Wise Front-End Distortion Suppression for Robust Speech Recognition

Band-Wise Front-End Distortion Suppression for Robust Speech...

引用

International Symposium on Chinese Spoken Language Processing

作者： Siyi Zhao Wei Wang Yanmin Qian Department of Computer Science and Engineering Auditory Cognition and Computational Acoustic Lab MoE Key Lab of Aritificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai

ISBN: (数字)9798331516826

ISBN: (纸本)9798331516833

Advancements in deep learning techniques have significantly improved automatic speech recognition (ASR). However, improving robustness to acoustic interferences, such as background noise and reverberation, remains crucial for accurate speech recognition in noisy environments. Traditional speech enhancement (SE) modules used as pre-processing front-ends often fail to integrate well with ASR models due to intelligibility distortions introduced by the SE module. Inspired by the observation adding technique and band-wise network design of recent SE models, we propose a novel adaptive band-wise observation adding (BOA) module for front-end distortion suppression. The lightweight BOA network learns time- and frequencyvariant OA factors, adaptively balancing the contributions of noisy and enhanced speech signals for diverse noisy conditions. Experiments conducted on both simulated and real-world test sets demonstrate that the BOA module produces more effective dynamic OA factors under various conditions compared to fixed OA factors, leading to improved ASR performance.

关键词： Deep learning Adaptation models Time-frequency analysis Acoustic distortion Speech enhancement Robustness Frequency diversity Noise robustness Noise measurement Reverberation

来源：评论

学校读者我要写书评

暂无评论

Insights from Hyperparameter Scaling of Online Speech Separation

Insights from Hyperparameter Scaling of Online Speech Separa...

引用

International Symposium on Chinese Spoken Language Processing

作者： Xin Zhou Wangyou Zhang Chenda Li Yanmin Qian Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai

ISBN: (数字)9798331516826

ISBN: (纸本)9798331516833

With the rapid development of deep learning, a large number of models with excellent performance for speech separation tasks have emerged in the literature. Despite their impressive performance, these models usually cannot be used for online speech separation tasks due to high computational and memory costs. In recent years, some approaches tailored for online speech separation have been developed to achieve state-of-the-art performance while keeping the model lightweight. However, it is often unclear how these models can scale with their architecture-specific hyperparameters as they are under-explored in the literature. To fill this gap, this paper investigates the effect of hyper-parameter scaling on the model performance, number of parameters, and computational cost based on two representative online speech separation models, BSRNN and SkiM. Experimental results on the Libri2Mix dataset reveal the relationship between model performance and composite hyperparameter configurations. We also derive the patterns of the number of parameters and computational costs varying with hyperparameters. Based on the results of the analysis, we give suggestions for scaling the two models and summarize two common trends.

关键词： Degradation Deep learning Analytical models Costs Computational modeling Market research Computational efficiency Speech processing Guidelines

来源：评论

学校读者我要写书评

暂无评论

ConMamba: A Convolution-Augmented Mamba Encoder Model for Efficient End-to-End ASR Systems

ConMamba: A Convolution-Augmented Mamba Encoder Model for Ef...

引用

International Symposium on Chinese Spoken Language Processing

作者： Haoxiang Hou Xun Gong Yanmin Qian Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai

ISBN: (数字)9798331516826

ISBN: (纸本)9798331516833

End-to-End Automatic Speech Recognition (ASR) models, such as Conformer, excel in accuracy but face limitations in computational complexity and positional awareness, hindering their use in real-time or resource-constrained settings. State Space Models (SSMs), particularly the Mamba model with its time-varying mechanism, offer a more efficient alternative. We propose ConMamba, a Convolution-Augmented Mamba En-coder model, which replaces the Conformer's Multi-Head Self-Attention with Mamba layers and adds convolutional layers for capturing both local and global features. Experiments on the LibriSpeech dataset show ConMamba matches traditional Conformer performance on short speech segments and outperforms them on longer ones, enhancing robustness and efficiency for practical ASR applications.

关键词： Accuracy Computational modeling Face recognition Speech enhancement Transformers Robustness Real-time systems Computational complexity Automatic speech recognition

来源：评论

学校读者我要写书评

暂无评论

label-Aware Auxiliary Learning for Dialogue State Tracking

Label-Aware Auxiliary Learning for Dialogue State Tracking

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Yuncong Liu Lu Chen Kai Yu Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence X-LANCE Lab SJTU AI Institute Shanghai Jiao Tong University Shanghai China

Dialogue State Tracking (DST) is an essential part of task-oriented dialogue systems. Many existing methods try to utilize external dialogue datasets to improve the performance of DST models. Instead of previous methods, in this paper, we propose label-Aware Auxiliary Learning for DST (LAL-DST) which focuses on exploiting the abundant internal information of the target DST dataset to improve the performance. We design label-aware auxiliary tasks, in which we apply noising functions to either the dialogue history or the belief state label and take the concatenation of them as input. The goal of each task is to restore the corrupted context. During the training process, we first further train the large pre-trained language model on the auxiliary tasks, then fine-tune it on DST. Through the experimental results, we empirically show the effect of LAL-DST by the performance improvements it brings to MultiWOZ2.0 and WOZ.

关键词：

来源：评论

学校读者我要写书评

暂无评论

AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures

arXiv

引用

arXiv 2024年

作者： Zhang, Situo Wang, Hankun Ma, Da Zhu, Zichen Chen, Lu Lan, Kunyao Yu, Kai X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China

Speculative Decoding (SD) is a popular lossless technique for accelerating the inference of Large Language Models (LLMs). We show that the decoding speed of SD frameworks with static draft structures can be significantly improved by incorporating context-aware adaptive draft structures. However, current studies on adaptive draft structures are limited by their performance, modeling approaches, and applicability. In this paper, we introduce AdaEAGLE, the first SD framework that explicitly models adaptive draft structures. AdaEAGLE leverages the Lightweight Draft Length Predictor (LDLP) module to explicitly predict the optimal number of draft tokens during inference to guide the draft model. It achieves comparable speedup results without manual thresholds and allows for deeper, more specialized optimizations. Moreover, together with threshold-based strategies, AdaEAGLE achieves a 1.62× speedup over the vanilla AR decoding and outperforms fixed-length SotA baseline while maintaining output quality. © 2024, CC BY.

关键词： Decoding

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：