检索结果-内蒙古大学图书馆

arXiv 2023年

作者： Wu, Qitian Zhao, Wentao Yang, Chenxiao Zhang, Hengrui Nie, Fan Jiang, Haitian Bian, Yatao Yan, Junchi Dept. of Computer Science and Engineering MoE Key Lab of AI Shanghai Jiao Tong University China Dept. of Computer Science University of Illinois at Chicago United States Courant Institute New York University United States Tencent AI Lab China

Learning representations on large-sized graphs is a long-standing challenge due to the inter-dep.ndence nature involved in massive data points. Transformers, as an emerging class of foundation encoders for graph-structured data, have shown promising performance on small graphs due to its global attention capable of capturing all-pair influence beyond neighboring nodes. Even so, existing approaches tend to inherit the spirit of Transformers in language and vision tasks, and embrace complicated models by stacking deep multi-head attentions. In this paper, we critically demonstrate that even using a one-layer attention can bring up surprisingly competitive performance across node property prediction benchmarks where node numbers range from thousand-level to billion-level. This encourages us to rethink the design philosophy for Transformers on large graphs, where the quadratic global attention is a computation overhead hindering the scalability. We frame the proposed scheme as Simplified Graph Transformers (SGFormer), which is empowered by a simple attention model that can efficiently propagate information among arbitrary nodes at the minimal cost of one propagation layer and linear complexity w.r.t. node numbers. Moreover, SGFormer requires none of positional encodings, feature/graph pre-processing or extra loss. Empirically, SGFormer successfully scales to the web-scale graph ogbn-papers100M and yields up to 141x inference acceleration over SOTA Transformers on medium-sized graphs. Beyond current results, we believe the proposed methodology alone enlightens a new technical path of indep.ndent interest for building Transformers on large graphs. The codes are publicly available at https://***/qitianwu/SGFormer. © 2023, CC BY.

关键词： Distribution transformers

来源：评论

学校读者我要写书评

暂无评论

SirLLM: Streaming Infinite Retentive LLM

arXiv

引用

arXiv 2024年

作者： Yao, Yao Li, Zuchao Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China National Engineering Research Center for Multimedia Software School of Computer Science Wuhan University Wuhan430072 China

As Large Language Models (LLMs) become increasingly prevalent in various domains, their ability to process inputs of any length and maintain a degree of memory becomes essential. However, the one-off input of overly long texts is limited, as studies have shown that when input lengths exceed the LLMs' pre-trained text length, there is a dramatic decline in text generation capabilities. Moreover, simply extending the length of pre-training texts is impractical due to the difficulty in obtaining long text data and the substantial memory consumption costs this would entail for LLMs. Recent efforts have employed streaming inputs to alleviate the pressure of excessively long text inputs, but this approach can significantly impair the model's long-term memory capabilities. Motivated by this challenge, we introduce Streaming Infinite Retentive LLM (SirLLM), which allows LLMs to maintain longer memory during infinite-length dialogues without the need for fine-tuning. SirLLM utilizes the Token Entropy metric and a memory decay mechanism to filter key phrases, endowing LLMs with both long-lasting and flexible memory. We designed three distinct tasks and constructed three datasets to measure the effectiveness of SirLLM from various angles: (1) DailyDialog;(2) Grocery Shopping;(3) Rock-Paper-Scissors. Our experimental results robustly demonstrate that SirLLM can achieve stable and significant improvements across different LLMs and tasks, compellingly proving its effectiveness. When having a coversation, "A sir could forget himself," but SirLLM never does! Our code is publicly available at https://***/Zoeyyao27/SirLLM. © 2024, CC BY.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Leveraging Speech PTM, Text LLM, And Emotional TTS For Speech Emotion Recognition

Leveraging Speech PTM, Text LLM, And Emotional TTS For Speec...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Ziyang Ma Wen Wu Zhisheng Zheng Yiwei Guo Qian Chen Shiliang Zhang Xie Chen Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Shanghai Jiao Tong University Shanghai China Department of Engineering University of Cambridge Cambridge UK Speech Lab of DAMO Academy Alibaba Group China

In this paper, we explored how to boost speech emotion recognition (SER) with the state-of-the-art speech pre-trained model (PTM), data2vec, text generation technique, GPT-4, and speech synthesis technique, Azure TTS. First, we investigated the representation ability of different speech self-supervised pre-trained models, and we found that data2vec has a good representation ability on the SER task. Second, we employed a powerful large language model (LLM), GPT-4, and emotional text-to-speech (TTS) model, Azure TTS, to generate emotionally congruent text and speech. We carefully designed the text prompt and dataset construction, to obtain the synthetic emotional speech data with high quality. Third, we studied different ways of data augmentation to promote the SER task with synthetic speech, including random mixing, adversarial training, transfer learning, and curriculum learning. Experiments and ablation studies on the IEMOCAP dataset demonstrate the effectiveness of our method, compared with other data augmentation methods, and data augmentation with other synthetic data.

关键词：

来源：评论

学校读者我要写书评

暂无评论

EventBERT: Incorporating Event-Based Semantics for Natural Language Understanding 21

EventBERT: Incorporating Event-Based Semantics for Natural L...

引用

21st China National Conference on Computational Linguistics, CCL 2022

作者： Zou, Anni Zhang, Zhuosheng Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China

ISBN: (纸本)9783031183140

Natural language understanding tasks require a comprehensive understanding of natural language and further reasoning about it, on the basis of holistic information at different levels to gain comprehensive knowledge. In recent years, pre-trained language models (PrLMs) have shown impressive performance in natural language understanding. However, they rely mainly on extracting context-sensitive statistical patterns without explicitly modeling linguistic information, such as semantic relationships entailed in natural language. In this work, we propose EventBERT, an event-based semantic representation model that takes BERT as the backbone and refines with event-based structural semantics in terms of graph convolution networks. EventBERT benefits simultaneously from rich event-based structures embodied in the graph and contextual semantics learned in pre-trained model BERT. Experimental results on the GLUE benchmark show that the proposed model consistently outperforms the baseline model. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Convolution

来源：评论

学校读者我要写书评

暂无评论

Generalizable Audio Deepfake Detection via Latent Space Refinement and Augmentation

Generalizable Audio Deepfake Detection via Latent Space Refi...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Wen Huang Yanmei Gu Zhiming Wang Huijia Zhu Yanmin Qian AI Institute Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Shanghai China SJTU Paris Elite Institute of Technology Ant Group Shanghai China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Advances in speech synthesis technologies, like text-to-speech (TTS) and voice conversion (VC), have made detecting deepfake speech increasingly challenging. Spoofing countermeasures often struggle to generalize effectively, particularly when faced with unseen attacks. To address this, we propose a novel strategy that integrates Latent Space Refinement (LSR) and Latent Space Augmentation (LSA) to improve the generalization of deepfake detection systems. LSR introduces multiple learnable prototypes for the spoof class, refining the latent space to better capture the intricate variations within spoofed data. LSA further diversifies spoofed data representations by applying augmentation techniques directly in the latent space, enabling the model to learn a broader range of spoofing patterns. We evaluated our approach on four representative datasets, i.e. ASVspoof 2019 LA, ASVspoof 2021 LA and DF, and In-The-Wild. The results show that LSR and LSA perform well individually, and their integration achieves competitive results, matching or surpassing current state-of-the-art methods.

关键词： Deepfakes Refining Prototypes Data visualization Signal processing Robustness Data models Acoustics Text to speech

来源：评论

学校读者我要写书评

暂无评论

ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary

arXiv

引用

arXiv 2024年

作者： Li, Yutong Chen, Lu Liu, aiwei Yu, Kai Wen, Lijie Tsinghua University Beijing China X-LANCE Lab Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence SJTU AI Institute Shanghai Jiao Tong University Shanghai China Suzhou Laboratory Suzhou China

The literature review is an indispensable step in the research process. It provides the benefit of comprehending the research problem and understanding the current research situation while conducting a comparative analysis of prior works. However, literature summary is challenging and time consuming. The previous LLM-based studies on literature review mainly focused on the complete process, including literature retrieval, screening, and summarization. However, for the summarization step, simple CoT method often lacks the ability to provide extensive comparative summary. In this work, we firstly focus on the indep.ndent literature summarization step and introduce ChatCite1, an LLM agent with human workflow guidance for comparative literature summary. This agent, by mimicking the human workflow, first extracts key elements from relevant literature and then generates summaries using a Reflective Incremental Mechanism. In order to better evaluate the quality of the generated summaries, we devised a LLM-based automatic evaluation metric, G-Score, in refer to the human evaluation criteria. The ChatCite agent outperformed other models in various dimensions in the experiments. The literature summaries generated by ChatCite can also be directly used for drafting literature *** Codes 68T50 © 2024, CC BY.

关键词： Quality control

来源：评论

学校读者我要写书评

暂无评论

Joint Discriminator and Transfer Based Fast Domain Adaptation For End-To-End Speech Recognition

Joint Discriminator and Transfer Based Fast Domain Adaptatio...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Hang Shao Tian Tan Wei Wang Xun Gong Yanmin Qian Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Shanghai Jiao Tong University Shanghai China AISpeech Ltd Suzhou China

Adapting End-to-End (E2E) models to unseen domains is still a big challenge since training E2E models requires lots of paired audio and text training data. We propose a novel domain adaptation framework for the E2E model, which only uses the text of the target domain. Moreover, the proposed methods can keep the performance on the source domain intact while greatly improving the performance on the target domain. The proposed framework consists of two parts: the discriminator and the transfer which were optimized separately. Finally, optimized discriminator and transfer were combined and evaluated on two domain adaption tasks. In the experiments of adapting the English Librispeech to Gigaspeech, we obtained an average relative 11.6% and 11.8% on word error rate (WER) reduction for the target domain dev and test sets, respectively, while almost without WER degradation on the source domain. For the inhouse Chinese corpus aviation and TV, the character error rate (CER) of the source domain increased within 5%, while the CER on the target domain achieved around relative 85% and 42% improvement, respectively. In addition, our approach is also more effective in the mixed domain scenarios in the evaluation.

关键词： Degradation Training Adaptation models TV Error analysis Training data Speech recognition

来源：评论

学校读者我要写书评

暂无评论

Advanced Long-Content Speech Recognition with Factorized Neural Transducer

arXiv

引用

arXiv 2024年

作者： Gong, Xun Wu, Yu Li, Jinyu Liu, Shujie Zhao, Rui Chen, Xie Qian, Yanmin Auditory Cognition and Computational Acoustics Lab The Department of Computer Science and Engineering The MoE Key Laboratory of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai200240 China Microsoft Corporation United States

Long-content automatic speech recognition (ASR) has obtained increasing interest in recent years, as it captures the relationship among consecutive historical utterances while decoding the current utterance. In this paper, we propose two novel approaches, which integrate long-content information into the factorized neural transducer (FNT) based architecture in both non-streaming (referred to as LongFNT) and streaming (referred to as SLongFNT) scenarios. We first investigate whether long-content transcriptions can improve the vanilla conformer transducer (C-T) models. Our experiments indicate that the vanilla C-T models do not exhibit improved performance when utilizing long-content transcriptions, possibly due to the predictor network of C-T models not functioning as a pure language model. Instead, FNT shows its potential in utilizing long-content information, where we propose the LongFNT model and explore the impact of long-content information in both text (LongFNT-Text) and speech (LongFNT-Speech). The proposed LongFNT-Text and LongFNT-Speech models further complement each other to achieve better performance, with transcription history proving more valuable to the model. The effectiveness of our LongFNT approach is evaluated on LibriSpeech and GigaSpeech corpora, and obtains relative 19% and 12% word error rate reduction, respectively. Furthermore, we extend the LongFNT model to the streaming scenario, which is named SLongFNT, consisting of SLongFNT-Text and SLongFNT-Speech approaches to utilize long-content text and speech information. Experiments show that the proposed SLongFNT model achieves relative 26% and 17% WER reduction on LibriSpeech and GigaSpeech respectively while keeping a good latency, compared to the FNT baseline. Overall, our proposed LongFNT and SLongFNT highlight the significance of considering long-content speech and transcription knowledge for improving both non-streaming and streaming speech recognition systems. Copyright © 2024, The Authors.

关键词： Transducers

来源：评论

学校读者我要写书评

暂无评论

Front-End Adapter: Adapting Front-End Input of Speech Based Self-Supervised Learning for Speech Recognition

Front-End Adapter: Adapting Front-End Input of Speech Based ...

引用

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Xie Chen Ziyang Ma Changli Tang Yujin Wang Zhisheng Zheng Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute X-Lance Lab Shanghai Jiao Tong University Shanghai China Department of Electronic Engineering Tsinghua University Beijing China

Recent years have witnessed a boom in self-supervised learning (SSL) in various areas including speech processing. Speech based SSL models present promising performance in a range of speech related tasks. However, the training of SSL models is computationally expensive and a common practice is to fine-tune a released SSL model on the specific task. It is essential to use consistent front-end input during pre-training and fine-tuning. This consistency may introduce potential issues when the optimal front-end is not the same as that used in pre-training. In this paper, we propose a simple but effective front-end adapter to address this front-end discrepancy. By minimizing the distance between the outputs of different front-ends, the filterbank feature (Fbank) can be compatible with SSL models which are pre-trained with waveform. The experiment results demonstrate the effectiveness of our proposed front-end adapter on several popular SSL models for the speech recognition task.

关键词： Training Adaptation models Computational modeling Filter banks Speech recognition Self-supervised learning Signal processing

来源：评论

学校读者我要写书评

暂无评论

Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmark

Ultra-High Resolution Segmentation with Ultra-Rich Context: ...

引用

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Deyi Ji Feng Zhao Hongtao Lu Mingyuan Tao Jieping Ye University of Science and Technology of China Alibaba Group Department of Computer Science and Engineering Shanghai Jiao Tong University MOE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University

With the increasing interest and rapid development of methods for Ultra-High Resolution (UHR) segmentation, a large-scale benchmark covering a wide range of scenes with full fine-grained dense annotations is urgently needed to facilitate the field. To this end, the URUR dataset is introduced, in the meaning of Ultra-High Resolution dataset with Ultra-Rich Context. As the name suggests, URUR contains amounts of images with high enough resolution (3,008 images of size 5,120 × 5,120), a wide range of complex scenes (from 63 cities), rich-enough context (1 million instances with 8 categories) and fine-grained annotations (about 80 billion manually annotated pixels), which is far superior to all the existing UHR datasets including DeepGlobe, Inria Aerial, UDD, etc.. Moreover, we also propose WSDNet, a more efficient and effective framework for UHR segmentation especially with ultra-rich context. Specifically, multi-level Discrete Wavelet Transform (DWT) is naturally integrated to release computation burden while preserve more spatial details, along with a Wavelet Smooth Loss (WSL) to reconstruct original structured context and texture with a smooth constrain. Experiments on several UHR datasets demonstrate its state-of-the-art performance. The dataset is available at https://***/jankyee/URUR.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：