检索结果-内蒙古大学图书馆

arXiv 2019年

作者： Luo, Liangchen Xiong, Yuanhao Liu, Yan Sun, Xu MOE Key Lab of Computational Linguistics School of EECS Peking University College of Information Science and Electronic Engineering Zhejiang University Department of Computer Science University of Southern California Center for Data Science Beijing Institute of Big Data Research Peking University

Adaptive optimization methods such as ADAGRAD, RMSPROP and ADAM have been proposed to achieve a rapid training process with an element-wise scaling term on learning rates. Though prevailing, they are observed to generalize poorly compared with SGD or even fail to converge due to unstable and extreme learning rates. Recent work has put forward some algorithms such as AMSGRAD to tackle this issue but they failed to achieve considerable improvement over existing methods. In our paper, we demonstrate that extreme learning rates can lead to poor performance. We provide new variants of ADAM and AMSGRAD, called ADABOUND and AMSBOUND respectively, which employ dynamic bounds on learning rates to achieve a gradual and smooth transition from adaptive methods to SGD and give a theoretical proof of convergence. We further conduct experiments on various popular tasks and models, which is often insufficient in previous work. Experimental results show that new variants can eliminate the generalization gap between adaptive methods and SGD and maintain higher learning speed early in training at the same time. Moreover, they can bring significant improvement over their prototypes, especially on complex deep networks. The implementation of the algorithm can be found at https://***/Luolc/AdaBound. Copyright © 2019, The Authors. All rights reserved.

关键词： Gradient methods

来源：评论

学校读者我要写书评

暂无评论

Chinese Emergency Event Extraction Based on Contrastive Learning with Event Semantic Features 19

Chinese Emergency Event Extraction Based on Contrastive Lear...

引用

19th International Conference on Natural Computation, Fuzzy Systems and knowledge Discovery, ICNC-FSKD 2023

作者： Sheng, Xinyi Gu, Jinguang Yan, Youcheng Xu, Fangfang Wuhan University of Science and Technology College of Computer Science and Technology Wuhan430065 China Institute of Big Data Science and Engineering Wuhan University of Science and Technology Wuhan430065 China The Key Lab. of Rich-Media Knowledge Org. and Serv. of Digit. Publ. Content Inst. of Sci. and Tech. Info. of China Beijing100038 China

ISBN: (纸本)9798350304398

Extracting emergency events from a large amount of unstructured information is essential for improving early warning and emergency response. Existing event extraction methods for specialist fields often rely on well-designed syntactic dependencies or external knowledge graphs, which can provide high accuracy in their specialist fields. However, these approaches may have poor generalization capabilities and are difficult to apply directly to the emergency field. In addition, most existing methods ignore the interdependencies between event triggers and event arguments, which may be crucial semantic features for event extraction. To solve these problems, we propose a Chinese emergent event extraction method based on contrastive learning with event semantic features(CNEmergEE). Firstly, the proposed approach automatically expands the pre-trained model's vocabulary on a domain corpus. Then, continuing learning on the same corpus allows the model to complete domain adaptation. Finally, we use contrast learning to enable models to learn semantic representations of events. Experimental results on the CEC (Chinese Emergency Corpus) dataset show that our approach achieves better improvement. © 2023 IEEE.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

SLMP: A Scientific Literature Management Platform Based on Large Language Models

SLMP: A Scientific Literature Management Platform Based on L...

引用

IEEE International Conference on Big knowledge (ICBK)

作者： Menghao Guo Jinling Jiang Fan Wu Shanxin Sun Chen Zhang Wenhui Li Zeyi Sun Guangyong Chen Xindong Wu Research Center for Life Sciences Computing Zhejiang Lab Hangzhou China Research Center for Data Hub and Security Zhejiang Lab Hangzhou China Research Center for High Efficiency Computing System Zhejiang Lab Hangzhou China Key Laboratory of Knowledge Engineering With Big Data Hefei University of Technology Hefei China

ISBN: (数字)9798331508821

ISBN: (纸本)9798331508838

This paper presents a Scientific Literature Management Platform (SLMP, demo link 1 ) based on large language models (LLMs). The platform consists of four modules: literature management, literature extraction, literature retrieval, and question answering. The core techniques used to support the four modules across the platform include a fine-tuned model PaperExtractGPT and a continual pre-training model ChatPaperGPT based on ChatGLM 2 using the data from scientific research literature, responsible for information extraction and communication, respectively. Due to their powerful capabilities in natural language understanding and generation, LLMs can understand complex scientific concepts based on the provided contexts, and thus generate high-quality texts and conduct in-depth information retrieval and question answering. Our platform can help researchers manage and utilize literature more effectively and efficiently for finding relevant literature, obtaining required information, and generating new knowledge.

关键词： Training Large language models knowledge graphs Information retrieval data models Question answering (information retrieval)

来源：评论

学校读者我要写书评

暂无评论

Pathological Brain Detection Based on Online Sequential Extreme Learning Machine

Pathological Brain Detection Based on Online Sequential Extr...

引用

2016 IEEE International Conference on Progress in Informatics and Computing

作者： Siyuan Lu Hainan Wang Xueyan Wu Shuihua Wang School of Computer Science and Technology Nanjing Normal University State Key Lab of CAD & CG Zhejiang University Key laboratory of symbolic computation and knowledge engineering of ministry of education Jilin University Key Laboratory of Statistical information technology and data mining State Statistics Bureau Department of Electrical Engineering The City College of New YorkCUNY

Magnetic resonance imaging(MRI) is a kind of imaging modality, which offers clearer images of soft tissues than computed tomography(CT). It is especially suitable for brain disease detection. It is beneficial to detect diseases automatically and accurately. We proposed a pathological brain detection method based on brain MR images and online sequential extreme learning machine. First, seven wavelet entropies(WE) were extracted from each brain MR image to form the feature vector. Then, an online sequential extreme learning machine(OS-ELM) was trained to differentiate pathological brains from the healthy *** experiment results over 132 brain MRIs showed that the proposed approach achieved a sensitivity of 93.51%, a specificity of 92.22%, and an overall accuracy of 93.33%,which suggested that our method is effective.

关键词： magnetic resonance imaging wavelet entropy online sequential extreme learning machine Classification pattern recognition

来源：评论

学校读者我要写书评

暂无评论

Attention-Constrained Inference For Robust Decoder-Only Text-to-Speech

Attention-Constrained Inference For Robust Decoder-Only Text...

引用

IEEE Spoken Language Technology Workshop

作者： Hankun Wang Chenpeng Du Yiwei Guo Shuai Wang Xie Chen Kai Yu Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Shanghai Jiao Tong University Shanghai China Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China

ISBN: (数字)9798350392258

ISBN: (纸本)9798350392265

Recent popular decoder-only text-to-speech models are known for their ability of generating natural-sounding speech. However, such models sometimes suffer from word skipping and repeating due to the lack of explicit monotonic alignment constraints. In this paper, we notice from the attention maps that some particular attention heads of the decoder-only model indicate the alignments between speech and text. We call the attention maps of those heads Alignment-Emerged Attention Maps (AEAMs). Based on this discovery, we propose a novel inference method without altering the training process, named Attention-Constrained Inference (ACI), to facilitate monotonic synthesis. It first identifies AEAMs using the Attention Sweeping algorithm and then applies constraining masks on AEAMs. Our experimental results on decoderonly TTS model VALL-E show that the WER of synthesized speech is reduced by up to 20.5% relatively with ACI while the naturalness and speaker similarity are comparable.

关键词： Training Conferences Robustness Inference algorithms Text to speech Optimization

来源：评论

学校读者我要写书评

暂无评论

ATTENTION-CONSTRAINED INFERENCE FOR ROBUST DECODER-ONLY TEXT-TO-SPEECH

arXiv

引用

arXiv 2024年

作者： Wang, Hankun Du, Chenpeng Guo, Yiwei Wang, Shuai Chen, Xie Yu, Kai MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China

Recent popular decoder-only text-to-speech models are known for their ability of generating natural-sounding speech. However, such models sometimes suffer from word skipping and repeating due to the lack of explicit monotonic alignment constraints. In this paper, we notice from the attention maps that some particular attention heads of the decoder-only model indicate the alignments between speech and text. We call the attention maps of those heads Alignment-Emerged Attention Maps (AEAMs). Based on this discovery, we propose a novel inference method without altering the training process, named Attention-Constrained Inference (ACI), to facilitate monotonic synthesis. It first identifies AEAMs using the Attention Sweeping algorithm and then applies constraining masks on AEAMs. Our experimental results on decoder-only TTS model VALL-E show that the WER of synthesized speech is reduced by up to 20.5% relatively with ACI while the naturalness and speaker similarity are comparable. Copyright © 2024, The Authors. All rights reserved.

关键词： Decoding

来源：评论

学校读者我要写书评

暂无评论

UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocoding

arXiv

引用

arXiv 2023年

作者： Du, Chenpeng Guo, Yiwei Shen, Feiyu Liu, Zhijun Liang, Zheng Chen, Xie Wang, Shuai Zhang, Hui Yu, Kai MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China Shenzhen Research Institute of Big Data Shenzhen China AISpeech Ltd Beijing China

The utilization of discrete speech tokens, divided into semantic tokens and acoustic tokens, has been proven superior to traditional acoustic feature mel-spectrograms in terms of naturalness and robustness for text-to-speech (TTS) synthesis. Recent popular models, such as VALL-E and SPEAR-TTS, allow zero-shot speaker adaptation through auto-regressive (AR) continuation of acoustic tokens extracted from a short speech prompt. However, these AR models are restricted to generate speech only in a left-to-right direction, making them unsuitable for speech editing where both preceding and following contexts are provided. Furthermore, these models rely on acoustic tokens, which have audio quality limitations imposed by the performance of audio codec models. In this study, we propose a unified context-aware TTS framework called UniCATS, which is capable of both speech continuation and editing. UniCATS comprises two components, an acoustic model CTX-txt2vec and a vocoder CTX-vec2wav. CTX-txt2vec employs contextual VQ-diffusion to predict semantic tokens from the input text, enabling it to incorporate the semantic context and maintain seamless concatenation with the surrounding context. Following that, CTX-vec2wav utilizes contextual vocoding to convert these semantic tokens into waveforms, taking into consideration the acoustic context. Our experimental results demonstrate that CTXvec2wav outperforms HifiGAN and AudioLM in terms of speech resynthesis from semantic tokens. Moreover, we show that UniCATS achieves state-of-the-art performance in both speech continuation and editing. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Prototype and Instance Contrastive Learning for Unsupervised Domain Adaptation in Speaker Verification

Prototype and Instance Contrastive Learning for Unsupervised...

引用

International Symposium on Chinese Spoken Language Processing

作者： Wen Huang Bing Han Zhengyang Chen Shuai Wang Yanmin Qian Department of Computer Science and Engineering Auditory Cognition and Computational Acoustics Lab MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen

ISBN: (数字)9798331516826

ISBN: (纸本)9798331516833

Speaker verification system trained on one domain usually suffers performance degradation when applied to another domain. To address this challenge, researchers commonly use feature distribution matching-based methods in unsupervised domain adaptation scenarios where some unlabeled target domain data is available. However, these methods often have limited performance improvement and lack generalization in various mismatch situations. In this paper, we propose Prototype and Instance Contrastive Learning (PICL), a novel method for unsupervised domain adaptation in speaker verification through dual-level contrastive learning. For prototype contrastive learning, we generate pseudo labels via clustering to create dynamically updated prototype representations, aligning instances with their corresponding class or cluster prototypes. For instance contrastive learning, we minimize the distance between different views or augmentations of the same instance, ensuring robust and invariant representations resilient to variations like noise. This dual-level approach provides both high-level and low-level supervision, leading to improved generalization and robustness of the speaker verification model. Unlike previous studies that only evaluated mismatches in one situation, we have conducted relevant explorations on various datasets and achieved state-of-the-art performance currently, which also proves the generalization of our method.

关键词： Degradation Adaptation models Noise Prototypes Contrastive learning Robustness

来源：评论

学校读者我要写书评

暂无评论

Neural pose transfer by spatially adaptive instance normalization

arXiv

引用

arXiv 2020年

作者： Wang, Jiashun Wen, Chao Fu, Yanwei Lin, Haitao Zou, Tianyun Xue, Xiangyang Zhang, Yinda Fudan University Google LLC School of Data Science MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Academy of Engineering and Technology Institute of AI and Robotics Fudan University

Pose transfer has been studied for decades, in which the pose of a source mesh is applied to a target mesh. Particularly in this paper, we are interested in transferring the pose of source human mesh to deform the target human mesh, while the source and target meshes may have different identity information. Traditional studies assume that the paired source and target meshes are existed with the point-wise correspondences of user annotated landmarks/mesh points, which requires heavy labelling efforts. On the other hand, the generalization ability of deep models is limited, when the source and target meshes have different identities. To break this limitation, we proposes the first neural pose transfer model that solves the pose transfer via the latest technique for image style transfer, leveraging the newly proposed component - spatially adaptive instance normalization. Our model does not require any correspondences between the source and target meshes. Extensive experiments show that the proposed model can effectively transfer deformation from source to target meshes, and has good generalization ability to deal with unseen identities or poses of meshes. Code is available at https://***/jiashunwang/Neural-Pose-Transfer. Copyright © 2020, The Authors. All rights reserved.

关键词： Mesh generation

来源：评论

学校读者我要写书评

暂无评论

Neural Pose Transfer by Spatially Adaptive Instance Normalization

Neural Pose Transfer by Spatially Adaptive Instance Normaliz...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Jiashun Wang Chao Wen Yanwei Fu Haitao Lin Tianyun Zou Xiangyang Xue Yinda Zhang Fudan University School of Data Science and MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Academy of Engineering and Technology and Institute of AI and Robotics Fudan University Google LLC

ISBN: (数字)9781728171685

ISBN: (纸本)9781728171692

Pose transfer has been studied for decades, in which the pose of a source mesh is applied to a target mesh. Particularly in this paper, we are interested in transferring the pose of source human mesh to deform the target human mesh, while the source and target meshes may have different identity information. Traditional studies assume that the paired source and target meshes are existed with the point-wise correspondences of user annotated landmarks/mesh points, which requires heavy labelling efforts. On the other hand, the generalization ability of deep models is limited, when the source and target meshes have different identities. To break this limitation, we proposes the first neural pose transfer model that solves the pose transfer via the latest technique for image style transfer, leveraging the newly proposed component -- spatially adaptive instance normalization. Our model does not require any correspondences between the source and target meshes. Extensive experiments show that the proposed model can effectively transfer deformation from source to target meshes, and has good generalization ability to deal with unseen identities or poses of meshes. Code is available at https://***/jiashunwang/Neural-Pose-Transfer.

关键词： Shape Three-dimensional displays Feature extraction Strain Decoding Machine learning Task analysis

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：