检索结果-内蒙古大学图书馆

The SJTU System for Short-duration Speaker Verification Challenge 2021

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Bing Chen, Zhengyang Zhou, Zhikai Qian, Yanmin MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China

This paper presents the SJTU system for both text-dep.ndent and text-indep.ndent tasks in short-duration speaker verification (SdSV) challenge 2021. In this challenge, we explored different strong embedding extractors to extract robust speaker embedding. For text-indep.ndent task, language-dep.ndent adaptive snorm is explored to improve the system performance under the cross-lingual verification condition. For text-dep.ndent task, we mainly focus on the in-domain fine-tuning strategies based on the model pre-trained on large-scale out-of-domain data. In order to improve the distinction between different speakers uttering the same phrase, we proposed several novel phrase-aware fine-tuning strategies and phrase-aware neural PLDA. With such strategies, the system performance is further improved. Finally, we fused the scores of different systems, and our fusion systems achieved 0.0473 in Task1 (rank 3) and 0.0581 in Task2 (rank 8) on the primary evaluation metric. Copyright © 2022, The Authors. All rights reserved.

关键词： Embeddings

VQTTS: High-Fidelity Text-to-Speech Synthesis with Self-Supervised VQ Acoustic Feature

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Du, Chenpeng Guo, Yiwei Chen, Xie Yu, Kai MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China

The mainstream neural text-to-speech(TTS) pipeline is a cascade system, including an acoustic model(AM) that predicts acoustic feature from the input transcript and a vocoder that generates waveform according to the given acoustic feature. However, the acoustic feature in current TTS systems is typically mel-spectrogram, which is highly correlated along both time and frequency axes in a complicated way, leading to a great difficulty for the AM to predict. Although high-fidelity audio can be generated by recent neural vocoders from ground-truth(GT) mel-spectrogram, the gap between the GT and the predicted mel-spectrogram from AM degrades the performance of the entire TTS system. In this work, we propose VQTTS, consisting of an AM txt2vec and a vocoder vec2wav, which uses self-supervised vector-quantized(VQ) acoustic feature rather than mel-spectrogram. We redesign both the AM and the vocoder accordingly. In particular, txt2vec basically becomes a classification model instead of a traditional regression model while vec2wav uses an additional feature encoder before HifiGAN generator for smoothing the discontinuous quantized feature. Our experiments show that vec2wav achieves better reconstruction performance than HifiGAN when using self-supervised VQ acoustic feature. Moreover, our entire TTS system VQTTS achieves state-of-the-art performance in terms of naturalness among all current publicly available TTS systems. Copyright © 2022, The Authors. All rights reserved.

关键词： Spectrographs

Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yao, Yao Li, Zuchao Zhao, Hai Department of Computer Science and Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China National Engineering Research Center for Multimedia Software School of Computer Science Wuhan University Wuhan430072 China

With the widespread use of language models (LMs) in NLP tasks, researchers have discovered the potential of Chain-of-thought (CoT) to assist LMs in accomplishing complex reasoning tasks by generating intermediate steps. However, human thought processes are often non-linear, rather than simply sequential chains of thoughts. Therefore, we propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph. By representing thought units as nodes and connections between them as edges, our approach captures the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. GoT adopts a two-stage framework with an additional GoT encoder for thought graph representation and fuses the graph representation with the original input representation through a gated fusion mechanism. We evaluate GoT’s performance on a text-only reasoning task (AQUA-RAT) and a multimodal reasoning task (scienceQA). Our model achieves significant improvement over the strong CoT baseline on the AQUA-RAT test set and boosts accuracy from 85.19% to 87.59% using the T5-base model over the state-of-the-art Multimodal-CoT (Zhang et al., 2023) on the scienceQA test set. Our code is publicly available at https://***/Zoeyyao27/Graphof-Thought © 2023, CC BY.

关键词： Rats

EMODIFF: INTENSITY CONTROLlabLE EMOTIONAL TEXT-TO-SPEECH WITH SOFT-labEL GUIDANCE

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Guo, Yiwei Du, Chenpeng Chen, Xie Yu, Kai MoE Key Lab of Artificial Intelligence AI Institute X-LANCE Lab Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China

Although current neural text-to-speech (TTS) models are able to generate high-quality speech, intensity controllable emotional TTS is still a challenging task. Most existing methods need external optimizations for intensity calculation, leading to suboptimal results or degraded quality. In this paper, we propose EmoDiff, a diffusion-based TTS model where emotion intensity can be manipulated by a proposed soft-label guidance technique derived from classifier guidance. Specifically, instead of being guided with a one-hot vector for the specified emotion, EmoDiff is guided with a soft label where the value of the specified emotion and Neutral is set to α and 1 − α respectively. The α here represents the emotion intensity and can be chosen from 0 to 1. Our experiments show that EmoDiff can precisely control the emotion intensity while maintaining high voice quality. Moreover, diverse speech with specified emotion intensity can be generated by sampling in the reverse denoising process. Copyright © 2022, The Authors. All rights reserved.

关键词： Quality control

Inductive Dummy-based Homogeneous Neighborhood Augmentation for Graph Collaborative Filtering

学校读者我要写书评

暂无评论

Inductive Dummy-based Homogeneous Neighborhood Augmentation ...

International Joint Conference on Neural Networks (IJCNN)

作者： Wei Ding Jiawei Sun Jie Li Chentao Wu Department of Computer Science and Engineering Shanghai Jiao Tong University China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China Yancheng Blockchain Research Institute China

In the era of information explosion, we urgently need recommendation systems to filter massive amounts of information. Recent advancements in graph neural networks have led to the widespread adoption of graph collaborative filtering algorithms for recommendation systems. Despite their effectiveness, graph collaborative filtering algorithms have several limitations, such as data sparsity and long-tailed distribution. This sparse data with numerous long-tailed nodes can be viewed as an inductive scenario in which models require a robust inductive ability to learn quality representations from sparse data. Inductive graph collaborative filtering methods, such as Pin-SAGE, improve the generalization ability via random neighbor sampling. However, these inductive methods are time-consuming or ineffective in transductive scenarios because of the complicated operations and information loss in random neighbor sampling methods. We propose IDHA, an inductive dummy-based homogeneous neighborhood augmentation method for graph collaborative filtering, to address the issues above. Our method employs dummy nodes connected to all nodes to take advantage of low-degree nodes in graph structure learning. To improve the model's generalization in inductive scenarios, we adopt one-hop random-walk sampling. We propose homogeneous neighborhood augmentation via contrastive learning for inductive graph collaborative filtering. This method exploits contrastive learning in sparse data to its full potential. In addition, we employ a lightweight model design to enhance performance and practicality while reducing model complexity. Extensive experiments on three datasets demonstrate that our method outperforms the state-of-the-art transductive and inductive graph collaborative filtering recommendation methods.

关键词：

SG-GS: Topology-aware Human Avatars with Semantically-guided Gaussian Splatting

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhao, Haoyu Yang, Chen Wang, Hao Zhao, Xingyue Shen, Wei MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China School of Computer Science Wuhan University China Wuhan National Laboratory for Optoelectronics Huazhong University of Science and Technology China School of Software Engineering Xi’an Jiao Tong University China

Reconstructing photo-realistic and topology-aware animatable human avatars from monocular videos remains challenging in computer vision and graphics. Recently, methods using 3D Gaussians to represent the human body have emerged, offering faster optimization and real-time rendering. However, due to ignoring the crucial role of human body semantic information which represents the explicit topological and intrinsic structure within human body, they fail to achieve fine-detail reconstruction of human avatars. To address this issue, we propose SG-GS, which uses semantics-embedded 3D Gaussians, skeleton-driven rigid deformation, and non-rigid cloth dynamics deformation to create photo-realistic human avatars. We then design a Semantic Human-Body Annotator (SHA) which utilizes SMPL’s semantic prior for efficient body part semantic labeling. The generated labels are used to guide the optimization of semantic attributes of Gaussian. To capture the explicit topological structure of the human body, we employ a 3D network that integrates both topological and geometric associations for human avatar deformation. We further implement three key strategies to enhance the semantic accuracy of 3D Gaussians and rendering quality: semantic projection with 2D regularization, semantic-guided density regularization and semantic-aware regularization with neighborhood consistency. Extensive experiments demonstrate that SG-GS achieves state-of-the-art geometry and appearance reconstruction performance. Our project is at https://***/. Copyright © 2024, The Authors. All rights reserved.

关键词： Topology

Guided Patch-Grouping Wavelet Transformer with Spatial Congruence for Ultra-High Resolution Segmentation

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ji, Deyi Zhao, Feng Lu, Hongtao University of Science and Technology of China China Alibaba Group China Department of Computer Science and Engineering Shanghai Jiao Tong University China MOE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

Most existing ultra-high resolution (UHR) segmentation methods always struggle in the dilemma of balancing memory cost and local characterization accuracy, which are both taken into account in our proposed Guided Patch-Grouping Wavelet Transformer (GPWFormer) that achieves impressive performances. In this work, GPWFormer is a Transformer (T )-CNN (C) mutual leaning framework, where T takes the whole UHR image as input and harvests both local details and fine-grained long-range contextual dep.ndencies, while C takes downsampled image as input for learning the category-wise deep context. For the sake of high inference speed and low computation complexity, T partitions the original UHR image into patches and groups them dynamically, then learns the low-level local details with the lightweight multi-head Wavelet Transformer (WFormer) network. Meanwhile, the fine-grained long-range contextual dep.ndencies are also captured during this process, since patches that are far away in the spatial domain can also be assigned to the same group. In addition, masks produced by C are utilized to guide the patch grouping process, providing a heuristics decision. Moreover, the congruence constraints between the two branches are also exploited to maintain the spatial consistency among the patches. Overall, we stack the multistage process in a pyramid way. Experiments show that GPWFormer outperforms the existing methods with significant improvements on five benchmark datasets. Copyright © 2023, The Authors. All rights reserved.

关键词： Machine learning

Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmark

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ji, Deyi Zhao, Feng Lu, Hongtao Tao, Mingyuan Ye, Jieping University of Science and Technology of China China Alibaba Group China Department of Computer Science and Engineering Shanghai Jiao Tong University China MOE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

With the increasing interest and rapid development of methods for Ultra-High Resolution (UHR) segmentation, a large-scale benchmark covering a wide range of scenes with full fine-grained dense annotations is urgently needed to facilitate the field. To this end, the URUR dataset is introduced, in the meaning of Ultra-High Resolution dataset with Ultra-Rich Context. As the name suggests, URUR contains amounts of images with high enough resolution (3,008 images of size 5,120×5,120), a wide range of complex scenes (from 63 cities), rich-enough context (1 million instances with 8 categories) and fine-grained annotations (about 80 billion manually annotated pixels), which is far superior to all the existing UHR datasets including DeepGlobe, Inria Aerial, UDD, etc.. Moreover, we also propose WSDNet, a more efficient and effective framework for UHR segmentation especially with ultra-rich context. Specifically, multi-level Discrete Wavelet Transform (DWT) is naturally integrated to release computation burden while preserve more spatial details, along with a Wavelet Smooth Loss (WSL) to reconstruct original structured context and texture with a smooth constrain. Experiments on several UHR datasets demonstrate its state-of-the-art performance. The dataset is available at https://***/jankyee/URUR. Copyright © 2023, The Authors. All rights reserved.

关键词： Textures

ADAPTIVE INCENTIVE FOR CROSS-SILO FEDERATED LEARNING: A MULTI-AGENT REINFORCEMENT LEARNING APPROACH

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yuan, Shijing Liu, Hongze Lv, Hongtao Feng, Zhanbo Li, Jie Chen, Hongyang Wu, Chentao Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China School of Software Shandong University Jinan China Research Center for Graph Computing Zhejiang Lab China

Cross-silo federated learning (FL) is a typical FL that enables organizations (e.g., financial or medical entities) to train global models on isolated data. Reasonable incentive is key to encouraging organizations to contribute data. However, existing works on incentivizing cross-silo FL lack consideration of the environmental dynamics (e.g., precision of the trained global model and data owned by uncertain clients during the training processes). Moreover, most of them assume that organizations share private information, which is unrealistic. To overcome these limitations, we propose a novel adaptive mechanism for cross-silo FL, towards incentivizing organizations to contribute data to maximize their long-term payoffs in a real dynamic training environment. The mechanism is based on multi-agent reinforcement learning, which learns near-optimal data contribution strategy from the history of potential games without organizations’ private information. Experiments demonstrate that our mechanism achieves adaptive incentive and effectively improves the long-term payoffs for organizations. Copyright © 2023, The Authors. All rights reserved.

关键词： Reinforcement learning