检索结果-内蒙古大学图书馆

PAL: Prompting Analytic Learning with Missing Modality for Multi-Modal Class-Incremental Learning

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yue, Xianghu Chen, Yiming Zhang, Xueyi Gao, Xiaoxue Feng, Mengling Lao, Mingrui Zhuang, Huiping Li, Haizhou Department of Electrical and Computer Engineering National University of Singapore 117583 Singapore Laboratory for Big Data and Decision National University of Defense Technology Changsha410073 China Institute for Infocomm Research A*STAR 138632 Singapore Saw Swee Hock School of Public Health National University of Singapore 117583 Singapore Shien-Ming Wu School of Intelligent Engineering South China University of Technology Guangdong Guangzhou510006 China Shenzhen Research Institute of Big Data School of Data Science The Chinese University of Hong Kong Shenzhen518172 China Department of Electrical and Computer Engineering National University of Singapore 117583 Singapore

Multi-modal class-incremental learning (MMCIL) seeks to leverage multi-modal data, such as audio-visual and image-text pairs, thereby enabling models to learn continuously across a sequence of tasks while mitigating forgetting. While existing studies primarily focus on the integration and utilization of multi-modal information for MMCIL, a critical challenge remains: the issue of missing modalities during incremental learning phases. This oversight can exacerbate severe forgetting and significantly impair model performance. To bridge this gap, we propose PAL, a novel exemplar-free framework tailored to MMCIL under missing-modality scenarios. Concretely, we devise modality-specific prompts to compensate for missing information, facilitating the model to maintain a holistic representation of the data. On this foundation, we reformulate the MMCIL problem into a Recursive Least-Squares task, delivering an analytical linear solution. Building upon these, PAL not only alleviates the inherent under-fitting limitation in analytic learning but also preserves the holistic representation of missing-modality data, achieving superior performance with less forgetting across various multi-modal incremental scenarios. Extensive experiments demonstrate that PAL significantly outperforms competitive methods across various datasets, including UPMC-Food101 and N24News, showcasing its robustness towards modality absence and its anti-forgetting ability to maintain high incremental accuracy. © 2025, CC BY.

关键词： Contrastive Learning

SA-WavLM: Speaker-Aware Self-Supervised Pre-training for Mixture Speech

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Lin, Jingru Ge, Meng Ao, Junyi Deng, Liqun Li, Haizhou Department of Electrical and Computer Engineering National University of Singapore Singapore Huawei Noah's Ark Lab Hong Kong School of Data Science Shenzhen Research Institute of Big Data The Chinese University of Hong Kong Shenzhen China

It was shown that pre-trained models with self-supervised learning (SSL) techniques are effective in various downstream speech tasks. However, most such models are trained on single-speaker speech data, limiting their effectiveness in mixture speech. This motivates us to explore pre-training on mixture speech. This work presents SA-WavLM, a novel pre-trained model for mixture speech. Specifically, SA-WavLM follows an "extract-merge-predict" pipeline in which the representations of each speaker in the input mixture are first extracted individually and then merged before the final prediction. In this pipeline, SA-WavLM performs speaker-informed extractions with the consideration of the interactions between different speakers. Furthermore, a speaker shuffling strategy is proposed to enhance the robustness towards the speaker absence. Experiments show that SA-WavLM either matches or improves upon the state-of-the-art pre-trained models. Copyright © 2024, The Authors. All rights reserved.

关键词： Speech recognition

Dual Encoders for Diffusion-based Image Inpainting

学校读者我要写书评

暂无评论

Dual Encoders for Diffusion-based Image Inpainting

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Dezhi Zheng Kaijun Deng Jinbao Wang Linlin Shen Computer Vision Institute School of Computer Science and Software Engineering Shenzhen University Shenzhen China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen China

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Current diffusion-based inpainting models struggle to preserve unmasked regions or generate highly coherent content. Additionally, it is hard for them to generate meaningful content for 3D inpainting. To tackle these challenges, we design a plug-and-play branch that runs through the entire generation process to enhance existing models. Specifically, we utilize dual encoders - a Convolutional Neural Network (CNN) encoder and the pre-trained Variational AutoEncoder (VAE) encoder, to encode masked images. The latent code and the feature map from the dual encoders are fed to diffusion models simultaneously. In addition, we apply Zero-padded initialization to solve the problem of mode collapse caused by this branch. Experiments on BrushBench and EditBench demonstrate that models with our plug-and-play branch can improve the coherence of inpainting, and our model achieves new state-of-the-art results.

关键词： Visualization Three-dimensional displays Codes Autoencoders Coherence Signal processing Diffusion models Decoding Convolutional neural networks Speech processing

A comprehensive imputation-based evaluation of tag SNP selection strategies 13

学校读者我要写书评

暂无评论

A comprehensive imputation-based evaluation of tag SNP selec...

13th International Conference Knowledge and Systems engineering, KSE 2021

作者： Nguyen, Dat Thanh Dinh, Hieu Quang Vu, Giang Minh Nguyen, Duong Thuy Vo, Nam Sy Center for Biomedical Informatics Vingroup Big Data Institute Hanoi Viet Nam Massachusetts Institute of Technology CambridgeMA United States Center for Biomedical Informatics Vingroup Big Data Institute Institute of Genome Research Vietnam Academy of Science and Technology Hanoi Viet Nam VinUniversity Center for Biomedical Informatics Vingroup Big Data Institute College of Engineering and Computer Science Hanoi Viet Nam

ISBN: (纸本)9781665499750

Regardless of the rapid development of sequencing technology, single nucleotide polymorphism (SNP) array has been widely used for many large-scale genomic studies due to its cost-effectiveness. Recently, in parallel with the advancement in imputation strategies, several genotyping platforms for various species have been developed. Despite the importance of imputation accuracy in SNP array design, to the best of our knowledge, there are no systematic studies for evaluating tag SNP selection methods based on this metric. In this paper, using the leave-one-out cross-validation approach on the 1000 genome high-coverage dataset, we comprehensively evaluated four well-known tag SNP selection algorithms based on imputation accuracy. Our results showed that although all widely used methods for SNP array design can provide reasonable imputation accuracy, pairwise linkage disequilibrium based tag SNP selection algorithm achieves the best performance. Our pipelines for running evaluated algorithms and leave-one-out cross-validation are available for public use at https://***/datngu/TagSNP_evaluation. © 2021 IEEE.

关键词： Cost effectiveness

stEnTrans: Transformer-based deep learning for spatial transcriptomics enhancement

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Xue, Shuailin Zhu, Fangfang Wang, Changmiao Min, Wenwen School of Information Science and Engineering Yunnan University Kunming China College of Nursing Health Sciences Yunnan Open University Kunming China Shenzhen Research Institute of Big Data Shenzhen China

The spatial location of cells within tissues and organs is crucial for the manifestation of their specific *** transcriptomics technology enables comprehensive measurement of the gene expression patterns in tissues while retaining spatial information. However, current popular spatial transcriptomics techniques either have shallow sequencing depth or low resolution. We present stEnTrans, a deep learning method based on Transformer architecture that provides comprehensive predictions for gene expression in unmeasured areas or unexpectedly lost areas and enhances gene expression in original and inputed spots. Utilizing a self-supervised learning approach, stEnTrans establishes proxy tasks on gene expression profile without requiring additional data, mining intrinsic features of the tissues as supervisory information. We evaluate stEnTrans on six datasets and the results indicate superior performance in enhancing spots resolution and predicting gene expression in unmeasured areas compared to other deep learning and traditional interpolation methods. Additionally, Our method also can help the discovery of spatial patterns in Spatial Transcriptomics and enrich to more biologically significant pathways. Our source code is available at https://***/shuailinxue/stEnTrans. Copyright © 2024, The Authors. All rights reserved.

关键词： Gene expression

A Local Regression-Based SLA-Aware Energy-Efficient Virtual Machine Consolidation Algorithm in data Centers

学校读者我要写书评

暂无评论

A Local Regression-Based SLA-Aware Energy-Efficient Virtual ...

International Conference on big data Computing and Communications (bigCOM)

作者： Xiaodong Wu Ronghai Wang Guoxin Lin School of Mathmatics and Computer Science Quanzhou Normal University Fujian Provincial Key Laboratory of Data-Intensive Computing Key Laboratory of Intelligent Computing and Information Processing Fujian Province University Fujian Provincial Big Data Research Institute of Intelligent Manufacturing Quanzhou China

Virtual Machine Consolidation (VMC) is an effective means to improve the resource utilization and reduce energy consumption of data centers. However, the VMC optimization that only considers energy consumption may lead to SLA (Service-Level Agreements) violations and thus fail to guarantee the QoS (Quality of Service) provided. Therefore, it is necessary to consider both energy saving and SLA violations in the process of VMC. In this paper, we propose an energy and SLA-aware VMC algorithm that considers both energy saving and SLA violation optimization in data centers. The proposed algorithm uses the local regression method to predict the utilization of virtual machines, based on which it determines whether the host will be overloaded, so as to make the decision of virtual machine migrations. The proposed VMC algorithm is evaluated with CloudSim toolkit. Simulation results demonstrate that our algorithm has good energy saving performance and can effectively reduce SLA violation in cloud data center. Considering energy consumption and SLA violation, the ESV can be reduced by 7.57% on average and 16.36% at most compared with other algorithms.

关键词：

Multi-Scale Accent Modeling and Disentangling for Multi-Speaker Multi-Accent Text-to-Speech Synthesis

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Xuehao Zhang, Mingyang Zhou, Yi Wu, Zhizheng Li, Haizhou Department of Electrical and Computer Engineering National University of Singapore Singapore Hong Kong Generative AI Research and Development Centre The Hong Kong University of Science and Technology Hong Kong Shenzhen Research Institute of Big Data School of Data Science The Chinese University of Hong Kong Shenzhen China

Generating speech across different accents while preserving speaker identity is crucial for various real-world applications. However, accurately and independently modeling both speaker and accent characteristics in text-to-speech (TTS) systems is challenging due to the complex variations of accents and the inherent entanglement between speaker and accent identities. In this paper, we propose a novel approach for multispeaker multi-accent TTS synthesis that aims to synthesize speech for multiple speakers, each with various accents. Our approach employs a multi-scale accent modeling strategy to address accent variations on different levels. Specifically, we introduce both global (utterance level) and local (phoneme level) accent modeling to capture overall accent characteristics within an utterance and fine-grained accent variations across phonemes, respectively. To enable independent control of speakers and accents, we use the speaker embedding to represent speaker identity and achieve speaker-independent accent control through speaker disentanglement within the multi-scale accent modeling. Additionally, we present a local accent prediction model that enables our system to generate accented speech directly from phoneme inputs. We conduct extensive experiments on an English accented speech corpus. Experimental results demonstrate that our proposed system outperforms baseline systems in terms of speech quality and accent rendering for generating multispeaker multi-accent speech. Ablation studies further validate the effectiveness of different components in our proposed system. Copyright © 2024, The Authors. All rights reserved.

关键词： Quantum entanglement

research on Digital Identity Technology and Application Based on Identification Code and Trusted Account Blockchain fusion 2

学校读者我要写书评

暂无评论

Research on Digital Identity Technology and Application Base...

2nd International Seminar on Artificial Intelligence, Networking and Information Technology, AINIT 2021

作者： Zhao, Wei Yang, Ning Li, Guangli Zhang, Keli China Information Security Research Institute Co. Ltd Beijing China National Engineering Laboratory for Big Data Analysis and Applications Center for Data Science Beijing China

ISBN: (纸本)9781665412964

Digital identity is one of the key elements to build a trust system in the digital economy era. Digital identity has empowered social governance and public services in the new era in many fields, and promoted the rapid development of digital economy. In this paper, a trusted digital identity based on identification code and trusted account blockchain fusion is proposed. Through the unified standard, unified coding identifier, combined with identity-based cryptographic algorithm and blockchain, identification code and trusted account blockchain fusion is realized, and a real, effective, safe and credible digital identity technology system is constructed. Trusted digital identity is a bridge connecting the virtual world and the physical world. It plays a strong supporting role in realizing the identity fusion and 'digital twin' of the physical world offline and the virtual world. It is the core element of the construction of new smart cities and the development of digital economy. © 2021 IEEE.

关键词： Blockchain

Beyond the Limit of Weight-Sharing: Pioneering Space-Evolving NAS with Large Language Models

学校读者我要写书评

暂无评论

Beyond the Limit of Weight-Sharing: Pioneering Space-Evolvin...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Xiu Su Shan You Hongyan Xu Xiuxing Li Jun Long Yi Chen Chang Xu Big Data Institute Central South University School of Computer Science Faculty of Engineering The University of Sydney SenseTime Research School of Computer Science and Engineering University of New South Wales Institute of Computing Technology Chinese Academy of Sciences School of Computer Science Hong Kong University of Science and Technology

Large language models (LLMs) offer impressive performance across diverse fields, but their increasing complexity raises both design costs and the need for specialized expertise. These challenges are intensified for Neural Architecture Search (NAS) methods reliant on weight-sharing techniques. This paper introduces GNAS, a new NAS method that boosts the search process with the aid of LLMs for efficient model discovery. With insights from existing architectures, GNAS swiftly identifies superior models that can adapt to changing resource constraints. We provide a mathematical framework to facilitate the transfer of knowledge across different model sizes, thereby improving search efficiency. Our experiments conducted on ImageNet, NAS-Bench-Macro, and ChannelBench-Macro confirm the effectiveness of GNAS across both CNN and Transformer architectures.

关键词：