检索结果-内蒙古大学图书馆

10th Conference on Sound and Music Technology, CSMT 2023

作者： Li, Peilin Chen, Kairan Wei, Weixin Zhao, Jiahao Li, Wei School of Computer Science and Technology Fudan University Shanghai Shanghai200438 China Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai Shanghai200433 China

ISBN: (纸本)9789819779611

Computational music emotion recognition (MER) is an important task that aims to recognize emotional content in music tracks. Understanding the emotional content of music can help tailor therapeutic interventions to specific emotional needs, potentially improving their effectiveness. Piano music is widely used in music therapy for depression, anxiety, and other mental health conditions. However, comparatively little work has been done on MER in piano music, which is a challenging task due to the unique difficulties in recognizing emotions in single-timbre music without vocals, especially on valence. In this paper, we propose a novel Double-Mix-Net model that utilizes the multi-layer feature mixing and multi-modal feature mixing methods to improve classification performance. Our model uses both the title of the YouTube music video and the music context to predict the emotion of a given music track. Additionally, we make full use of different granularity features through the multi-layer feature mixing method. Our proposed model achieved state-of-the-art accuracy performance of 73.8% on the Emopia dataset and is the first multi-modal network designed for piano emotion recognition. The results of our study highlight the importance of considering the music title and utilizing multi-layer feature mixing methods in piano music emotion recognition. This work helps the emotional understanding of piano music. Our proposed model can be applied in various domains such as music recommendation systems, film scoring, and game design to enhance the emotional impact of music. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Game design

来源：评论

学校读者我要写书评

暂无评论

Audio-Driven Identity Manipulation for Face Inpainting 24

Audio-Driven Identity Manipulation for Face Inpainting

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Sun, Yuqi Lin, Qing Tan, Weimin Yan, Bo Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Singapore Nanyang Technological University Singapore

ISBN: (纸本)9798400706868

Recent advances in multimodal artificial intelligence have greatly improved the integration of vision-language-audio cues to enrich the content creation process. Inspired by these developments, in this paper, we first integrate audio into the face inpainting task to facilitate identity manipulation. Our main insight is that a person's voice carries distinct identity markers, such as age and gender, which provide an essential supplement for identity-aware face inpainting. By extracting identity information from audio as guidance, our method can naturally support tasks of identity preservation and identity swapping in face inpainting. Specifically, we introduce a dual-stream network architecture comprising a face branch and an audio branch. The face branch is tasked with extracting deterministic information from the visible parts of the input masked face, while the audio branch is designed to capture heuristic identity priors from the speaker's voice. The identity codes from two streams are integrated using a multi-layer perceptron (MLP) to create a virtual unified identity embedding that represennts comprehensive identity features. In addition, to explicitly exploit the information from audio, we introduce an audio-face generator to generate an 'fake' audio face directly from audio and fuse the multi-scale intermediate features from the audio-face generator into face inpainting network through an audio-visual feature fusion (AVFF) module. Extensive experiments demonstrate the positive impact of extracting identity information from audio on face inpainting task, especially in identity preservation. © 2024 ACM.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

Mixture of Experts for Audio-Visual Learning 38

Mixture of Experts for Audio-Visual Learning

引用

38th Conference on Neural information processing Systems, NeurIPS 2024

作者： Cheng, Ying Li, Yang He, Junjie Feng, Rui School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China

With the rapid development of multimedia technology, audio-visual learning has emerged as a promising research topic within the field of multimodal analysis. In this paper, we explore parameter-efficient transfer learning for audio-visual learning and propose the Audio-Visual Mixture of Experts (AVMoE) to inject adapters into pre-trained models flexibly. Specifically, we introduce unimodal and cross-modal adapters as multiple experts to specialize in intra-modal and inter-modal information, respectively, and employ a lightweight router to dynamically allocate the weights of each expert according to the specific demands of each task. Extensive experiments demonstrate that our proposed approach AVMoE achieves superior performance across multiple audio-visual tasks, including AVE, AVVP, AVS, and AVQA. Furthermore, visual-only experimental results also indicate that our approach can tackle challenging scenes where modality information is missing. The source code is available at https://***/yingchengy/AVMOE. © 2024 Neural information processing systems foundation. All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

SPIKING CONVOLUTIONAL NEURAL NETWORKS FOR TEXT CLASSIFICATION 11

SPIKING CONVOLUTIONAL NEURAL NETWORKS FOR TEXT CLASSIFICATIO...

引用

11th International Conference on Learning Representations, ICLR 2023

作者： Lv, Changze Xu, Jianhan Zheng, Xiaoqing School of Computer Science Fudan University Shanghai 200433 China Shanghai Key Laboratory of Intelligent Information Processing

Spiking neural networks (SNNs) offer a promising pathway to implement deep neural networks (DNNs) in a more energy-efficient manner since their neurons are sparsely activated and inferences are event-driven. However, there have been very few works that have demonstrated the efficacy of SNNs in language tasks partially because it is non-trivial to represent words in the forms of spikes and to deal with variable-length texts by SNNs. This work presents a "conversion + fine-tuning" two-step method for training SNNs for text classification and proposes a simple but effective way to encode pre-trained word embeddings as spike trains. We show empirically that after fine-tuning with surrogate gradients, the converted SNNs achieve comparable results to their DNN counterparts with much less energy consumption across multiple datasets for both English and Chinese. We also show that such SNNs are more robust to adversarial attacks than DNNs. © 2023 11th International Conference on Learning Representations, ICLR 2023. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

A Power Consumption Forecasting Method Based on Knowledge Embedding Under the Influence of the COVID-19 Pandemic 8th

A Power Consumption Forecasting Method Based on Knowledge E...

引用

8th China National Conference on Big Data and Social Computing, BDSC 2023

作者： Zhang, Qifan Cao, Xiu Shanghai Key Lab of Intelligent Information Processing Department of Computer Science Fudan University Shanghai200433 China

ISBN: (纸本)9789819939244

This paper addresses the new challenges of power consumption forecasting under the COVID-19 pandemic by proposing a knowledge embedding-based method (PEC-19). This method constructs a multi-source enterprise knowledge graph by analyzing and combining the specific impact of COVID-19 on power consumption, and designs the Connection Count Calculation of PEC-19 (3CP) algorithm and Similarity Degree Calculation of PEC-19 (SDCP) algorithm based on the graph to measure the similarity between different users affected by the COVID-19 pandemic and enable knowledge transfer and embedding among them. Moreover, the PEC-19 framework introduces a SEIR-based caṇse number prediction module and a comprehensive feature extraction module to account for the trend of the COVID-19 pandemic and increase the variety of time series features. Compared with existing baseline methods, the PEC-19 framework can significantly improve the power consumption forecasting performance in both univariate and multivariate time series forecasting tasks, and each module contributes differently to the proposed method. The framework also has some applicability and provides a new solution for various public health emergencies, which helps to reduce power operation cost, coordinate supply and demand, and conduct accurate scheduling. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： COVID-19

来源：评论

学校读者我要写书评

暂无评论

CPT: a pre-trained unbalanced transformer for both Chinese language understanding and generation

引用

science China(information sciences) 2024年第5期67卷 43-55页

作者： Yunfan SHAO Zhichao GENG Yitao LIU Junqi DAI Hang YAN Fei YANG Zhe LI Hujun BAO Xipeng QIU School of Computer Science Fudan University Shanghai Key Laboratory of Intelligent Information Processing Fudan University Zhejiang Lab

In this paper, we take the advantage of previous pre-trained models(PTMs) and propose a novel Chinese pre-trained unbalanced transformer(CPT). Different from previous Chinese PTMs, CPT is designed to utilize the shared knowledge between natural language understanding(NLU) and natural language generation(NLG) to boost the performance. CPT consists of three parts: a shared encoder, an understanding decoder, and a generation decoder. Two specific decoders with a shared encoder are pretrained with masked language modeling(MLM) and denoising auto-encoding(DAE) tasks, *** the partially shared architecture and multi-task pre-training, CPT can(1) learn specific knowledge of both NLU or NLG tasks with two decoders and(2) be fine-tuned flexibly that fully exploits the potential of the model. Moreover, the unbalanced transformer saves the computational and storage cost, which makes CPT competitive and greatly accelerates the inference of text generation. Experimental results on a wide range of Chinese NLU and NLG tasks show the effectiveness of CPT.

关键词： pre-trained model transformer language model generation unified model

来源：评论

学校读者我要写书评

暂无评论

Calibrating the Confidence of Large Language Models by Eliciting Fidelity

Calibrating the Confidence of Large Language Models by Elici...

引用

2024 Conference on Empirical Methods in Natural Language processing, EMNLP 2024

作者： Zhang, Mozhi Huang, Mianqiu Shi, Rundong Guo, Linsen Peng, Chong Yan, Peng Zhou, Yaqian Qiu, Xipeng School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China Meituan China

ISBN: (纸本)9798891761643

Large language models optimized with techniques like RLHF have achieved good alignment in being helpful and harmless. However, post-alignment, these language models often exhibit overconfidence, where the expressed confidence does not accurately calibrate with their correctness rate. In this paper, we decompose the language model confidence into the Uncertainty about the question and the Fidelity to the answer generated by language models. Then, we propose a plug-and-play method, UF Calibration, to estimate the confidence of language models. Our method has shown good calibration performance by conducting experiments with 6 RLHF-LMs on four MCQA datasets. Moreover, we propose two novel metrics, IPR and CE, to evaluate the calibration of the model, and we have conducted a detailed discussion on Truly Well-Calibrated Confidence for large language models. Our method could serve as a strong baseline, and we hope that this work will provide some insights into the model confidence calibration. © 2024 Association for Computational Linguistics.

关键词： Calibration

来源：评论

学校读者我要写书评

暂无评论

QS-Craft: Learning to Quantize, Scrabble and Craft for Conditional Human Motion Animation 16th

QS-Craft: Learning to Quantize, Scrabble and Craft for Co...

引用

16th Asian Conference on computer Vision, ACCV 2022

作者： Hong, Yuxin Qian, Xuelin Luo, Simian Guo, Guodong Xue, Xiangyang Fu, Yanwei School of Data Science and MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China School of Computer Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China Department of CSEE West Virginia University Morgantown United States

ISBN: (纸本)9783031263507

This paper studies the task of conditional Human Motion Animation (cHMA). Given a source image and a driving video, the model should animate the new frame sequence, in which the person in the source image should perform a similar motion as the pose sequence from the driving video. Despite the success of Generative Adversarial Network (GANs) methods in image and video synthesis, it is still very challenging to conduct cHMA due to the difficulty in efficiently utilizing the conditional guided information such as images or poses, and generating images of good visual quality. To this end, this paper proposes a novel model of learning to Quantize, Scrabble, and Craft (QS-Craft) for conditional human motion animation. The key novelties come from the newly introduced three key steps: quantize, scrabble and craft. Particularly, our QS-Craft employs transformer in its structure to utilize the attention architectures. The guided information is represented as a pose coordinate sequence extracted from the driving videos. Extensive experiments on human motion datasets validate the efficacy of our model. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Animation

来源：评论

学校读者我要写书评

暂无评论

UNRAVELING THE ENIGMA OF DOUBLE DESCENT: AN IN-DEPTH ANALYSIS THROUGH THE LENS OF LEARNED FEATURE SPACE 12

UNRAVELING THE ENIGMA OF DOUBLE DESCENT: AN IN-DEPTH ANALYSI...

引用

12th International Conference on Learning Representations, ICLR 2024

作者： Gu, Yufei Zheng, Xiaoqing Aste, Tomaso School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing China Department of Computer Science University College London United Kingdom

Double descent presents a counter-intuitive aspect within the machine learning domain, and researchers have observed its manifestation in various models and tasks. While some theoretical explanations have been proposed for this phenomenon in specific contexts, an accepted theory for its occurring mechanism in deep learning remains yet to be established. In this study, we revisit the phenomenon of double descent and demonstrate that the presence of noisy data strongly influences its occurrence. By comprehensively analysing the feature space of learned representations, we unveil that double descent arises in imperfect models trained with noisy data. We argue that while small and intermediate models before the interpolation threshold follow the traditional bias-variance trade-off, over-parameterized models interpolate noisy samples among robust data thus acquiring the capability to separate the information from the noise. The source code is available at https://***/Yufei-Gu-451/double_descent_***. © 2024 12th International Conference on Learning Representations, ICLR 2024. All rights reserved.

关键词： Economic and social effects

来源：评论

学校读者我要写书评

暂无评论

SCSGNet: Spatial-Correlated and Shape-Guided Network for Breast Mass Segmentation 48

SCSGNet: Spatial-Correlated and Shape-Guided Network for Bre...

引用

48th IEEE International Conference on Acoustics, Speech and Signal processing, ICASSP 2023

作者： Li, Qingqiu Xu, Jilan Yuan, Runtian Zhang, Yuejie Feng, Rui Fudan University School of Academy for Engineering and Technology Shanghai China Fudan University School of Computer Science Shanghai Key Lab of Intelligent Information Processing Shanghai China

ISBN: (纸本)9781728163277

Automatic and accurate breast mass segmentation plays a crucial role in the early diagnosis of breast cancer. However, it has been a challenging task for two main reasons: (1) Breast masses are diverse;and (2) The boundaries of masses are ambiguous. To address these problems, we propose a Spatial-Correlated and Shape-Guided Network (SCSGNet), which combines global context extraction with local boundary refinement. Specifically, the high-level features are aggregated to produce a global map as the initial guidance area, and a Series-Parallel Feature Fusion (SPFF) module is added to capture masses of different shapes and sizes. Besides, we design a Dynamic Long-range Correlation Capture (DLCC) module to capture the spatial correlation of masses at different positions. Finally, we devise a Triplet Attention Guide (TAG) module to iteratively update the feature map and refine the boundary. Experiments on two public datasets demonstrate that our method achieves superior performance over other state-of-the-art methods. © 2023 IEEE.

关键词： Iterative methods

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：