检索结果-内蒙古大学图书馆

CPT: a pre-trained unbalanced transformer for both Chinese language understanding and generation

science China(information sciences) 2024年第5期67卷 43-55页

作者： Yunfan SHAO Zhichao GENG Yitao LIU Junqi DAI Hang YAN Fei YANG Zhe LI Hujun BAO Xipeng QIU School of Computer Science Fudan University Shanghai Key Laboratory of Intelligent Information Processing Fudan University Zhejiang Lab

In this paper, we take the advantage of previous pre-trained models(PTMs) and propose a novel Chinese pre-trained unbalanced transformer(CPT). Different from previous Chinese PTMs, CPT is designed to utilize the shared knowledge between natural language understanding(NLU) and natural language generation(NLG) to boost the performance. CPT consists of three parts: a shared encoder, an understanding decoder, and a generation decoder. Two specific decoders with a shared encoder are pretrained with masked language modeling(MLM) and denoising auto-encoding(DAE) tasks, *** the partially shared architecture and multi-task pre-training, CPT can(1) learn specific knowledge of both NLU or NLG tasks with two decoders and(2) be fine-tuned flexibly that fully exploits the potential of the model. Moreover, the unbalanced transformer saves the computational and storage cost, which makes CPT competitive and greatly accelerates the inference of text generation. Experimental results on a wide range of Chinese NLU and NLG tasks show the effectiveness of CPT.

关键词： pre-trained model transformer language model generation unified model

来源：评论

学校读者我要写书评

暂无评论

Evidential Open Set Recognition for Imbalanced Medical Images via Multi-level Data Augmentation

Evidential Open Set Recognition for Imbalanced Medical Image...

引用

2024 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2024

作者： Zhao, Rui-Wei Xu, Yiqian Zhang, Ying Feng, Rui Shanghai Maritime University College of Information Engineering China Fudan University Academy for Engineering and Technology China Fudan University School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing China

ISBN: (纸本)9798350386226

Due to the exstence of common and rare diseases, the complex clinical scenario often poses the challenging class imbalanced open set recognition problem. Unfortunately, most existing approaches are ill-suited for such situations with limited and imbalanced data during training and the possibility of encountering unseen classes during test. In this work, we propose a novel Multi-level mixup-based Evidential Open Set Recognition (ME-OSR) approach to more explicitly and effectively address the open set recognition for class-imbalanced medical images. Briefly, we first extract disentangled discriminative and background image features. Then, based on the original images and extracted features, we propose to sample and conduct Multi-level Open Mixup (MOM) for a more balanced open set data augmentation. It includes extensive intra- and interclass mixup operations in both image and feature spaces, which can augment rare classes with different feature combinations and generate potential pseudo-unknown class examples in the open set to boost the model training. Based on the augmented data and their extracted discriminative features, we propose a Regularized Evidential Deep Learning (REDL) classifier to work with the augmented data to achieve open set recognition with prediction uncertainty estimation and unknown example rejection. Through comparative experiments and ablation studies on several representative medical datasets, we showed that our proposed method outperforms other state-of-the-arts on four popular medical OSR datasets. © 2024 IEEE.

关键词： Medical imaging

来源：评论

学校读者我要写书评

暂无评论

Motion-Aware Video Paragraph Captioning via Exploring Object-Centered Internal Knowledge 48

Motion-Aware Video Paragraph Captioning via Exploring Object...

引用

48th IEEE International Conference on Acoustics, Speech and Signal processing, ICASSP 2023

作者： Hu, Yimin Yu, Guorui Zhang, Yuejie Feng, Rui Zhang, Tao Lu, Xuequan Gao, Shang Fudan University School of Computer Science Shanghai Key Lab of Intelligent Information Processing China Shanghai University of Finance and Economics School of Information Management and Engineering China Deakin University School of Information Technology Australia

ISBN: (纸本)9781728163277

Video paragraph captioning task aims at generating a fine-grained, coherent and relevant paragraph for a video. Different from the images where objects are static, the temporal states of objects are changing in videos. The dynamic information could be contributed to understanding the whole video content. Existing works rarely put focus on modeling the dynamic changing state of the objects in the videos, causing the activities occurred in videos are poorly or wrongly depicted in paragraphs. To address this problem, we propose a novel Object State Tracking Network, which can capture the temporal state change of objects. However, due to the similarity of the consecutive frames in the videos, the information of the video is redundant and noisy. We further propose a semantic alignment mechanism, and enable the sentence information to refine the visual information. Extensive experiments on ActivityNet Captions demonstrate the effectiveness of our method. © 2023 IEEE.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Video Captioning via Relation-Aware Graph Learning 48

Video Captioning via Relation-Aware Graph Learning

引用

48th IEEE International Conference on Acoustics, Speech and Signal processing, ICASSP 2023

作者： Zheng, Yi Jing, Heming Xie, Qiujie Zhang, Yuejie Feng, Rui Zhang, Tao Gao, Shang Fudan University School of Computer Science Shanghai Key Lab of Intelligent Information Processing China Shanghai University of Finance and Economics School of Information Management and Engineering China Deakin University School of Information Technology Australia

ISBN: (纸本)9781728163277

Recent neural models for video captioning usually employed an encoder-decoder framework. However, most approaches either neglected the spatial and temporal interactions between objects in a video or implicitly modelled the interactions, resulting in less desired performance. In this paper, we propose a novel relation-aware graph learning framework. It explicitly models both spatial and temporal relations for objects. In particular, a relation-aware graph is designed to depict the spatial relations between different objects in a scene. Parallelly, a temporal graph network is designed to perform relational reasoning for the same objects in adjacent frames. Features of both types of relations are learned and fused for the follow-up language decoder. Experiments on two bench-mark datasets show the effectiveness of our framework. It achieves state-of-the-art performance with CIDEr scores on MSVD and MSR-VTT. © 2023 IEEE.

关键词： Relation-Aware Graph Scene Graph Temporal Graph Video captioning

来源：评论

学校读者我要写书评

暂无评论

QS-Craft: Learning to Quantize, Scrabble and Craft for Conditional Human Motion Animation 16th

QS-Craft: Learning to Quantize, Scrabble and Craft for Co...

引用

16th Asian Conference on computer Vision, ACCV 2022

作者： Hong, Yuxin Qian, Xuelin Luo, Simian Guo, Guodong Xue, Xiangyang Fu, Yanwei School of Data Science and MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China School of Computer Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China Department of CSEE West Virginia University Morgantown United States

ISBN: (纸本)9783031263507

This paper studies the task of conditional Human Motion Animation (cHMA). Given a source image and a driving video, the model should animate the new frame sequence, in which the person in the source image should perform a similar motion as the pose sequence from the driving video. Despite the success of Generative Adversarial Network (GANs) methods in image and video synthesis, it is still very challenging to conduct cHMA due to the difficulty in efficiently utilizing the conditional guided information such as images or poses, and generating images of good visual quality. To this end, this paper proposes a novel model of learning to Quantize, Scrabble, and Craft (QS-Craft) for conditional human motion animation. The key novelties come from the newly introduced three key steps: quantize, scrabble and craft. Particularly, our QS-Craft employs transformer in its structure to utilize the attention architectures. The guided information is represented as a pose coordinate sequence extracted from the driving videos. Extensive experiments on human motion datasets validate the efficacy of our model. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

关键词： Animation

来源：评论

学校读者我要写书评

暂无评论

Multiband Kick Drum Separation with Genre-Guided Optimization 10th

Multiband Kick Drum Separation with Genre-Guided Optimizati...

引用

10th Conference on Sound and Music technology, CSMT 2023

作者： Li, Si Jiang, Yuanfeng Wang, Zhaowen Qian, Jiale Li, Wei School of Computer Science and Technology Fudan University Shanghai200438 China Department of Music AI and Information Technology Central Conservatory of Music Beijing100091 China Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai200433 China

ISBN: (纸本)9789819779611

Kick drum plays an important role in electronic music. When hearing an excellent kick drum sound from a reference music track, music production beginners always feel confused about how it will sound if we solo the kick track and why it sounds perfect in a specific song. Reproducing the kick of a music track is not easy even for professional producers and arrangers. To investigate the ability of artificial intelligence in separating kick-drum sounds, in this paper we propose an end-to-end system which is capable of separating multi-genre high-quality electronic kick drum clips from mixed song tracks. Unlike most of the previous music source separation tasks, we only focus on the electronic kick drum rather than the whole percussion track. First, we design a multiband data representation approach for non-pitched percussions such as kick drums, then adopt a multi-discriminator module trained on a multi-genre kick drum corpus to further optimize the separated kick sound. To the authors’ knowledge, this study marks the first attempt at kick drum separation with stylistic optimization. Objective and subjective evaluations demonstrate that joint multiband representation and genre-guided optimisation can form a high-performance framework, showing a promising solution to the kick drum separation task. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Source separation

来源：评论

学校读者我要写书评

暂无评论

Retrieval Augmented Recipe Generation

Retrieval Augmented Recipe Generation

引用

2025 IEEE/CVF Winter Conference on Applications of computer Vision, WACV 2025

作者： Liu, Guoshan Yin, Hailong Zhu, Bin Chen, Jingjing Ngo, Chong-Wah Jiang, Yu-Gang School of Computer Science Fudan University Shanghai Key Lab of Intelligent Information Processing China Shanghai Collaborative Innovation Center on Intelligent Visual Computing China Singapore Management University Singapore

ISBN: (纸本)9798331510831

The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method - first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs), which have achieved notable success across a variety of vision and language tasks, shed light on generating both ingredients and instructions directly from images. Nevertheless, LMMs still face the common issue of hallu-cinations during recipe generation, leading to suboptimal performance. To tackle this issue, we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement, integrating them into the prompt to add diverse and rich context to the input image. Additionally, Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates, which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method, which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset. © 2025 IEEE.

关键词： Food ingredients

来源：评论

学校读者我要写书评

暂无评论

SepMLP: An All-MLP Architecture for Music Source Separation 9th

SepMLP: An All-MLP Architecture for Music Source Separation

引用

9th Conference on Sound and Music technology, CSMT 2021

作者： Qian, Jiale Gao, Yongwei Wei, Weixing Zhao, Jiahao Li, Wei School of Computer Science and Technology Fudan University Shanghai200438 China Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai200433 China

ISBN: (纸本)9789811947025

Most previous deep learning based methods use convolutional neural networks (CNNs) or Recurrent neural networks (RNNs) to model the separation process in music signals. In this paper, we propose a MLP-like encoder-decoder architecture, in which per-location features and spatial information in music signals are exclusively handled by multi-layer perceptrons (MLPs). Additionally, We introduce a novel fully-connected decoder for feature aggregation without using skip-connection. The experimental results on the benchmark dataset MUSDB18 show the proposed model achieves comparable performance to previous CNN and RNN based methods, which demonstrates that MLP serves as a promising backbone for the data-driven methods in music source separation. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

Joint AP Clustering and Beamforming Design for RIS-Aided Cell-Free Networks

引用

IEEE Transactions on Vehicular technology 2025年第5期74卷 8315-8320页

作者： Xu, Chunmei Jia, Yuanqi Chen, Youjia Huang, Wei GuildfordGU2 7XH United Kingdom Southeast University School of Information Science and Engineering Nanjing210096 China Fuzhou University Fujian Key Lab for Intelligent Processing and Wireless Transmission of Media Information College of Physics and Information Engineering Fuzhou350108 China Hefei University of Technology School of Computer Science and Information Engineering Hefei230601 China

Cell-free networks and reconfigurable intelligent surfaces (RIS) are two promising techniques for future wireless communications. The integration of RIS into cell-free networks, termed RIS-aided cell-free networks, offers the potential to significantly enhance network performance. However, the realization of this potential is constrained by the limited capacities of the fronthaul links. To address this challenge, we investigate the joint design of access point (AP) clustering, transmit and passive beamforming in RIS-aided cell-free networks. The objective is to maximize the weighted sum-rate performance while minimizing the number of clustered APs to alleviate the fronthaul overhead. The problem is formulated in a group sparse manner, employing a mixed zero-norm/two-norm term to represent the number of the clustered APs. To solve this problem, we first approximate the mixed zero-norm/two-norm term by the mixed one-norm/two-norm term and provide its equivalent formulation by introducing receive beamforming vectors and weight parameters. Then, an iterative method is proposed based on the block coordinate descent (BCD) technique, which is guaranteed to converge to a stationary point. Simulation results demonstrate the effectiveness of the proposed method. © 1967-2012 IEEE.

关键词： Iterative methods

来源：评论

学校读者我要写书评

暂无评论

Boosting Fine-Grained Sketch-Based Image Retrieval with Self-Supervised Learning 48

Boosting Fine-Grained Sketch-Based Image Retrieval with Self...

引用

48th IEEE International Conference on Acoustics, Speech and Signal processing, ICASSP 2023

作者： Zhang, Zhaolong Chen, Yangdong Zhang, Yuejie Feng, Rui Zhang, Tao Fudan University Shanghai Key Lab of Intelligent Information Processing School of Computer Science Shanghai200433 China Shanghai University of Finance and Economics School of Information Management and Engineering Shanghai200433 China

ISBN: (纸本)9781728163277

Fine-grained sketch-based image retrieval (FG-SBIR) aims at aligning images and sketches at the instance level. It is a challenging task as there are significant differences between sketch and image. Existing methods usually produce less desired performance due to the lack of large-scale fine-grained image-sketch datasets and the strong dependence on the classification models pretrained on ImageNet. In this paper, we propose a better self-supervised pre-trained FG-SBIR model which does not depend on large-scale annotated datasets. Only images and their corresponding edge maps are used at the pre-training stage. Mixed modal transformation is designed to generate different mixed-up views. The FG-SBIR model is pre-trained by minimizing the distance between the views of the same instance and then fine-tuned by a simple triplet loss. With a plain downstream network, it achieves generally better performance than state-of-the-art models on three widely used FG-SBIR datasets. © 2023 IEEE.

关键词： Contrastive Learning Self-supervised Learning Sketch Sketch-based Image Retrieval

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：