检索结果-内蒙古大学图书馆

MFAE: Masked frame-level autoencoder with hybrid-supervision for low-resource music transcription

学校读者我要写书评

暂无评论

MFAE: Masked frame-level autoencoder with hybrid-supervision...

IEEE International Conference on Multimedia and Expo (ICME)

作者： Yulun Wu Jiahao Zhao Yi Yu Wei Li School of Computer Science and Technology Fudan University Shanghai China Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China

Automantic Music Transcription (AMT) is an essential topic in music information retrieval (MIR), and it aims to transcribe audio recordings into symbolic representations. Recently, large-scale piano datasets with high-quality notations have been proposed for high-resolution piano transcription, which resulted in domain-specific AMT models achieved state-of- the-art results. However, those methods are hardly generalized to other ’low-resource’ instruments (such as guitar, cello, clarinet, etc.) transcription. In this paper, we propose a hybrid-supervised framework, the masked frame-level autoencoder (MFAE), to solve this issue. The proposed MFAE reconstructs the frame-level features of low-resource data to understand generic representations of low-resource instruments and improves low-resource transcription performance. Experimental results on several low- resource datasets (MAPS, MusicNet, and Guitarset) show that our framework achieves state-of-the-art performance in note-wise scores (Note F1 83.4%\64.1%\86.7%, Note-with-offset F1 59.8%\41.4%\71.6%). Moreover, our framework can be well generalized to various genres of instrument transcription, both in data-plentiful and data-limited scenarios.

关键词：

On the Universal Adversarial Perturbations for Efficient Data-free Adversarial Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gao, Songyang Dou, Shihan Zhang, Qi Huang, Xuanjing Ma, Jin Shan, Ying School of Computer Science Fudan University Shanghai China Shanghai Key Laboratory of Intelligent Information Processing Shanghai China Tencent PCG China

Detecting adversarial samples that are carefully crafted to fool the model is a critical step to socially-secure applications. However, existing adversarial detection methods require access to sufficient training data, which brings noteworthy concerns regarding privacy leakage and generalizability. In this work, we validate that the adversarial sample generated by attack algorithms is strongly related to a specific vector in the high-dimensional inputs. Such vectors, namely UAPs (Universal Adversarial Perturbations), can be calculated without original training data. Based on this discovery, we propose a data-agnostic adversarial detection framework, which induces different responses between normal and adversarial samples to UAPs. Experimental results show that our method achieves competitive detection performance on various text classification tasks, and maintains an equivalent time consumption to normal inference. © 2023, CC BY-SA.

关键词： Classification (of information)

PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Zhang, Mozhi Yan, Hang Zhou, Yaqian Qiu, Xipeng School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing Fudan University China

Few-shot Named Entity Recognition (NER) is a task aiming to identify named entities via limited annotated samples. Recently, prototypical networks have shown promising performance in few-shot NER. Most of prototypical networks will utilize the entities from the support set to construct label prototypes and use the query set to compute span-level similarities and optimize these label prototype representations. However, these methods are usually unsuitable for fine-tuning in the target domain, where only the support set is available. In this paper, we propose PromptNER: a novel prompting method for few-shot NER via k nearest neighbor search. We use prompts that contains entity category information to construct label prototypes, which enables our model to fine-tune with only the support set. Our approach achieves excellent transfer learning ability, and extensive experiments on the Few-NERD and CrossNER datasets demonstrate that our model achieves superior performance over state-of-the-art methods. Copyright © 2023, The Authors. All rights reserved.

关键词： Nearest neighbor search

OneVOS: Unifying Video Object Segmentation with All-in-One Transformer Framework

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Wanyun Guo, Pinxue Zhou, Xinyu Hong, Lingyi He, Yangji Zheng, Xiangyu Zhang, Wei Zhang, Wenqiang School of Computer Science Fudan University China Shanghai Key Laboratory of Intelligent Information Processing China Academy for Engineering and Technology Fudan University China

Contemporary Video Object Segmentation (VOS) approaches typically consist stages of feature extraction, matching, memory management, and multiple objects aggregation. Recent advanced models either employ a discrete modeling for these components in a sequential manner, or optimize a combined pipeline through substructure aggregation. However, these existing explicit staged approaches prevent the VOS framework from being optimized as a unified whole, leading to the limited capacity and suboptimal performance in tackling complex videos. In this paper, we propose OneVOS, a novel framework that unifies the core components of VOS with All-in-One Transformer. Specifically, to unify all aforementioned modules into a vision transformer, we model all the features of frames, masks and memory for multiple objects as transformer tokens, and integrally accomplish feature extraction, matching and memory management of multiple objects through the flexible attention mechanism. Furthermore, a Unidirectional Hybrid Attention is proposed through a double decoupling of the original attention operation, to rectify semantic errors and ambiguities of stored tokens in OneVOS framework. Finally, to alleviate the storage burden and expedite inference, we propose the Dynamic Token Selector, which unveils the working mechanism of OneVOS and naturally leads to a more efficient version of OneVOS. Extensive experiments demonstrate the superiority of OneVOS, achieving state-of-the-art performance across 7 datasets, particularly excelling in complex LVOS and MOSE datasets with 70.1% and 66.4% J&F scores, surpassing previous state-of-the-art methods by 4.2% and 7.0%, respectively. And our code will be available for reproducibility and further research. Copyright © 2024, The Authors. All rights reserved.

关键词： Feature extraction

From Node Interaction to Hop Interaction: New Effective and Scalable Graph Learning Paradigm

学校读者我要写书评

暂无评论

From Node Interaction to Hop Interaction: New Effective and ...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Jie Chen Zilong Li Yin Zhu Junping Zhang Jian Pu Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Institute of Science and Technology for Brain-Inspired Intelligence Fudan University Shanghai China

Existing Graph Neural Networks (GNNs) follow the message-passing mechanism that conducts information interaction among nodes iteratively. While considerable progress has been made, such node interaction paradigms still have the following limitation. First, the scalability limitation precludes the broad application of GNNs in large-scale industrial settings since the node interaction among rapidly expanding neighbors incurs high computation and memory costs. Second, the over-smoothing problem restricts the discrimination ability of nodes, i.e., node representations of different classes will converge to indistinguishable after repeated node interactions. In this work, we propose a novel hop interaction paradigm to address these limitations simultaneously. The core idea is to convert the interaction target among nodes to pre-processed multi-hop features inside each node. We design a simple yet effective HopGNN framework that can easily utilize existing GNNs to achieve hop interaction. Furthermore, we propose a multi-task learning strategy with a self-supervised learning objective to enhance HopGNN. We conduct extensive experiments on 12 benchmark datasets in a wide range of domains, scales, and smoothness of graphs. Experimental results show that our methods achieve superior performance while maintaining high scalability and efficiency. The code is at https://***/JC-202/HopGNN.

关键词：

AUCSeg: AUC-oriented pixel-level long-tail semantic segmentation 24

学校读者我要写书评

暂无评论

AUCSeg: AUC-oriented pixel-level long-tail semantic segmenta...

Proceedings of the 38th International Conference on Neural information processing Systems

作者： Boyu Han Qianqian Xu Zhiyong Yang Shilong Bao Peisong Wen Yangbangyan Jiang Qingming Huang Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS and School of Computer Science and Tech. University of Chinese Academy of Sciences Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS and Peng Cheng Laboratory School of Computer Science and Tech. University of Chinese Academy of Sciences School of Computer Science and Tech. University of Chinese Academy of Sciences and Key Lab. of Intelligent Information Processing Institute of Computing Technology CAS and Key Laboratory of Big Data Mining and Knowledge Management CAS

ISBN: (纸本)9798331314385

The Area Under the ROC Curve (AUC) is a well-known metric for evaluating instance-level long-tail learning problems. In the past two decades, many AUC optimization methods have been proposed to improve model performance under long-tail distributions. In this paper, we explore AUC optimization methods in the context of pixel-level long-tail semantic segmentation, a much more complicated scenario. This task introduces two major challenges for AUC optimization techniques. On one hand, AUC optimization in a pixel-level task involves complex coupling across loss terms, with structured inner-image and pairwise inter-image dependencies, complicating theoretical analysis. On the other hand, we find that mini-batch estimation of AUC loss in this case requires a larger batch size, resulting in an unaffordable space complexity. To address these issues, we develop a pixel-level AUC loss function and conduct a dependency-graph-based theoretical analysis of the algorithm's generalization ability. Additionally, we design a Tail-Classes Memory Bank (T-Memory Bank) to manage the significant memory demand. Finally, comprehensive experiments across various benchmarks confirm the effectiveness of our proposed AUCSeg method. The code is available at https://***/boyuh/AUCSeg.

关键词：

StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

学校读者我要写书评

暂无评论

StyleAdv: Meta Style Adversarial Training for Cross-Domain F...

Conference on computer Vision and Pattern Recognition (CVPR)

作者： Yuqian Fu Yu Xie Yanwei Fu Yu-Gang Jiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Purple Mountain Laboratories Nanjing China School of Data Science Fudan University

Cross-Domain Few-Shot Learning (CD-FSL) is a recently emerging task that tackles few-shot learning across different domains. It aims at transferring prior knowledge learned on the source dataset to novel target datasets. The CD-FSL task is especially challenged by the huge domain gap between different datasets. Critically, such a domain gap actually comes from the changes of visual styles, and wave-SAN [10] empirically shows that spanning the style distribution of the source data helps alleviate this issue. However, wave-SAN simply swaps styles of two images. Such a vanilla operation makes the generated styles “real” and “easy”, which still fall into the original set of the source styles. Thus, inspired by vanilla adversarial learning, a novel model-agnostic meta Style Adversarial training (StyleAdv) method together with a novel style adversarial attack method is proposed for CD-FSL. Particularly, our style attack method synthesizes both “virtual” and “hard” adversarial styles for model training. This is achieved by perturbing the original style with the signed style gradients. By continually attacking styles and forcing the model to recognize these challenging adversarial styles, our model is gradually robust to the visual styles, thus boosting the generalization ability for novel target datasets. Besides the typical CNN-based backbone, we also employ our StyleAdv method on large-scale pre-trained vision transformer. Extensive experiments conducted on eight various target datasets show the effectiveness of our method. Whether built upon ResNet or ViT, we achieve the new state of the art for CD-FSL. Code is available at https://***/lovelyqian/StyleAdv-CDFSL.

关键词：

LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Model

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Cao, Chenjie Cai, Yunuo Dong, Qiaole Wang, Yikai Fu, Yanwei School of Data Science Fudan University China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Alibaba Group China

This paper introduces LeftRefill, an innovative approach to efficiently harness large Text-to-Image (T2I) diffusion models for reference-guided image synthesis. As the name implies, LeftRefill horizontally stitches reference and target views together as a whole input. The reference image occupies the left side, while the target canvas is positioned on the right. Then, LeftRefill paints the right-side target canvas based on the left-side reference and specific task instructions. Such a task formulation shares some similarities with contextual inpainting, akin to the actions of a human painter. This novel formulation efficiently learns both structural and textured correspondence between reference and target without other image encoders or adapters. We inject task and view information through cross-attention modules in T2I models, and further exhibit multi-view reference ability via the re-arranged self-attention modules. These enable LeftRefill to perform consistent generation as a generalized model without requiring test-time fine-tuning or model modifications. Thus, LeftRefill can be seen as a simple yet unified framework to address reference-guided synthesis. As an exemplar, we leverage LeftRefill to address two different challenges: reference-guided inpainting and novel view synthesis, based on the pre-trained StableDiffusion. Codes&models are released at https://***/ewrfcas/LeftRefill. © 2023, CC BY-NC-SA.

关键词： Textures

LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Hong, Lingyi Liu, Zhongying Chen, Wenchao Tan, Chenzhi Feng, Yuang Zhou, Xinyu Guo, Pinxue Li, Jinglun Chen, Zhaoyu Gao, Shuyong Zhang, Wei Zhang, Wenqiang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China The Shanghai Engineering Research Center of AI&Robotics Academy for Engineering&Technology Fudan University Shanghai China Engineering Research Center of AI&Robotics Ministry of Education Academy for Engineering&Technology Fudan University Shanghai China The Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

Video object segmentation (VOS) aims to distinguish and track target objects in a video. Despite the excellent performance achieved by off-the-shell VOS models, existing VOS benchmarks mainly focus on short-term videos lasting about 5 seconds, where objects remain visible most of the time. However, these benchmarks poorly represent practical applications, and the absence of long-term datasets restricts further investigation of VOS in realistic scenarios. Thus, we propose a novel benchmark named LVOS, comprising 720 videos with 296,401 frames and 407,945 high-quality annotations. Videos in LVOS last 1.14 minutes on average, approximately 5 times longer than videos in existing datasets. Each video includes various attributes, especially challenges deriving from the wild, such as long-term reappearing and cross-temporal similar objects. Compared to previous benchmarks, our LVOS better reflects VOS models' performance in real scenarios. Based on LVOS, we evaluate 20 existing VOS models under 4 different settings and conduct a comprehensive analysis. On LVOS, these models suffer a large performance drop, highlighting the challenge of achieving precise tracking and segmentation in real-world scenarios. Attribute-based analysis indicates that key factor to accuracy decline is the increased video length, emphasizing LVOS's crucial role. We hope our LVOS can advance development of VOS in real scenes. Data and code are available at https://***/lvos. ***/. © 2024, CC BY.

关键词： Benchmarking