检索结果-内蒙古大学图书馆

arXiv 2023年

作者： Weng, Zejia Yang, Xitong Li, Ang Wu, Zuxuan Jiang, Yu-Gang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Meta AI United States

Contrastive Language-Image Pretraining (CLIP) has demonstrated impressive zero-shot learning abilities for image understanding, yet limited effort has been made to investigate CLIP for zero-shot video recognition. We introduce Open-VCLIP, a simple yet effective approach that transforms CLIP into a strong zero-shot video classifier that can recognize unseen actions and events at test time. Our framework extends CLIP with minimal modifications to model spatial-temporal relationships in videos, making it a specialized video classifier, while striving for generalization. We formally show that training an Open-VCLIP is equivalent to continual learning with zero historical data. To address this problem, we propose Interpolated Weight Optimization, which utilizes the benefit of weight interpolation in both training and test time. We evaluate our method on three popular and challenging action recognition datasets following various zero-shot evaluation protocols and we demonstrate our approach outperforms state-of-the-art methods by clear margins. In particular, we achieve 87.9%, 58.3%, 81.1% zero-shot accuracy on UCF, HMDB and Kinetics-600 respectively, outperforming state-of-the-art methods by 8.3%, 7.8% and 12.2%. Code is released at https://***/wengzejia1/Open-VCLIP. Copyright © 2023, The Authors. All rights reserved.

关键词： Zero-shot learning

来源：评论

学校读者我要写书评

暂无评论

Tail Classes Matter: Long-Tailed Object Detection Revisited

Tail Classes Matter: Long-Tailed Object Detection Revisited

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Yinglu Zhang Chenbo Zhang Lu Zhang Tianying Liu Jihong Guan Xinkai Liang Jiajia Zhao Shuigeng Zhou Shanghai Key Lab of Intelligent Information Processing and School of Computer Science Fudan University China Department of Computer Science & Technology Tongji University China Science and Technology on Complex System Conroland InelligentAgent Cooperation Laboratory Beijing Electro-Mechanical Engineering Institute China

Real-world data ubiquitously exhibit long-tailed distribution, which sparks the increasing interest in long-tailed object detection (LTOD). However, existing methods neglect that a lack of diverse data in tail classes will cause underrepresented tail class features, making their efforts for balancing foreground classes tend to over-fit tail classes and be less effective. In this paper, we propose a multi-class co-attention generation network to increase data diversity of tail classes by generating augmented samples. To alleviate imbalance, we develop a distribution-aware up-sampling strategy, performing differential up-sampling for different classes and design a bi-directional regulation loss to adjust both positive and negative gradients. Moreover, we construct a new dataset LVIS-X with more rare classes based on existing LTOD benchmark dataset LVIS. Experiments on LVIS and LVIS-X demonstrate the superiority of the proposed method.

关键词：

来源：评论

学校读者我要写书评

暂无评论

DISTANCE RESTRICTED TRANSFORMER ENCODER FOR MULTI-labEL CLASSIFICATION

DISTANCE RESTRICTED TRANSFORMER ENCODER FOR MULTI-LABEL CLAS...

引用

2021 IEEE International Conference on Multimedia and Expo, ICME 2021

作者： Wang, Xiaomei Li, Yaqian Luo, Tong Guo, Yandong Fu, Yanwei Xue, Xiangyang School of Computer Science Fudan University Shanghai China OPPO Research Institute Shanghai China The School of Data Science MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China

ISBN: (纸本)9781665438643

Multi-label image classification is a fundamental but challenging task in Multimedia *** aims to predict a set of labels presented in an image. Great progress has been made by exploring convolutional neural network with binary cross-entropy loss recently. However, conventional approaches are limited to highlight the key visual contents associated with target labels and pay little attention to confining the distances between visual and positive/negative label representations. To target these aspects, we firstly introduce a variant transformer encoder model for acquiring the underlying and crucial visual information related to ground truth labels. Specifically, a novel primal feature guided net is designed to maintain the original visual features during encoding process. Secondly, we exploit a distance restricted learning strategy in a common semantic space to shrink the distances of images with positive labels while expand with the negative ones during training stage. Extensive experiments are executed on MSCOCO and WIDER Attribute datasets and outstanding performance is achieved compared with other state-of-the-art models. © 2021 IEEE

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning

arXiv

引用

arXiv 2022年

作者： Wang, Rui Chen, Dongdong Wu, Zuxuan Chen, Yinpeng Dai, Xiyang Liu, Mengchen Yuan, Lu Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan Univeristy China Microsoft Cloud + AI

Benefiting from masked visual modeling, self-supervised video representation learning has achieved remarkable progress. However, existing methods focus on learning representations from scratch through reconstructing low-level features like raw pixel RGB values. In this paper, we propose masked video distillation (MVD), a simple yet effective two-stage masked feature modeling framework for video representation learning: firstly we pretrain an image (or video) model by recovering low-level features of masked patches, then we use the resulting features as targets for masked feature modeling. For the choice of teacher models, we observe that students taught by video teachers perform better on temporally-heavy video tasks, while image teachers transfer stronger spatial representations for spatially-heavy video tasks. Visualization analysis also indicates different teachers produce different learned patterns for students. Motivated by this observation, to leverage the advantage of different teachers, we design a spatial-temporal co-teaching method for MVD. Specifically, we distill student models from both video teachers and image teachers by masked feature modeling. Extensive experimental results demonstrate that video transformers pretrained with spatial-temporal co-teaching outperform models distilled with a single teacher on a multitude of video datasets. Our MVD with vanilla ViT achieves state-of-the-art performance compared with previous supervised or self-supervised methods on several challenging video downstream tasks. For example, with the ViT-Large model, our MVD achieves 86.4% and 76.7% Top-1 accuracy on Kinetics-400 and Something-Something-v2, outperforming VideoMAE by 1.2% and 2.4% respectively. When a larger ViT-Huge model is adopted, MVD achieves the state-of-the-art performance with 77.3% Top-1 accuracy on Something- Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

Inspire the Large Language Model by External Knowledge on BioMedical Named Entity Recognition

arXiv

引用

arXiv 2023年

作者： Bian, Junyi Zheng, Jiaxuan Zhang, Yuyi Zhu, Shanfeng School of Computer Science Fudan University Shanghai200433 China Institute of Science and Technology for Brain-Inspired Intelligence Fudan University China Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai200433 China

Large language models (LLMs) have demonstrated dominating performance in many NLP tasks, especially on generative tasks. However, they often fall short in some information extraction tasks, particularly those requiring domain-specific knowledge, such as Biomedical Named Entity Recognition (NER). In this paper, inspired by Chain-of-thought, we leverage the LLM to solve the Biomedical NER step-by-step: break down the NER task into entity span extraction and entity type determination. Additionly, for entity type determination, we inject entity knowledge to address the problem that LLM’s lack of domain knowledge when predicting entity category. Experimental results show a significant improvement in our two-step BioNER approach compared to previous few-shot LLM baseline. Additionally, the incorporation of external knowledge significantly enhances entity category determination performance. Copyright © 2023, The Authors. All rights reserved.

关键词： Domain Knowledge

来源：评论

学校读者我要写书评

暂无评论

FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network 39

FaceA-Net: Facial Attribute-Driven ID Preserving Image Gener...

引用

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Wang, Jiayu Yu, Yue Chen, Jingjing Dai, Qi Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Microsoft Research Asia China

ISBN: (纸本)157735897X

Recent advances in diffusion-based generative models have demonstrated superior performance in subject-driven image generation. Identity (ID) preserving image generation, as a subtask of subject-driven image generation, aims to generate customized images for specific human identity and has broad application potential. However, this task remains challenging due to the requirement for high ID fidelity and precise detail preservation. Additionally, generating high-quality context presents another challenge, as existing methods struggle to achieve both high ID fidelity and satisfactory context simultaneously. To address the issues of insufficient ID fidelity, we introduce a simple yet effective test-time fine-tuning approach. Specifically, we propose an attribute-driven training method that establishes global-level and local-level tasks to learn the global face feature and fine-grained attribute features, respectively. Furthermore, we introduce a novel ID-context decoupling framework that decouples image context generation from human ID generation, ensuring the quality of contextual content as well as facilitating the learning of ID information. Through extensive experiments, we demonstrate the effectiveness of the proposed method and showcase its capabilities across various applications. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Optical Character Recognition Using Optimized Convolutional Networks*

Optical Character Recognition Using Optimized Convolutional ...

引用

International Conference on Fog and Mobile Edge Computing (FMEC)

作者： Anum Nawaz Muhammad Irfan Tomi Westerlund Shanghai Key Laboratory of Intelligent Information Processing School of Computer ScienceFudan University Shanghai China Turku Intelligent and Embedded Robotic Systems Lab Faculty of Technology University of Turku Turku Finland

Optical Character Recognition (OCR) has been a prominent area of research in pattern recognition for several decades, owing to its broad application potential in smart living. To improve offline OCR on mobile devices with limited computing resource, we have optimized Convolutional Neural Networks (CNNs) to efficiently detect text using minimal resources. To achieve this, we employed two distinct pretrained CNN models, namely AlexNet and Inception-V3, for feature extraction. Leveraging these models' unique characteristics and capabilities to extract diverse features, we aimed to enhance the classifier's accuracy. This, in turn, facilitates the development of an efficient edge-device application for faster and higher-quality OCR. Experimental results demonstrate that our proposed optimized algorithm outperforms existing CNN-based methods in the field of OCR, particularly in the categorization and detection of handwritten digits and character recognition. The conducted research yielded impressive accuracy results, with up to 97% accuracy on the MNIST dataset and 95.5% accuracy on the NIST dataset.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Text information aggregation with centrality attention

引用

science China(information sciences) 2021年第12期64卷 91-103页

作者： Jingjing GONG Hang YAN Yining ZHENG Qipeng GUO Xipeng QIU Xuanjing HUANG School of Computer Science Fudan University Shanghai Key Laboratory of Intelligent Information Processing Fudan University

A lot of natural language processing problems need to encode the text sequence as a fix-length vector, which usually involves an aggregation process of combining the representations of all the words, such as pooling or self-attention. However, these widely used aggregation approaches do not take higher-order relationships among the words into consideration. Hence we propose a new way of obtaining aggregation weights, called eigen-centrality self-attention. More specifically, we build a fully-connected graph for all the words in a sentence, then compute the eigen-centrality as the attention score of each word. The explicit modeling of relationships as a graph is able to capture some higher-order dependency among words, which helps us achieve better results in 5 text classification tasks and one SNLI task than baseline models such as pooling, self-attention, and dynamic routing. Besides, in order to compute the dominant eigenvector of the graph, we adopt a power method algorithm to get the eigen-centrality measure. Moreover, we also derive an iterative approach to get the gradient for the power method process to reduce both memory consumption and computation requirement.

关键词： information aggregation eigen centrality text classification natural language processing deep learning

来源：评论

学校读者我要写书评

暂无评论

Trajectory set Empowered Hypergraph Transformer for Mobile Sensor Based Traffic Prediction

Trajectory set Empowered Hypergraph Transformer for Mobile S...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Hanyuan Zhang Xinyu Zhang Qize Jiang Liang Li Baihua Zheng Weiwei Sun School of Computer Science Shanghai Key Laboratory of Data Science Fudan University Shanghai Institute of Intelligent Electronics & Systems School of Computing and Information Systems Singapore Management University

Traffic speed prediction is vital for intelligent transportation systems. However, most existing methods focus on costly static sensors. In contrast, utilizing GPS devices from vehicles as mobile sensors offers a cost-effective means to gather dynamic traffic data. Despite the presence of historical trajectory data, mobile sensor-based traffic prediction remains under-explored. Existing methods often treat trajectories as substitutes for static sensors, missing the full utilization of the spatial-temporal signals within the complete trajectory set. To address this, we propose TrajHGT, a novel trajectory set empowered hypergraph transformer model that captures trafficrelated spatial-temporal features through adaptive attention and fusion mechanisms in both the trajectory hypergraph space and the road graph space. Real dataset experiments demonstrate the superiority of TrajHGT.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Compositional Scene Representation Learning via Reconstruction: A Survey

arXiv

引用

arXiv 2022年

作者： Yuan, Jinyang Chen, Tonglin Li, Bin Xue, Xiangyang The Shanghai Key Laboratory of Intelligent Information Processing The School of Computer Science Fudan University Shanghai200433 China

Visual scenes are composed of visual concepts and have the property of combinatorial explosion. An important reason for humans to efficiently learn from diverse visual scenes is the ability of compositional perception, and it is desirable for artificial intelligence to have similar abilities. Compositional scene representation learning is a task that enables such abilities. In recent years, various methods have been proposed to apply deep neural networks, which have been proven to be advantageous in representation learning, to learn compositional scene representations via reconstruction, advancing this research direction into the deep learning era. Learning via reconstruction is advantageous because it may utilize massive unlabeled data and avoid costly and laborious data annotation. In this survey, we first outline the current progress on reconstruction-based compositional scene representation learning with deep neural networks, including development history and categorizations of existing methods from the perspectives of the modeling of visual scenes and the inference of scene representations;then provide benchmarks, including an open source toolbox to reproduce the benchmark experiments, of representative methods that consider the most extensively studied problem setting and form the foundation for other methods;and finally discuss the limitations of existing methods and future directions of this research topic. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：