检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Tao, Zeng Wang, Yan Lin, Junxiong Wang, Haoran Mai, Xinji Yu, Jiawen Tong, Xuan Zhou, Ziheng Yan, Shaoqi Zhao, Qing Han, Liyuan Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China School of Information Science and Technology Fudan University Shanghai China Institute of Automation Chinese Academy of Sciences Beijing China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

The performance of CLIP in dynamic facial expression recognition (DFER) task doesn’t yield exceptional results as observed in other CLIP-based classification tasks. While CLIP’s primary objective is to achieve alignment between images and text in the feature space, DFER poses challenges due to the abstract nature of text and the dynamic nature of video, making label representation limited and perfect alignment difficult. To address this issue, we have designed A3lign-DFER, which introduces a new DFER labeling paradigm to comprehensively achieve alignment, thus enhancing CLIP’s suitability for the DFER task. Specifically, our A3lign-DFER method is designed with multiple modules that work together to obtain the most suitable expanded-dimensional embeddings for classification and to achieve alignment in three key aspects: affective, dynamic, and bidirectional. We replace the input label text with a learnable Multi-Dimensional Alignment Token (MAT), enabling alignment of text to facial expression video samples in both affective and dynamic dimensions. After CLIP feature extraction, we introduce the Joint Dynamic Alignment Synchronizer (JAS), further facilitating synchronization and alignment in the temporal dimension. Additionally, we implement a Bidirectional Alignment Training Paradigm (BAP) to ensure gradual and steady training of parameters for both modalities. Our insightful and concise A3lign-DFER method achieves state-of-the-art results on multiple DFER datasets, including DFEW, FERV39k, and MAFW. Extensive ablation experiments and visualization studies demonstrate the effectiveness of A3lign-DFER. The code will be available in the future. © 2024, CC BY.

关键词： Alignment

来源：评论

学校读者我要写书评

暂无评论

Diabetic Retinopathy Grading with Weakly-Supervised Lesion Priors

Diabetic Retinopathy Grading with Weakly-Supervised Lesion P...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Junlin Hou Fan Xiao Jilan Xu Rui Feng Yuejie Zhang Haidong Zou Lina Lu Wenwen Xue School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China Academy for Engineering and Technology Fudan University Shanghai China Shanghai Eye Diseases Prevention & Treatment Center Shanghai China

Explicit information of lesions can provide visual instructions for diabetic retinopathy (DR) grading on fundus images. However, pixel-level lesion annotations are extremely difficult and time-consuming to acquire. In this work, we propose a novel weakly-supervised lesion-aware network for DR grading, which enhances the discriminative features with lesion priors by only image-level supervision. Specifically, we design a lesion attention module that generates lesion activation maps by introducing an auxiliary task of binary DR identification. Lesion activation maps are utilized to assist the network to focus on the most relevant regions for boosting DR grading performance. Besides, we particularly devise an adaptive joint loss to balance the DR identification and DR grading tasks dynamically. Extensive results on the public DR dataset demonstrate the superiority and generality of our proposed lesion-aware network. The interpretability of generated lesion activation maps is also verified by the comparison with ground truth segmentation masks.

关键词： Visualization Image segmentation Retinopathy Annotations Signal processing Boosting Diabetes

来源：评论

学校读者我要写书评

暂无评论

Lesion-Aware Open Set Medical Image Recognition with Domain Shift

Lesion-Aware Open Set Medical Image Recognition with Domain ...

引用

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Yiqian Xu Rui-Wei Zhao Rui Feng School of Computer Science Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China Academy for Engineering and Technology Fudan University Shanghai China National Children’s Medical Center Children’s Hospital of Fudan University Shanghai China Shanghai Collaborative Innovation Center of Intelligent Visual Computing Shanghai China

Medical image diagnosis in real clinical scenario faces the challenging open set recognition problem with domain shift. Based on the observation that it is often merely the local lesion areas in the whole image decide or cause the true seen or potential unseen disease labels, we propose a unified deep learning framework to address it by establishing a lesion-aware mask generator to guide the model towards focusing on the lesion areas for both seen and unseen diseases, followed by a multi-task feature learning module to extract the lesion image features; and an open set recognizer with improved capability to reject unseen classes. Last but not least, we propose tailored training strategies with mixed up data and mask augmentation for improved OSR and multi-clue cross-domain feature alignment for domain generalization. Through comparative experiments and ablation studies on the representative skin lesion benchmarks, we successfully validate the effectiveness of our proposed framework.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Decoding Continuous Character-based Language from Non-invasive Brain Recordings

arXiv

引用

arXiv 2024年

作者： Zhang, Cenyuan Zheng, Xiaoqing Yin, Ruicheng Geng, Shujie Xu, Jianhan Gao, Xuan Lv, Changze Ling, Zixuan Huang, Xuanjing Cao, Miao Feng, Jianfeng School of Computer Science Fudan University Shanghai China Shanghai Key Laboratory of Intelligent Information Processing Fudan University Shanghai China Institute of Science and Technology for Brain-Inspired Intelligence Fudan University Shanghai China Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence Fudan University Ministry of Education China

Deciphering natural language from brain activity through non-invasive devices remains a formidable challenge. Previous non-invasive decoders either require multiple experiments with identical stimuli to pinpoint cortical regions and enhance signal-to-noise ratios in brain activity, or they are limited to discerning basic linguistic elements such as letters and words. We propose a novel approach to decoding continuous language from single-trial non-invasive fMRI recordings, in which a three-dimensional convolutional network augmented with information bottleneck is developed to automatically identify responsive voxels to stimuli, and a character-based decoder is designed for the semantic reconstruction of continuous language characterized by inherent character structures. The resulting decoder can produce intelligible textual sequences that faithfully capture the meaning of perceived speech both within and across subjects, while existing decoders exhibit significantly inferior performance in cross-subject contexts. The ability to decode continuous language from single trials across subjects demonstrates the promising applications of non-invasive language brain-computer interfaces in both healthcare and neuroscience. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

DISTANCE RESTRICTED TRANSFORMER ENCODER FOR MULTI-labEL CLASSIFICATION

DISTANCE RESTRICTED TRANSFORMER ENCODER FOR MULTI-LABEL CLAS...

引用

2021 IEEE International Conference on Multimedia and Expo, ICME 2021

作者： Wang, Xiaomei Li, Yaqian Luo, Tong Guo, Yandong Fu, Yanwei Xue, Xiangyang School of Computer Science Fudan University Shanghai China OPPO Research Institute Shanghai China The School of Data Science MOE Frontiers Center for Brain Science Shanghai Key Lab of Intelligent Information Processing Fudan University Shanghai China

ISBN: (纸本)9781665438643

Multi-label image classification is a fundamental but challenging task in Multimedia *** aims to predict a set of labels presented in an image. Great progress has been made by exploring convolutional neural network with binary cross-entropy loss recently. However, conventional approaches are limited to highlight the key visual contents associated with target labels and pay little attention to confining the distances between visual and positive/negative label representations. To target these aspects, we firstly introduce a variant transformer encoder model for acquiring the underlying and crucial visual information related to ground truth labels. Specifically, a novel primal feature guided net is designed to maintain the original visual features during encoding process. Secondly, we exploit a distance restricted learning strategy in a common semantic space to shrink the distances of images with positive labels while expand with the negative ones during training stage. Extensive experiments are executed on MSCOCO and WIDER Attribute datasets and outstanding performance is achieved compared with other state-of-the-art models. © 2021 IEEE

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning

arXiv

引用

arXiv 2022年

作者： Wang, Rui Chen, Dongdong Wu, Zuxuan Chen, Yinpeng Dai, Xiyang Liu, Mengchen Yuan, Lu Jiang, Yu-Gang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan Univeristy China Microsoft Cloud + AI

Benefiting from masked visual modeling, self-supervised video representation learning has achieved remarkable progress. However, existing methods focus on learning representations from scratch through reconstructing low-level features like raw pixel RGB values. In this paper, we propose masked video distillation (MVD), a simple yet effective two-stage masked feature modeling framework for video representation learning: firstly we pretrain an image (or video) model by recovering low-level features of masked patches, then we use the resulting features as targets for masked feature modeling. For the choice of teacher models, we observe that students taught by video teachers perform better on temporally-heavy video tasks, while image teachers transfer stronger spatial representations for spatially-heavy video tasks. Visualization analysis also indicates different teachers produce different learned patterns for students. Motivated by this observation, to leverage the advantage of different teachers, we design a spatial-temporal co-teaching method for MVD. Specifically, we distill student models from both video teachers and image teachers by masked feature modeling. Extensive experimental results demonstrate that video transformers pretrained with spatial-temporal co-teaching outperform models distilled with a single teacher on a multitude of video datasets. Our MVD with vanilla ViT achieves state-of-the-art performance compared with previous supervised or self-supervised methods on several challenging video downstream tasks. For example, with the ViT-Large model, our MVD achieves 86.4% and 76.7% Top-1 accuracy on Kinetics-400 and Something-Something-v2, outperforming VideoMAE by 1.2% and 2.4% respectively. When a larger ViT-Huge model is adopted, MVD achieves the state-of-the-art performance with 77.3% Top-1 accuracy on Something- Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

来源：评论

学校读者我要写书评

暂无评论

MOTION MATTERS: A NOVEL MOTION MODELING FOR CROSS-VIEW GAIT FEATURE LEARNING

arXiv

引用

arXiv 2022年

作者： Li, Jingqi Gao, Jiaqi Zhang, Yuzhen Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

As a unique biometric that can be perceived at a distance, gait has broad applications in person authentication, social security and so on. Existing gait recognition methods suffer from changes in viewpoint and clothing and barely consider extracting diverse motion features, a fundamental characteristic in gaits, from gait sequences. This paper proposes a novel motion modeling method to extract the discriminative and robust representation. Specifically, we first extract the motion features from the encoded motion sequences in the shallow layer. Then we continuously enhance the motion feature in deep layers. This motion modeling approach is independent of mainstream work in building network architectures. As a result, one can apply this motion modeling method to any backbone to improve gait recognition performance. In this paper, we combine motion modeling with one commonly used backbone (GaitGL) as GaitGL-M to illustrate motion modeling. Extensive experimental results on two commonly-used cross-view gait datasets demonstrate the superior performance of GaitGL-M over existing state-of-the-art methods. Copyright © 2022, The Authors. All rights reserved.

关键词： Network architecture

来源：评论

学校读者我要写书评

暂无评论

SIAM: A Simple Alternating Mixer for Video Prediction

arXiv

引用

arXiv 2023年

作者： Zheng, Xin Peng, Ziang Cao, Yuan Shan, Hongming Zhang, Junping Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai200433 China Department of Aeronautics and Astronautics Fudan University Shanghai200433 China Institute of Science and Technology for Brain-inspired Intelligence Fudan University Shanghai200433 China

Video prediction, predicting future frames from the previous ones, has broad applications such as autonomous driving and weather forecasting. Existing state-of-the-art methods typically focus on extracting either spatial, temporal, or spatiotemporal features from videos. Different feature focuses, resulting from different network architectures, may make the resultant models excel at some video prediction tasks but perform poorly on others. Towards a more generic video prediction solution, we explicitly model these features in a unified encoder-decoder framework and propose a simple alternating Mixer (SIAM). The novelty of SIAM lies in the design of dimension alternating mixing (DaMi) blocks, which can model spatial, temporal, and spatiotemporal features through alternating the dimensions of the feature maps. Extensive experimental results demonstrate the superior performance of the proposed SIAM on four benchmark video datasets covering both synthetic and real-world scenarios. Copyright © 2023, The Authors. All rights reserved.

关键词： Benchmarking

来源：评论

学校读者我要写书评

暂无评论

PG-Attack: A Precision-Guided Adversarial Attack Framework Against Vision Foundation Models for Autonomous Driving

arXiv

引用

arXiv 2024年

作者： Fu, Jiyuan Chen, Zhaoyu Jiang, Kaixun Guo, Haijing Gao, Shuyong Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University China

Vision foundation models are increasingly employed in autonomous driving systems due to their advanced capabilities. However, these models are susceptible to adversarial attacks, posing significant risks to the reliability and safety of autonomous vehicles. Adversaries can exploit these vulnerabilities to manipulate the vehicle’s perception of its surroundings, leading to erroneous decisions and potentially catastrophic consequences. To address this challenge, we propose a novel Precision-Guided Adversarial Attack (PG-Attack) framework that combines two techniques: Precision Mask Perturbation Attack (PMP-Attack) and Deceptive Text Patch Attack (DTP-Attack). PMP-Attack precisely targets the attack region to minimize the overall perturbation while maximizing its impact on the target object’s representation in the model’s feature space. DTP-Attack introduces deceptive text patches that disrupt the model’s understanding of the scene, further enhancing the attack’s effectiveness. Our experiments demonstrate that PG-Attack successfully deceives a variety of advanced multi-modal large models, including GPT-4V, Qwen-VL, and imp-V1. Additionally, we won First-Place in the CVPR 2024 Workshop Challenge: Black-box Adversarial Attacks on Vision Foundation Models‡ and codes are available at https://***/fuhaha824/PG-Attack. Copyright © 2024, The Authors. All rights reserved.

关键词： Autonomous vehicles

来源：评论

学校读者我要写书评

暂无评论

FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network 39

FaceA-Net: Facial Attribute-Driven ID Preserving Image Gener...

引用

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Wang, Jiayu Yu, Yue Chen, Jingjing Dai, Qi Jiang, Yu-Gang Shanghai Key Lab of Intell. Info. Processing School of Computer Science Fudan University China Shanghai Collaborative Innovation Center of Intelligent Visual Computing China Microsoft Research Asia China

ISBN: (纸本)157735897X

Recent advances in diffusion-based generative models have demonstrated superior performance in subject-driven image generation. Identity (ID) preserving image generation, as a subtask of subject-driven image generation, aims to generate customized images for specific human identity and has broad application potential. However, this task remains challenging due to the requirement for high ID fidelity and precise detail preservation. Additionally, generating high-quality context presents another challenge, as existing methods struggle to achieve both high ID fidelity and satisfactory context simultaneously. To address the issues of insufficient ID fidelity, we introduce a simple yet effective test-time fine-tuning approach. Specifically, we propose an attribute-driven training method that establishes global-level and local-level tasks to learn the global face feature and fine-grained attribute features, respectively. Furthermore, we introduce a novel ID-context decoupling framework that decouples image context generation from human ID generation, ensuring the quality of contextual content as well as facilitating the learning of ID information. Through extensive experiments, we demonstrate the effectiveness of the proposed method and showcase its capabilities across various applications. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：