检索结果-内蒙古大学图书馆

A Weakly Supervised and Self-Supervised Learning Approach for Semantic Segmentation of Land Cover in Satellite Images with National Forest Inventory Data

引用

REMOTE SENSING 2025年第4期17卷 711-711页

作者： Moraes, Daniel Campagnolo, Manuel L. Caetano, Mario Univ Nova Lisboa NOVA Informat Management Sch NOVA IMS Campus Campolide P-1070312 Lisbon Portugal Direcao Geral Terr Rua Artilharia 107 P-1099052 Lisbon Portugal Univ Lisbon Forest Res Ctr CEF Sch Agr Associate Lab TERRA P-1349017 Tapada Da Ajuda Lisbon Portugal

National Forest Inventories (NFIs) provide valuable land cover (LC) information but often lack spatial continuity and an adequate update frequency. Satellite-based remote sensing offers a viable alternative, employing machine learning to extract thematic data. State-of-the-art methods such as convolutional neural networks rely on fully pixel-level annotated images, which are difficult to obtain. Although reference LC datasets have been widely used to derive annotations, NFIs consist of point-based data, providing only sparse annotations. Weakly supervised and self-supervised learning approaches help address this issue by reducing dependence on fully annotated images and leveraging unlabeled data. However, their potential for large-scale LC mapping needs further investigation. This study explored the use of NFI data with deep learning and weakly supervised and self-supervised methods. Using Sentinel-2 images and the Portuguese NFI, which covers other LC types beyond forest, as sparse labels, we performed weakly supervised semantic segmentation with a convolutional neural network to create an updated and spatially continuous national LC map. Additionally, we investigated the potential of self-supervised learning by pretraining a masked autoencoder on 65,000 Sentinel-2 image chips and then fine-tuning the model with NFI-derived sparse labels. The weakly supervised baseline achieved a validation accuracy of 69.60%, surpassing Random Forest (67.90%). The self-supervised model achieved 71.29%, performing on par with the baseline using half the training data. The results demonstrated that integrating both learning approaches enabled successful countrywide LC mapping with limited training data.

关键词： land cover deep learning convolutional neural networks masked autoencoder national forest inventory sentinel-2

来源：评论

学校读者我要写书评

暂无评论

SkelETT-Skeleton-to-Emotion Transfer Transformer

引用

IEEE ACCESS 2025年 13卷 23344-23358页

作者： Paiva, Pedro Victor Vieira Ramos, Josue Junior Guimaraes Gavrilova, Marina de Carvalho, Marco Antonio Garcia Univ Estadual Campinas Sch Technol BR-13484332 Limeira Brazil Univ Calgary Dept Comp Sci Calgary AB T2N 1N4 Canada Renato Archer IT Ctr BR-13069901 Campinas Brazil

Emotion recognition plays an essential role in human-computer interaction, spanning diverse domains from human-robot communication and virtual reality to mental health assessment and affective computing. Traditionally, this field has heavily relied on visual and auditory cues, such as facial expressions and speech analysis. However, these modalities alone may not comprehensively capture the full spectrum of human emotion and suffer limitations due to noise or occlusion. Human skeletons, derived from depth sensors or pose estimation algorithms, offer an alternative for facial expression, including valuable spatial and temporal cues. In this paper, we introduce a novel approach to emotion recognition by pre-training a transformer model on a large dataset of unsupervised human skeleton representations and subsequently fine-tuning it for emotion classification. By exposing the model to an extensive corpus of unlabeled human skeleton data, we can effectively learn to represent complex spatial and temporal dependencies inherent in body movements. Following this foundational knowledge acquisition, the model undergoes fine-tuning on a smaller, labeled dataset tailored for emotion classification tasks. We introduce SkelETT, an encoder-only transformer architecture for body emotion recognition. Comprising a series of encoder layers, SkelETT patches 2D body pose representations, it also includes multi-head self-attention mechanisms and position-wise feed-forward networks, providing a powerful framework for extracting hierarchical features from sequential body pose data. We propose and evaluate the impact of different fine-tuning strategies on pose data using the MPOSE action recognition dataset as a pre-training source. Transfer performance is measured on the BoLD body emotion recognition dataset. Compared to the state-of-the-art, we report significant gains in accuracy (approximate to 34% higher), training time (approximate to 50% less), and model complexity reduction (approx

关键词： Emotion recognition Transformers Skeleton Adaptation models Accuracy Computational modeling Transfer learning Training Solid modeling Data models Attention-based design body emotion recognition gait analysis masked autoencoder affective computing

来源：评论

学校读者我要写书评

暂无评论

VIS-MAE: An Efficient Self-supervised Learning Approach on Medical Image Segmentation and Classification 15th

VIS-MAE: An Efficient Self-supervised Learning Approach on M...

引用

15th International Workshop on Machine Learning in Medical Imaging

作者： Liu, Zelong Tieu, Andrew Patel, Nikhil Soultanidis, George Deyer, Louisa Wang, Ying Huver, Sean Zhou, Alexander Mei, Yunhao Fayad, Zahi A. Deyer, Timothy Mei, Xueyan Icahn Sch Med Mt Sinai BioMed Engn & Imaging Inst New York NY 10029 USA Univ Oklahoma Dept Math Norman OK 73019 USA NVIDIA Santa Clara CA USA Erasmus Univ Rotterdam Netherlands East River Med Imaging New York NY USA Cornell Med Dept Radiol New York NY USA Icahn Sch Med Mt Sinai Windreich Dept Artificial Intelligence & Human Hl New York NY 10029 USA

ISBN: (纸本)9783031732928;9783031732904

Artificial Intelligence (AI) has the potential to revolutionize diagnosis and segmentation in medical imaging. However, development and clinical implementation face multiple challenges including limited data availability, lack of generalizability, and the necessity to incorporate multi-modal data effectively. A foundation model, which is a large-scale pre-trained AI model, offers a versatile base that can be adapted to a variety of specific tasks and contexts. Here, we present VIsualization and Segmentation masked autoencoder (VIS-MAE), novel model weights specifically designed for medical imaging. Specifically, VIS-MAE is trained on a dataset of 2.5 million unlabeled images from various modalities (CT, MR, PET, X-rays, and ultrasound), using self-supervised learning techniques. It is then adapted to classification and segmentation tasks using explicit labels. VIS-MAE has high label efficiency, outperforming several benchmark models in both in-domain and out-of-domain applications. In addition, VIS-MAE has improved label efficiency as it can achieve similar performance to other models with a reduced amount of labeled training data (50% or 80%) compared to other pre-trained weights. VIS-MAE represents a significant advancement in medical imaging AI, offering a generalizable and robust solution for improving segmentation and classification tasks while reducing the data annotation workload. The source code of this work is available at https://***/lzl199704/VIS-MAE.

关键词： Self-supervised Learning masked autoencoder Medical Image Segmentation and Classification Label efficiency

来源：评论

学校读者我要写书评

暂无评论

Pre-training with Synthetic Patterns for Audio

Pre-training with Synthetic Patterns for Audio

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Ishikawa, Yuchi Komatsu, Tatsuya Aoki, Yoshimitsu LY Corporation Tokyo Japan Keio University Kanagawa Japan

ISBN: (纸本)9798350368741

In this paper, we propose to pre-train audio encoders using synthetic patterns instead of real audio data. Our proposed framework consists of two key elements. The first one is masked autoencoder (MAE), a self-supervised learning framework that learns from reconstructing data from randomly masked counterparts. MAEs tend to focus on low-level information such as visual patterns and regularities within data. Therefore, it is unimportant what is portrayed in the input, whether it be images, audio mel-spectrograms, or even synthetic patterns. This leads to the second key element, which is synthetic data. Synthetic data, unlike real audio, is free from privacy and licensing infringement issues. By combining MAEs and synthetic patterns, our framework enables the model to learn generalized feature representations without real data, while addressing the issues related to real audio. To evaluate the efficacy of our framework, we conduct extensive experiments across a total of 13 audio tasks and 17 synthetic datasets. The experiments provide insights into which types of synthetic patterns are effective for audio. Our results demonstrate that our framework achieves performance comparable to models pre-trained on AudioSet-2M and partially outperforms image-based pre-training methods. © 2025 IEEE.

关键词： audio masked autoencoder self-supervised learning synthetic data

来源：评论

学校读者我要写书评

暂无评论

Partial Reconstruction Error for Deepfake Detection

Partial Reconstruction Error for Deepfake Detection

引用

2025 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2025

作者： Zhang, Yufei Meng, Zheling Peng, Bo Dong, Jing Chu, Beilin Wang, Wei School of Cyberspace Security Beijing University of Posts and Telecommunications Beijing China Institute of Automation Chinese Academy of Sciences Beijing China

ISBN: (纸本)9798350368741

The rapid development of deepfake technology poses a formidable challenge to personal privacy and security, underscoring the urgent need for deepfake detection. Recently, the methods based on the reconstruction error, such as DIRE and RECCE, achieve impressive performance in forgery detection. However, their performance on facial forgery datasets is relatively poor. The reconstruction process is performed on the whole images, neglecting contextual information for reconstruction. In this paper, we propose Partial Reconstruction Error to perform deepfake detection based on the reconstruction of masked regions in an image. In this way, contextual information helps to reveal the inconsistencies between the original and reconstructed regions thereby improving the detection performance. This method outperforms the best global reconstruction-based approaches on the FF++, Celeb-DF, and DiFF datasets by 4.00%, 2.83%, and 2.67%, respectively. © 2025 IEEE.

关键词： Deepfake Detection masked autoencoder Partial Reconstruction

来源：评论

学校读者我要写书评

暂无评论

masked autoencoders as Single Object Tracking Learners

引用

Computers, Materials & Continua 2024年第7期80卷 1105-1122页

作者： Chunjuan Bo XinChen Junxing Zhang School of Information and Communication Engineering Dalian Minzu UniversityDalian116600China School of Information and Communication Engineering Dalian University of TechnologyDalian116024China

Significant advancements have beenwitnessed in visual tracking applications leveragingViT in recent years,mainly due to the formidablemodeling capabilities of Vision Transformer(ViT).However,the strong performance of such trackers heavily relies on ViT models pretrained for long periods,limitingmore flexible model designs for tracking *** address this issue,we propose an efficient unsupervised ViT pretraining method for the tracking task based on masked autoencoders,called *** pretraining,we employ two shared-parameter ViTs,serving as the appearance encoder and motion encoder,*** appearance encoder encodes randomly masked image data,while the motion encoder encodes randomly masked pairs of video ***,an appearance decoder and a motion decoder separately reconstruct the original image data and video frame data at the pixel *** this way,ViT learns to understand both the appearance of images and the motion between video frames *** results demonstrate that ViT-Base and ViT-Large models,pretrained with TrackMAE and combined with a simple tracking head,achieve state-of-the-art(SOTA)performance without additional ***,compared to the currently popular MAE pretraining methods,TrackMAE consumes only 1/5 of the training time,which will facilitate the customization of diverse models for *** instance,we additionally customize a lightweight ViT-XS,which achieves SOTA efficient tracking performance.

关键词： Visual object tracking vision transformer masked autoencoder visual representation learning

来源：评论

学校读者我要写书评

暂无评论

masked autoencoders for Spatial-Temporal Relationship in Video-Based Group Activity Recognition

引用

IEEE ACCESS 2024年 12卷 132084-132095页

作者： Yadav, Rajeshwar Halder, Raju Banda, Gourinath Indian Inst Technol Patna Dept Comp Sci & Engn Bihta 801106 India Indian Inst Technol Indore Dept Comp Sci & Engn Indore 453552 Madhya Pradesh India

Group Activity Recognition (GAR) is a challenging problem involving several intricacies. The core of GAR lies in delving into spatiotemporal features to generate appropriate scene representations. Previous methods, however, either feature a complex framework requiring individual action labels or need more adequate modelling of spatial and temporal features. To address these concerns, we propose a masking strategy for learning task-specific GAR scene representations through reconstruction. Furthermore, we elucidate how this methodology can effectively capture task-specific spatiotemporal features. In particular, three notable findings emerge from our framework: 1) GAR is simplified, eliminating the need for individual action labels;2) the generation of target-specific spatiotemporal features yields favourable outcomes for various datasets;and 3) this method demonstrates effectiveness even for datasets with a small number of videos, highlighting its capability with limited training data. Further, the existing GAR datasets have fewer videos per class and only a few actors are considered, restricting the existing model from being generalised effectively. To this aim, we introduce 923 videos for a crime activity named IITP Hostage, which contains two categories, hostage and non-hostage. To our knowledge, this is the first attempt to recognize crime-based activities in GAR. Our framework achieves MCA of 96.8%, 97.0%, 97.0% on Collective Activity Dataset (CAD), new CAD, extended CAD datasets and 84.3%, 95.6%, 96.78% for IITP Hostage, hostage+CAD and subset of UCF crime datasets. The hostage and non-hostage scenarios introduce additional complexity, making it more challenging for the model to accurately recognize the activities compared to hostage+CAD and other datasets. This observation underscores the necessity to delve deeper into the complexity of GAR activities.

关键词： Group activity recognition (GAR) hostage crime IITP hostage dataset spatial and temporal interaction vision transformer masked autoencoder

来源：评论

学校读者我要写书评

暂无评论

masked autoencoders for Low-dose CT Denoising 20

Masked Autoencoders for Low-dose CT Denoising

引用

20th IEEE International Symposium on Biomedical Imaging (ISBI)

作者： Wang, Dayang Xu, Yongshun Han, Shuo Yu, Hengyong Univ Massachusetts Lowell Dept Elect & Comp Engn Lowell MA 01854 USA

ISBN: (纸本)9781665473583

Low-dose computed tomography (LDCT) reduces the X-ray radiation but compromises image quality with more noises and artifacts. A plethora of transformer models have been developed recently to improve LDCT image quality. However, the success of a transformer model relies on a large amount of paired noisy and clean data, which is often unavailable in clinical applications. In computer vision and natural language processing fields, masked autoencoders (MAE) have been proposed as an effective label-free self-pretraining method for transformers, due to its excellent feature representation ability. Here, we redesign the classical encoder-decoder learning model based on SwinIR to match the denoising task and apply it to LDCT denoising problem. Then, the redesigned MAE can leverage the unlabeled data and facilitate structural preservation for the LDCT denoising model when there are insufficient target data. Experiments on the Mayo dataset validate that the MAE can boost the transformer's denoising performance and relieve the dependence on the ground truth data.

关键词： Low-dose CT masked autoencoder Self-pretraining Transformer

来源：评论

学校读者我要写书评

暂无评论

PersonMAE: Person Re-Identification Pre-Training With masked autoencoders

引用

IEEE TRANSACTIONS ON MULTIMEDIA 2024年 26卷 10029-10040页

作者： Hu, Hezhen Dong, Xiaoyi Bao, Jianmin Chen, Dongdong Yuan, Lu Chen, Dong Li, Houqiang Univ Sci & Technol China Hefei 230027 Peoples R China Microsoft Res Asia Beijing 100080 Peoples R China Microsoft Redmond WA 98052 USA

Pre-training is playing an increasingly important role in learning generic feature representation for Person Re-identification (ReID). We argue that a high-quality ReID representation should have three properties, namely, multi-level awareness, occlusion robustness, and cross-region invariance. To this end, we propose a simple yet effective pre-training framework, namely PersonMAE, which involves two core designs into masked autoencoders to better serve the task of Person Re-ID. 1) PersonMAE generates two regions from the given image with RegionA as the input and RegionB as the prediction target. RegionA is corrupted with block-wise masking to mimic common occlusion in ReID and its remaining visible parts are fed into the encoder. 2) Then PersonMAE aims to predict the whole RegionB at both pixel level and semantic feature level. It encourages its pre-trained feature representations with the three properties mentioned above. These properties make PersonMAE compatible with downstream Person ReID tasks, leading to State-of-the-Art performance on four downstream ReID tasks, i.e., supervised (holistic and occluded setting), and unsupervised (UDA and USL setting). Notably, on the commonly adopted supervised setting, PersonMAE with ViT-B backbone achieves 79.8% and 69.5% mAP on the MSMT17 and OccDuke datasets, surpassing the previous State-of-the-Art by a large margin of +8.0 mAP, and +5.3 mAP, respectively.

关键词： Task analysis Pedestrians Semantics Robustness Decoding Visualization Image color analysis High-quality ReID representation masked autoencoder pre-training

来源：评论

学校读者我要写书评

暂无评论

Dynamic background reconstruction via masked autoencoders for infrared small target detection

引用

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE 2024年 135卷

作者： Peng, Jingchao Zhao, Haitao Zhao, Kaijie Wang, Zhongze Yao, Lujian East China Univ Sci & Technol Sch Informat Sci & Engn Automat Dept 130 Meilong Rd Shanghai 200237 Peoples R China

Infrared small target detection (ISTD) under complex backgrounds is a difficult problem, for the differences between targets and backgrounds are not easy to distinguish. Background reconstruction is one of the methods to deal with this problem. This paper proposes an ISTD method based on background reconstruction called Dynamic Background Reconstruction (DBR). DBR consists of three modules: a dynamic shift window module (DSW), a background reconstruction module (BR), and a detection head (DH). BR takes advantage of masked autoencoders in reconstructing missing patches and adopts a grid masking strategy with a masking ratio of 50% to reconstruct clean backgrounds without targets. To avoid dividing one target into two neighboring patches, resulting in reconstructing failure, DSW is performed before input embedding. DSW calculates offsets, according to which infrared images dynamically shift. To reduce False Positive (FP) cases caused by regarding reconstruction errors as targets, DH utilizes a structure of densely connected Transformer to further improve the detection performance. Experimental results show that DBR achieves the best F 1-score on the two ISTD datasets, which can be widely used in anti-unmanned aerial vehicles, surveillance, and automatic driving systems. The model and dataset will be available at https://***/PengJingchao/DBR.

关键词： Infrared modality Small target detection Background reconstruction Dynamic shift window masked autoencoder

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：