检索结果-内蒙古大学图书馆

A Survey on Remote Sensing Foundation Models: From Vision to Multimodality

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Huang, Ziyue Yan, Hongxi Zhan, Qiqi Yang, Shuai Zhang, Mingming Zhang, Chenkai Lei, YiMing Liu, Zeming Liu, Qingjie Wang, Yunhong State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China Hangzhou Innovation Institute Beihang University Hangzhou310051 China

The rapid advancement of remote sensing foundation models, particularly vision and multimodal models, has significantly enhanced the capabilities of intelligent geospatial data interpretation. These models combine various data modalities, such as optical, radar, and LiDAR imagery, with textual and geographic information, enabling more comprehensive analysis and understanding of remote sensing data. The integration of multiple modalities allows for improved performance in tasks like object detection, land cover classification, and change detection, which are often challenged by the complex and heterogeneous nature of remote sensing data. However, despite these advancements, several challenges remain. The diversity in data types, the need for large-scale annotated datasets, and the complexity of multimodal fusion techniques pose significant obstacles to the effective deployment of these models. Moreover, the computational demands of training and fine-tuning multimodal models require significant resources, further complicating their practical application in remote sensing image interpretation tasks. This paper provides a comprehensive review of the state-of-the-art in vision and multimodal foundation models for remote sensing, focusing on their architecture, training methods, datasets and application scenarios. We discuss the key challenges these models face, such as data alignment, cross-modal transfer learning, and scalability, while also identifying emerging research directions aimed at overcoming these limitations. Our goal is to provide a clear understanding of the current landscape of remote sensing foundation models and inspire future research that can push the boundaries of what these models can achieve in real-world applications. The list of resources collected by the paper can be found in the https://***/IRIP-BUAA/A-Review-forremote-sensing-vision-language-models. © 2025, CC BY.

关键词： Photointerpretation

Application of an Improved Residual Attention Neural Network in Mechanical Part Classification 9th

学校读者我要写书评

暂无评论

Application of an Improved Residual Attention Neural Network...

9th International Symposium on Artificial Intelligence and Robotics, ISAIR 2024

作者： Zhang, Tianrui Wang, Aizeng School of Mechanical Engineering and Automation Beihang University Beijing100191 China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China

ISBN: (纸本)9789819629138

This paper introduces an improved neural network that integrates residual network and attention mechanisms, achieving a significant improvement of 99.50% accuracy in mechanical part classification tasks. The proposed network enhance accuracy, precision, recall, and F1-score compared to traditional CNNs and other advanced networks in handling complex backgrounds and multi-angle part images. This advancement shows potential in smart manufacturing, offering greater automation, minimizing human intervention. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

关键词： Smart manufacturing

Heterogeneous Mixture of Experts for Remote Sensing Image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Bowen Chen, keyan Yang, Mohan Zou, Zhengxia Shi, Zhenwei Department of Aerospace Intelligent Science and Technology School of Astronautics Beihang University Beijing100191 China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing100191 China

Remote sensing image super-resolution (SR) aims to reconstruct high-resolution remote sensing images from low-resolution inputs, thereby addressing limitations imposed by sensors and imaging conditions. However, the inherent characteristics of remote sensing images, including diverse ground object types and complex details, pose significant challenges to achieving high-quality reconstruction. Existing methods typically employ a uniform structure to process various types of ground objects without distinction, making it difficult to adapt to the complex characteristics of remote sensing images. To address this issue, we introduce a Mixture of Experts (MoE) model and design a set of heterogeneous experts. These experts are organized into multiple expert groups, where experts within each group are homogeneous while being heterogeneous across groups. This design ensures that specialized activation parameters can be employed to handle the diverse and intricate details of ground objects effectively. To better accommodate the heterogeneous experts, we propose a multi-level feature aggregation strategy to guide the routing process. Additionally, we develop a dual-routing mechanism to adaptively select the optimal expert for each pixel. Experiments conducted on the UCMerced and AID datasets demonstrate that our proposed method achieves superior SR reconstruction accuracy compared to state-of-the-art methods. The code will be available at https://***/Mr-Bamboo/MFG-HMoE. Copyright © 2025, The Authors. All rights reserved.

关键词： Optical resolving power

SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Cai, Wenrui Liu, Qingjie Wang, Yunhong State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China Zhongguancun Laboratory Beijing China Hangzhou Innovation Institute Beihang University Hangzhou China

Most state-of-the-art trackers adopt one-stream paradigm, using a single Vision Transformer for joint feature extraction and relation modeling of template and search region images. However, relation modeling between different image patches exhibits significant variations. For instance, background regions dominated by target-irrelevant information require reduced attention allocation, while foreground, particularly boundary areas, need to be be emphasized. A single model may not effectively handle all kinds of relation modeling simultaneously. In this paper, we propose a novel tracker called SPMTrack based on mixture-of-experts tailored for visual tracking task (TMoE), combining the capability of multiple experts to handle diverse relation modeling more flexibly. Benefiting from TMoE, we extend relation modeling from image pairs to spatio-temporal context, further improving tracking accuracy with minimal increase in model parameters. Moreover, we employ TMoE as a parameter-efficient fine-tuning method, substantially reducing trainable parameters, which enables us to train SPMTrack of varying scales efficiently and preserve the generalization ability of pretrained models to achieve superior performance. We conduct experiments on seven datasets, and experimental results demonstrate that our method significantly outperforms current state-of-the-art trackers. The source code is available at https://***/WenRuiCai/SPMTrack. Copyright © 2025, The Authors. All rights reserved.

关键词： Tuning

Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Wang, Yin Li, Mu Liu, Jiapeng Leng, Zhiying Li, Frederick W.B. Zhang, Ziyao Liang, Xiaohui B. State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China Department of Computer Science University of Durham United Kingdom Zhongguancun Laboratory Beijing China

We address the challenging problem of fine-grained text-driven human motion generation. Existing works generate imprecise motions that fail to accurately capture relationships specified in text due to: (1) lack of effective text parsing for detailed semantic cues regarding body parts, (2) not fully modeling linguistic structures between words to comprehend text comprehensively. To tackle these limitations, we propose a novel fine-grained framework Fg-T2M++ that consists of: (1) an LLMs semantic parsing module to extract body part descriptions and semantics from text, (2) a hyperbolic text representation module to encode relational information between text units by embedding the syntactic dependency graph into hyperbolic space, and (3) a multi-modal fusion module to hierarchically fuse text and motion features. Extensive experiments on HumanML3D and KIT-ML datasets demonstrate that Fg-T2M++ outperforms SOTA methods, validating its ability to accurately generate motions adhering to comprehensive text semantics. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantics

A survey of deep learning methods for density estimation and crowd counting

学校读者我要写书评

暂无评论

Vicinagearth 2025年第1期2卷 1-37页

作者： Gao, Guangshuai Gao, Junyu Liu, Qingjie Wang, Qi Wang, Yunhong School of Information and Communication Engineering Zhongyuan University of Technology Zhengzhou China State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China School of Artificial Intelligence OPtics and ElectroNics (iOPEN) Northwestern Polytechnical University Xi’an China

Accurately estimating the number of objects in a single image is a challenging yet meaningful task and has been applied in many applications such as urban planning and public safety. In various object counting tasks, crowd counting is particularly prominent due to its specific significance to social security and development. Fortunately, the development of the techniques for crowd counting can be generalized to other related fields such as vehicle counting and environment survey, if without taking their characteristics into account. Therefore, many researchers are devoting to crowd counting, and many excellent works have spurted out and significantly promote the development of crowd counting. However, one question we should consider is how far are we from solving the counting problem?. Limited by the costs of time and energy, we cannot analyze all algorithms. In this paper, we have surveyed 300+ works to comprehensively and systematically study the crowd counting models, mainly involving CNN-based density map estimation methods. Finally, according to evaluation metrics, we select the top three performers on their crowd counting datasets and analyze their merits and drawbacks. Through our analysis, we expect to make a reasonable inference and prediction for the future development of crowd counting, and meanwhile, it can also provide feasible solutions for the problem of object counting in other fields. We provide density maps and prediction results of some mainstream algorithms on the validation set of NWPU dataset for comparison and testing. Meanwhile, density map generation and evaluation tools are also provided. All the codes and evaluation results are made publicly available at https://***/gaoguangshuai/survey-for-crowd-counting .

关键词：

Adaptive Score Alignment Learning for Continual Perceptual Quality Assessment of 360-Degree Videos in virtual reality

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhou, Kanglei Hao, Zikai Wang, Liyuan Liang, Xiaohui State Key Laboratory of Virtual Reality Technology and Systems Beihang University Beijing China Zhongguancun Laboratory Beijing China Beihang University Beijing China Tsinghua University Beijing China

virtual reality Video Quality Assessment (VR-VQA) aims to evaluate the perceptual quality of 360-degree videos, which is crucial for ensuring a distortion-free user experience. Traditional VR-VQA methods trained on static datasets with limited distortion diversity struggle to balance correlation and precision. This becomes particularly critical when generalizing to diverse VR content and continually adapting to dynamic and evolving video distribution variations. To address these challenges, we propose a novel approach for assessing the perceptual quality of VR videos, Adaptive Score Alignment Learning (ASAL). ASAL integrates correlation loss with error loss to enhance alignment with human subjective ratings and precision in predicting perceptual quality. In particular, ASAL can naturally adapt to continually changing distributions through a feature space smoothing process that enhances generalization to unseen content. To further improve continual adaptation to dynamic VR environments, we extend ASAL with adaptive memory replay as a novel Continul Learning (CL) framework. Unlike traditional CL models, ASAL utilizes key frame extraction and feature adaptation to address the unique challenges of non-stationary variations with both the computation and storage restrictions of VR devices. We establish a comprehensive benchmark for VR-VQA and its CL counterpart, introducing new data splits and evaluation metrics. Our experiments demonstrate that ASAL outperforms recent strong baseline models, achieving overall correlation gains of up to 4.78% in the static joint training setting and 12.19% in the dynamic CL setting on various datasets. This validates the effectiveness of ASAL in addressing the inherent challenges of VR-VQA. Our code is available at https://***/ZhouKanglei/ASAL_CVQA. Copyright © 2025, The Authors. All rights reserved.

关键词： virtual environments

LBPE: Long-token-first Tokenization to Improve Large Language Models

学校读者我要写书评

暂无评论

LBPE: Long-token-first Tokenization to Improve Large Languag...

International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

作者： Haoran Lian Yizhe Xiong Zijia Lin Jianwei Niu Shasha Mo Hui Chen Peng Liu Guiguang Ding Beihang University Tsinghua University BNRist State Key Laboratory of Virtual Reality Technology and Systems Beihang University Zhongguancun Laboratory Zhengzhou University Research Institute of Industrial Technology Zhengzhou University Beijing Institute of Technology

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

The prevalent use of Byte Pair Encoding (BPE) in Large Language Models (LLMs) facilitates robust handling of subword units and avoids issues of out-of-vocabulary words. Despite its success, a critical challenge persists: long tokens, rich in semantic information, have fewer occurrences in tokenized datasets compared to short tokens, which can result in imbalanced learning issue across different tokens. To address that, we propose LBPE, which prioritizes long tokens during the encoding process. LBPE generates tokens according to their descending order of token length rather than their ranks in the vocabulary, granting longer tokens higher priority during the encoding process. Consequently, LBPE smooths the frequency differences between short and long tokens, and thus mitigates the learning imbalance. Extensive experiments across diverse language modeling tasks demonstrate that LBPE consistently outperforms the original BPE, well demonstrating its effectiveness.

关键词： Vocabulary Large language models Semantics Signal processing Encoding Tokenization Acoustics Speech processing

Efficient Semantic Splatting for Remote Sensing Multiview Segmentation

学校读者我要写书评

暂无评论

IEEE Transactions on Geoscience and Remote Sensing 2025年 63卷

作者： Qi, Zipeng Chen, Hao Zhang, Haotian Zou, Zhengxia Shi, Zhenwei Beihang University School of Astronautics State Key Laboratory of Virtual Reality Technology and Systems Key Laboratory of Spacecraft Design Optimization and Dynamic Simulation Technologies Ministry of Education Department of Aerospace Intelligent Science and Technology Beijing100191 China Shanghai Artificial Intelligence Laboratory Shanghai200232 China

Remote sensing multiview image segmentation is essential for achieving accurate and consistent stereoscopic perception of target scenes. This task involves processing RGB images from multiple viewpoints to generate high-accuracy, view-consistent semantic segmentation across all views. Traditional training-based methods struggle with maintaining cross-view consistency, while optimization-driven approaches using implicit neural networks improve view consistency but suffer from slow parameter optimization and inference. To overcome these limitations, we propose a novel Gaussian splatting-based semantic segmentation framework. Our method efficiently projects the color attributes and semantic features of 3-D Gaussians onto the image plane, enabling the simultaneous generation of both RGB images and segmentation outputs. By leveraging explicit spatial structures and a splatting rendering strategy, our approach significantly enhances optimization efficiency and rendering speed. In addition, we incorporate SAM2 to generate pseudo-labels for boundary regions, addressing the lack of supervision in sparsely labeled views (e.g., 3%). To further enforce cross-view consistency and feature coherence of 3-D Gaussians, we introduce a two-level aggregation loss that operates at both the 2-D feature map and 3-D spatial levels. Extensive experiments across nine datasets demonstrate the superiority of our method, achieving competitive segmentation quality with limited supervisory views. Notably, our approach reduces rendering (inference) times by 90%, while improving the average mean intersection over union (mIoU) by up to 3.5%. © 1980-2012 IEEE.

关键词： Semantic Segmentation