检索结果-内蒙古大学图书馆

Integrating global context contrast and local sensitivity for blind image quality assessment 24

学校读者我要写书评

暂无评论

Integrating global context contrast and local sensitivity fo...

Proceedings of the 41st International Conference on Machine Learning

作者： Xudong Li Runze Hu Jingyuan Zheng Yan Zhang Shengchuan Zhang Xiawu Zheng Ke Li Yunhang Shen Yutao Liu Pingyang Dai Rongrong Ji Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University P.R. China School of Information and Electronics Beijing Institute of Technology Beijing China School of Medicine Xiamen University. Tencent Youtu Lab. School of Computer Science and Technology Ocean University of China

Blind Image Quality Assessment (BIQA) mirrors subjective made by human observers. Generally, humans favor comparing relative qualities over predicting absolute qualities directly. However, current BIQA models focus on mining the "local" context, i.e., the relationship between information among individual images and the absolute quality of the image, ignoring the "global" context of the relative quality contrast among different images in the training data. In this paper, we present the Perceptual Context and Sensitivity BIQA (CSIQA), a novel contrastive learning paradigm that seamlessly integrates "global" and "local" perspectives into the BIQA. Specifically, the CSIQA comprises two primary components: 1) A Quality Context Contrastive Learning module, which is equipped with different contrastive learning strategies to effectively capture potential quality correlations in the global context of the dataset. 2) A Quality-aware Mask Attention Module, which employs the random mask to ensure the consistency with visual local sensitivity, thereby improving the model's perception of local distortions. Extensive experiments on eight standard BIQA datasets demonstrate the superior performance to the state-of-the-art BIQA methods.

关键词：

Energy and Latency-Aware Resource Management for UAV-Assisted Mobile Edge computing Against Jamming

学校读者我要写书评

暂无评论

Energy and Latency-Aware Resource Management for UAV-Assiste...

IEEE Conference on Global Communications (GLOBECOM)

作者： Ziling Shao Helin Yang Liang Xiao Wei Su Zehui Xiong Institute of Artificial Intelligence Xiamen University Xiamen China School of Informatics Xiamen University Xiamen China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Xiamen University Xiamen China Pillar of Information System Technology & Design Singapore University of Technology and Design Singapore

Unmanned aerial vehicles (UAVs) have been increasingly employed as aerial servers in mobile edge computing (MEC) systems, providing essential computing, communication, and storage services for edge users. This UAV-assisted MEC paradigm shows great promise in enhancing both computing and communication performances. However, the presence of malicious jammers poses significant challenges to the system's reliability and efficiency. In this study, we explore the resource management problem in a multi-UAV-assisted MEC scenario under the influence of multiple malicious jammers. To mitigate the impact of jamming attacks, we propose a resource management approach with the primary objective of minimizing system energy consumption and latency while adhering to UAV energy constraints. Due to the dynamic and time-varying nature of the communication environment, we present a deep reinforcement learning (DRL)-based algorithm that dynamically adjusts the CPU frequency and communication bandwidth of the UAV to optimize the system performance even under jamming attacks. Through simulations, we demonstrate the effectiveness of the proposed algorithm in significantly reducing the overall system latency (both computational and communication latency) as well as minimizing energy consumption.

关键词：

ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Yan, Ming Lin, Xincheng Luo, Yuhua Fan, Shuqi Dai, Yudi Zhong, Qixin Zhong, Lincai Ma, Yuexin Xu, Lan Wen, Chenglu Shen, Siqi Wang, Cheng Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China National Institute for Data Science in Health and Medicine Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University China China National Climbing Team China Ningbo Sports Work Training Team China ShanghaiTech University China ETH AI Center ETH Zürich Switzerland

Human Motion Recovery (HMR) research mainly focuses on ground-based motions such as running. The study on capturing climbing motion, an off-ground motion, is sparse. This is partly due to the limited availability of climbing motion datasets, especially large-scale and challenging 3D labeled datasets. To address the insufficiency of climbing motion datasets, we collect AscendMotion, a large-scale well-annotated, and challenging climbing motion dataset. It consists of 412k RGB, LiDAR frames, and IMU measurements, including the challenging climbing motions of 22 skilled climbing coaches across 12 different rock walls. Capturing the climbing motions is challenging as it requires precise recovery of not only the complex pose but also the global position of climbers. Although multiple global HMR methods have been proposed, they cannot faithfully capture climbing motions. To address the limitations of HMR methods for climbing, we propose ClimbingCap, a motion recovery method that reconstructs continuous 3D human climbing motion in a global coordinate system. One key insight is to use the RGB and LiDAR modalities to separately reconstruct motions in camera coordinates and global coordinates and to optimize them jointly. We demonstrate the quality of the AscendMotion dataset and present promising results from ClimbingCap. The AscendMotion dataset and source code release publicly at http://***/climbingcap/ © 2025, CC BY-NC-ND.

关键词： Motion capture

Multi-Modal Prompt Learning on Blind Image Quality Assessment

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Pan, Wensheng Gao, Timin Zhang, Yan Hu, Runze Zheng, Xiawu Zhang, Enwei Gao, Yuting Liu, Yutao Shen, Yunhang Li, Ke Zhang, Shengchuan Cao, Liujuan Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University Xiamen361005 China Tencent Youtu Lab Shanghai200233 China School of Information and Electronics Beijing Institute of Technology Beijing100086 China School of Computer Science and Technology Ocean University of China Qingdao266100 China

Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Currently, leveraging semantic information to enhance IQA is a crucial research direction. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. However, the generalist nature of these pre-trained Vision-Language (VL) models often renders them suboptimal for IQA-specific tasks. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. Existing prompt-based VL models overly focus on incremental semantic information from text, neglecting the rich insights available from visual data analysis. This imbalance limits their performance improvements in IQA tasks. This paper introduces an innovative multi-modal prompt-based methodology for IQA. Our approach employs carefully crafted prompts that synergistically mine incremental semantic information from both visual and linguistic data. Specifically, in the visual branch, we introduce a multi-layer prompt structure to enhance the VL model’s adaptability. In the text branch, we deploy a dual-prompt scheme that steers the model to recognize and differentiate between scene category and distortion type, thereby refining the model’s capacity to assess image quality. Our experimental findings underscore the effectiveness of our method over existing Blind Image Quality Assessment (BIQA) approaches. Notably, it demonstrates competitive performance across various datasets. Our method achieves Spearman Rank Correlation Coefficient (SRCC) values of 0.961(surpassing 0.946 in CSIQ) and 0.941 (exceeding 0.930 in KADID), illustrating its robustness and accuracy in diverse contexts. Our code is available at https://***/stephencurry-web/IQA. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Yan, Ming Wang, Xin Dai, Yudi Shen, Siqi Wen, Chenglu Xu, Lan Ma, Yuexin Wang, Cheng Fujian Key Laboratory of Sensing and Computing for Smart Cities Xiamen University China National Institute for Data Science in Health and Medicine Xiamen University China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China School of Informatics Xiamen University China Shanghai Engineering Research Center of Intelligent Vision and Imaging ShanghaiTech University China

Motion capture is a long-standing research problem. Although it has been studied for decades, the majority of research focus on ground-based movements such as walking, sitting, dancing, etc. Off-grounded actions such as climbing are largely overlooked. As an important type of action in sports and firefighting field, the climbing movements is challenging to capture because of its complex back poses, intricate human-scene interactions, and difficult global localization. The research community does not have an in-depth understanding of the climbing action due to the lack of specific datasets. To address this limitation, we collect CIMI4D, a large rock ClImbing MotIon dataset from 12 persons climbing 13 different climbing walls. The dataset consists of around 180,000 frames of pose inertial measurements, LiDAR point clouds, RGB videos, high-precision static point cloud scenes, and reconstructed scene meshes. Moreover, we frame-wise annotate touch rock holds to facilitate a detailed exploration of human-scene interaction. The core of this dataset is a blending optimization process, which corrects for the pose as it drifts and is affected by the magnetic conditions. To evaluate the merit of CIMI4D, we perform four tasks which include human pose estimations (with/without scene constraints), pose prediction, and pose generation. The experimental results demonstrate that CIMI4D presents great challenges to existing methods and enables extensive research opportunities. We share the dataset with the research community in http://***/cimi4d/. Copyright © 2023, The Authors. All rights reserved.

关键词： Blending

Long-Tailed Visual Recognition via Self-Heterogeneous Integration with Knowledge Excavation

学校读者我要写书评

暂无评论

Long-Tailed Visual Recognition via Self-Heterogeneous Integr...

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yan Jin Mengke Li Yang Lu Yiu-ming Cheung Hanzi Wang Fujian Key Laboratory of Sensing and Computing for Smart City School of Informatics Xiamen University Xiamen China Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University Xiamen China Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) Shenzhen China Department of Computer Science Hong Kong Baptist University Hong Kong SAR China

Deep neural networks have made huge progress in the last few decades. However, as the real-world data often exhibits a long-tailed distribution, vanilla deep models tend to be heavily biased toward the majority classes. To address this problem, state-of-the-art methods usually adopt a mixture of experts (MoE) to focus on different parts of the long-tailed distribution. Experts in these methods are with the same model depth, which neglects the fact that different classes may have different preferences to be fit by models with different depths. To this end, we propose a novel MoE-based method called Self-Heterogeneous Integration with Knowledge Excavation (SHIKE). We first propose Depth-wise Knowledge Fusion (DKF) to fuse features between different shallow parts and the deep part in one network for each expert, which makes experts more diverse in terms of representation. Based on DKF, we further propose Dynamic Knowledge Transfer (DKT) to reduce the influence of the hardest negative class that has a non-negligible impact on the tail classes in our MoEframework. As a result, the classification accuracy of long-tailed data can be significantly improved, especially for the tail classes. SHIKE achieves the state-of-the-art performance of 56.3%, 60.3%, 75.4% and 41.9% on CIFAR100-LT (IF100), ImageNet-LT, iNaturalist 2018, and Places-LT, respectively. The source code is available at https://***/jinyan-06/SHIKE.

关键词：

DAMamba: Vision State Space Model with Dynamic Adaptive Scan

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Li, Tanzhe Li, Caoshuo Lyu, Jiayi Pei, Hongjuan Zhang, Baochang Jin, Taisong Ji, Rongrong Key Laboratory of Multimedia Trusted Perception and Effcient Computing Ministry of Education of China Xiamen University China School of Informatics Xiamen University China School of Engineering Science University of Chinese Academy of Sciences China School of Artificial Intelligence Beihang University China

State space models (SSMs) have recently garnered significant attention in computer vision. However, due to the unique characteristics of image data, adapting SSMs from natural language processing to computer vision has not outperformed the state-of-the-art convolutional neural networks (CNNs) and Vision Transformers (ViTs). Existing vision SSMs primarily leverage manually designed scans to flatten image patches into sequences locally or globally. This approach disrupts the original semantic spatial adjacency of the image and lacks flexibility, making it difficult to capture complex image structures. To address this limitation, we propose Dynamic Adaptive Scan (DAS), a data-driven method that adaptively allocates scanning orders and regions. This enables more flexible modeling capabilities while maintaining linear computational complexity and global modeling capacity. Based on DAS, we further propose the vision backbone DAMamba, which significantly outperforms current state-of-the-art vision Mamba models in vision tasks such as image classification, object detection, instance segmentation, and semantic segmentation. Notably, it surpasses some of the latest state-of-the-art CNNs and ViTs. Code will be available at https://***/ltzovo/DAMamba. Copyright © 2025, The Authors. All rights reserved.

关键词： Semantic Segmentation

Spatial-Temporal Knowledge Transfer for Dynamic Constrained Multiobjective Optimization

学校读者我要写书评

暂无评论

IEEE Transactions on Evolutionary Computation 2024年 1-1页

作者： Wang, Zhenzhong Xu, Dejun Jiang, Min Tan, Kay Chen Department of Computing The Hong Kong Polytechnic University Hong Kong P.R. China Department of Artificial Intelligence Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan Ministry of Culture and Tourism School of Informatics Xiamen University Xiamen Fujian China Department of Data Science and Artificial Intelligence The Hong Kong Polytechnic University Hong Kong P.R. China

Dynamic Constrained Multiobjective Optimization Problems (DCMOPs) are characterized by multiple conflicting optimization objectives and constraints that vary over time. The presence of both dynamism and constraints underscores the importance of preserving population diversity. This diversity is essential not only to escape local optima following environmental changes but also to climb infeasible barriers to approach feasible regions. However, existing constraint-handling techniques for enhancing solution feasibility could steer infeasible solutions toward partially feasible regions, potentially resulting in the loss of diversity. To maintain both diversity and feasibility, this work establishes two synergistic tasks: one task concentrates on exploring the unconstrained search space to preserve diversity, while the other delves into searching the constrained search space to prioritize feasibility. Particularly, in light of evolutionary transfer optimization, two knowledge transfer modules, i.e., the spatial knowledge transfer module and temporal knowledge transfer module are designed. The spatial knowledge transfer module facilitates knowledge transfer between the constrained and unconstrained search spaces to accelerate the exploration of both spaces. On the other hand, the temporal transfer module leverages historical knowledge to enhance search efficiency within the new environment. To advance the test suite toward real-world cases, we designed fourteen test problems with various properties. Experiments conducted on the proposed test problems and a real-world problem have demonstrated the efficacy of our proposed algorithm. IEEE

关键词： Constrained optimization

Exploring the Collaborative Advantage of Low-level Information on Generalizable AI-Generated Image Detection

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Zhou, Ziyin Sun, Ke Chen, Zhongxi Lin, Xianming Luo, Yunpeng Yan, Ke Ding, Shouhong Sun, Xiaoshuai Key Laboratory of Multimedia Trusted Perception and Efficient Computing Ministry of Education of China Xiamen University China Tencent YouTu Lab

Existing state-of-the-art AI-Generated image detection methods mostly consider extracting low-level information from RGB images to help improve the generalization of AI-Generated image detection, such as noise patterns. However, these methods often consider only a single type of low-level information, which may lead to suboptimal generalization. Through empirical analysis, we have discovered a key insight: different low-level information often exhibits generalization capabilities for different types of forgeries. Furthermore, we found that simple fusion strategies are insufficient to leverage the detection advantages of each low-level and high-level information for various forgery types. Therefore, we propose the Adaptive Low-level Experts Injection (ALEI) framework. Our approach introduces Lora Experts, enabling the backbone network, which is trained with high-level semantic RGB images, to accept and learn knowledge from different low-level information. We utilize a cross-attention method to adaptively fuse these features at intermediate layers. To prevent the backbone network from losing the modeling capabilities of different low-level features during the later stages of modeling, we developed a Low-level Information Adapter that interacts with the features extracted by the backbone network. Finally, we propose Dynamic Feature Selection, which dynamically selects the most suitable features for detecting the current image to maximize generalization detection capability. Extensive experiments demonstrate that our method, finetuned on only four categories of mainstream ProGAN data, performs excellently and achieves state-of-the-art results on multiple datasets containing unseen GAN and Diffusion methods. Copyright © 2025, The Authors. All rights reserved.

关键词： Feature Selection