检索结果-内蒙古大学图书馆

Hybrid RAG-empowered Multi-modal LLM for Secure data Management in Internet of Medical Things: A Diffusion-based Contract Approach

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Su, Cheng Wen, Jinbo Kang, Jiawen Wang, Yonghua Su, Yuanjia Pan, Hudan Zhong, Zishao Hossain, M. Shamim School of Automation The GuangdongHongKong-Macao Joint Laboratory for Smart Discrete Manufacturing Guangdong University of Technology Guangzhou510006 China College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics Nanjing210016 China School of Automation The Guangdong Basic Research Center of Excellence for Ecological Security and Green Development Key Laboratory for City Cluster Environmental Safety Green Development The Ministry of Education Guangdong University of Technology Guangzhou510006 China School of Automation The Key Laboratory of Intelligent Detection and IoT in Manufacturing Ministry of Education Guangdong University of Technology Guangzhou510006 China School of Automation Key Laboratory of Intelligent Information Processing and System Integration of IoT Ministry of Education Guangdong University of Technology Guangzhou510006 China Guangzhou University of Traditional Chinese Medicine State Key Laboratory of Traditional Chinese Medicine Syndrome The Second Affiliated Hospital of Guangzhou University of Chinese Medicine Chinese Medicine Guangdong Laboratory Hengqin 519000 China Second Affiliated Hospital of Guangzhou University of Chinese Medicine Chinese Medicine Guangdong Laboratory Hengqin 519000 China Department of Software Engineering College of Computer and Information Sciences King Saud University Riyadh12372 Saudi Arabia

Secure data management and effective data sharing have become paramount in the rapidly evolving healthcare landscape, especially with the growing integration of the Internet of Medical Things (IoMT). The rise of generative artificial intelligence has further elevated Multi-modal Large Language Models (MLLMs) as essential tools for managing and optimizing healthcare data in IoMT. MLLMs can support multi-modal inputs and generate diverse types of content by leveraging large-scale training on vast amounts of multi-modal data. However, critical challenges persist in developing medical MLLMs, including security and freshness issues of healthcare data, affecting the output quality of MLLMs. To this end, in this paper, we propose a hybrid Retrieval-Augmented Generation (RAG)-empowered medical MLLM framework for healthcare data management. This framework leverages a hierarchical cross-chain architecture to facilitate secure data training. Moreover, it enhances the output quality of MLLMs through hybrid RAG, which employs multi-modal metrics to filter various unimodal RAG results and incorporates these retrieval results as additional inputs to MLLMs. Additionally, we employ age of information to indirectly evaluate the data freshness impact of MLLMs and utilize contract theory to incentivize healthcare data holders to share their fresh data, mitigating information asymmetry during data sharing. Finally, we utilize a generative diffusion model-based deep reinforcement learning algorithm to identify the optimal contract for efficient data sharing. Numerical results demonstrate the effectiveness of the proposed schemes, which achieve secure and efficient healthcare data management. Copyright © 2024, The Authors. All rights reserved.

关键词： Information management

Low-Light Enhancement Effect on Classification and Detection: An Empirical Study

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Xu Lai, Zhihui Jie, Zhou Gao, Can Hou, Xianxu Zhang, Ya-Nan Shen, Linlin The Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen518060 China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen518060 China School of AI and Advanced Computing Xi’an Jiaotong-Liverpool University China The National Engineering Laboratory for Big Data System Computing Technology Shenzhen University SZU Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society Guangdong Shenzhen518060 China

Low-light images are commonly encountered in real-world scenarios, and numerous low-light image enhancement (LLIE) methods have been proposed to improve the visibility of these images. The primary goal of LLIE is to generate clearer images that are more visually pleasing to humans. However, the impact of LLIE methods in high-level vision tasks, such as image classification and object detection, which rely on high-quality image datasets, is not well explored. To explore the impact, we comprehensively evaluate LLIE methods on these high-level vision tasks by utilizing an empirical investigation comprising image classification and object detection experiments. The evaluation reveals a dichotomy: While Low-Light Image Enhancement (LLIE) methods enhance human visual interpretation, their effect on computer vision tasks is inconsistent and can sometimes be harmful. Our findings suggest a disconnect between image enhancement for human visual perception and for machine analysis, indicating a need for LLIE methods tailored to support high-level vision tasks effectively. This insight is crucial for the development of LLIE techniques that align with the needs of both human and machine vision. Copyright © 2024, The Authors. All rights reserved.

关键词： Machine vision

Learning Task-aware Robust Deep Learning systems

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Han, Keji Li, Yun Long, Xianzhong Ge, Yao Nanjing University of Posts and Telecommunications China Jiangsu Key Laboratory of Big Data Security & Intelligent Processing

Many works demonstrate that deep learning system is vulnerable to adversarial attack. A deep learning system consists of two parts: the deep learning task and the deep model. Nowadays, most existing works investigate the impact of the deep model on robustness of deep learning systems, ignoring the impact of the learning task. In this paper, we adopt the binary and interval label encoding strategy to redefine the classification task and design corresponding loss to improve robustness of the deep learning system. Our method can be viewed as improving the robustness of deep learning systems from both the learning task and deep model. Experimental results demonstrate that our learning task-aware method is much more robust than traditional classification while retaining the accuracy. © 2020, CC BY.

关键词： Deep learning

Adaptive Perception for Unified Visual Multi-modal Object Tracking

学校读者我要写书评

暂无评论

IEEE Transactions on Artificial Intelligence

IEEE Transactions on Artificial Intelligence 2025年

作者： Hu, Xiantao Zhong, Bineng Liang, Qihua Shi, Liangtao Mo, Zhiyi Tai, Ying Yang, Jian Guangxi Normal University Guangxi Key Lab of Multi-Source Information Mining Guilin541004 China Guangxi Normal University Security Guilin541004 China Guangxi Normal University Intelligent Technology Guilin541004 China Guangxi Normal University Ministry of Education Guilin541004 China Wuzhou University School of Data Science Wuzhou543002 China Wuzhou University Software Engineering Wuzhou543002 China Nanjing University State Key Laboratory for Novel Software Technology Suzhou215163 China Nanjing University of Science and Technology School of Computer Science Nanjing210094 China Nanjing University of Science and Technology Engineering Nanjing210094 China

Recently, many multi-modal trackers prioritize RGB as the dominant modality, treating other modalities as auxiliary, and fine-tuning separately various multi-modal tasks. This imbalance in modality dependence limits the ability of methods to dynamically utilize complementary information from each modality in complex scenarios, making it challenging to fully perceive the advantages of multi-modal. As a result, a unified parameter model often underperforms in various multi-modal tracking tasks. To address this issue, we propose APTrack, a novel unified tracker designed for multi-modal adaptive perception. Unlike previous methods, APTrack explores a unified representation through an equal modeling strategy. This strategy allows the model to dynamically adapt to various modalities and tasks without requiring additional fine-tuning between different tasks. Moreover, our tracker integrates an adaptive modality interaction (AMI) module that efficiently bridges cross-modality interactions by generating learnable tokens. Experiments conducted on five diverse multi-modal datasets (RGBT234, LasHeR, VisEvent, DepthTrack, and VOT-RGBD2022) demonstrate that APTrack not only surpasses existing state-of-the-art unified multi-modal trackers but also outperforms trackers designed for specific multi-modal tasks. © 2020 IEEE.

关键词： Multicarrier modulation

Video-Based 3D Human Pose Estimation Research

学校读者我要写书评

暂无评论

Video-Based 3D Human Pose Estimation Research

IEEE Conference on Industrial Electronics and Applications (ICIEA)

作者： Siting Tao Zhi Zhang College of Computer Science and Technology Wuhan University of Science and Technology Wuhan China Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System Wuhan China China Big Data Science and Engineering Research Institute Wuhan University of Science and Technology Wuhan China China

ISBN: (纸本)9781665409858

At present, most of the video-based 3D human pose estimation methods process the video frame by frame, ignoring the timing information of the upper and lower frames of the video. When there is a single-frame estimation error caused by self-occlusion, the occlusion part will be Some strange movements appear due to information loss, and the estimated results make it difficult to produce accurate motion sequences. Therefore, considering the continuity of the upper and lower frames of the video, this paper adopts the HMR algorithm and introduces a bidirectional long-term memory network (BI-LSTM) in the encoding part to process the video sequence, and then outputs the parameters of the human body model through the SMPL parameters and inputs them into the discriminator for confrontation, train. In order for the discriminator to also take into account the semantic information of the temporal context, the discriminator in this paper also introduces a BI-LSTM network to improve the authenticity of the generated poses. Experimental results show that the accuracy of the model is improved when self-occlusion occurs.

关键词： Training Industrial electronics Solid modeling Estimation error Three-dimensional displays Biological system modeling Pose estimation

Towards Understanding and Boosting Adversarial Transferability from a Distribution Perspective

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Zhu, Yao Chen, Yuefeng Li, Xiaodan Chen, Kejiang He, Yuan Tian, Xiang Zheng, Bolun Chen, Yaowu Huang, Qingming The Zhejiang University Hangzhou310013 China The Security Department of Alibaba Group China The CAS Key Laboratory of Electro-Magnetic Space Information University of Science and Technology of China China The Hangzhou Dianzi University China Zhejiang Provincial Key Laboratory for Network Multimedia Technologies China The Zhejiang University Zhejiang Provincial Key Laboratory for Network Multimedia Technologies China The Zhejiang University Zhejiang University Embedded System Engineering Research Center Ministry of Education of China China The University of Chinese Academy of Sciences China The Key Laboratory of Intelligent Information Processing Chinese Academy of Sciences China

Transferable adversarial attacks against Deep neural networks (DNNs) have received broad attention in recent years. An adversarial example can be crafted by a surrogate model and then attack the unknown target model successfully, which brings a severe threat to DNNs. The exact underlying reasons for the transferability are still not completely understood. Previous work mostly explores the causes from the model perspective, e.g., decision boundary, model architecture, and model capacity. adversarial attacks against Deep neural networks (DNNs) have received broad attention in recent years. An adversarial example can be crafted by a surrogate model and then attack the unknown target model successfully, which brings a severe threat to DNNs. The exact underlying reasons for the transferability are still not completely understood. Previous work mostly explores the causes from the model perspective, e.g., decision boundary, model architecture, and model capacity. Here, we investigate the transferability from the data distribution perspective and hypothesize that pushing the image away from its original distribution can enhance the adversarial transferability. To be specific, moving the image out of its original distribution makes different models hardly classify the image correctly, which benefits the untargeted attack, and dragging the image into the target distribution misleads the models to classify the image as the target class, which benefits the targeted attack. Towards this end, we propose a novel method that crafts adversarial examples by manipulating the distribution of the image. We conduct comprehensive transferable attacks against multiple DNNs to demonstrate the effectiveness of the proposed method. Our method can significantly improve the transferability of the crafted attacks and achieves state-of-the-art performance in both untargeted and targeted scenarios, surpassing the previous best method by up to 40% in some cases. In summary, our work provides new in

关键词： Deep neural networks

FuXi Weather: A data-to-forecast machine learning system for global weather

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Sun, Xiuyu Zhong, Xiaohui Xu, Xiaoze Huang, Yuanqing Li, Hao Neelin, J. David Chen, Deliang Feng, Jie Han, Wei Wu, Libo Qi, Yuan Shanghai Academy of Artificial Intelligence for Science Shanghai200232 China Artificial Intelligence Innovation and Incubation Institute Fudan University Shanghai200433 China School of Atmospheric Physics Nanjing University of Information Science and Technology Nanjing210044 China Earth System Modeling and Prediction Centre China Meteorological Administration Beijing100081 China Department of Atmospheric and Oceanic Sciences University of California Los Angeles90095 United States Ministry of Education Key Laboratory for Earth System Modeling Department of Earth System Science Tsinghua University Beijing100084 China University of Gothenburg Gothenburg Sweden Department of Atmospheric and Oceanic Sciences Institute of Atmospheric Sciences Fudan University Shanghai200433 China School of Data Science Fudan University Shanghai200433 China Institute for Big Data Fudan University Shanghai200433 China MOE Laboratory for National Development and Intelligent Governance Fudan University Shanghai200433 China

Weather forecasting traditionally relies on numerical weather prediction (NWP) systems that integrates global observational systems, data assimilation (DA), and forecasting models. Despite steady improvements in forecast accuracy over recent decades, further advances are increasingly constrained by high computational costs, the underutilization of vast observational datasets, and the challenges of obtaining finer resolution. These limitations, alongside the uneven distribution of observational networks, result in global disparities in forecast accuracy, leaving some regions vulnerable to extreme weather. Recent advances in machine learning present a promising alternative, providing more efficient and accurate forecasts using the same initial conditions as NWP. However, current machine learning models still depend on the initial conditions generated by NWP systems, which require extensive computational resources and expertise. Here we introduce FuXi Weather, a machine learning weather forecasting system that assimilates data from multiple satellites. Operating on a 6-hourly DA and forecast cycle, FuXi Weather generates reliable and accurate 10-day global weather forecasts at a spatial resolution of 0.25◦. FuXi Weather is the first system to achieve all-grid, all-surface, all-channel, and all-sky DA and forecasting, extending skillful forecast lead times beyond those of the European Centre for Medium-range Weather Forecasts (ECMWF) high-resolution forecasts (HRES) while using significantly fewer observations. FuXi Weather consistently outperforms ECMWF HRES in observation-sparse regions, such as central Africa, demonstrating its potential to improve forecasts where observational infrastructure is limited. © 2024, CC BY.

关键词： Weather forecasting

Modeling and analysis of the dynamic mechanism of rumor propagation considering the process of rumor refuting

学校读者我要写书评

暂无评论

Nonlinear Science

Nonlinear Science 2025年 3卷

作者： Xutao Li Junshan Pan Yong Hu Hubei Provincial Engineering Research Center of Intelligent Connected Vehicle Network Security Key Laboratory of Intelligent Sensing System and Security (Hubei University) Ministry of Education School of Cyber Science and Technology Hubei University Wuhan 430062 Hubei China School of Physics Huazhong University of Science and Technology Wuhan 430074 Hubei China

The boost of rumor spreading by the Internet has caused a series of social problems. Refuting the rumor timely can alleviate the harm caused by rumor dissemination. To investigate the propagation mechanism of rumor affected by rumor refuting on online social platforms, we propose the Susceptible-Infective-Remover-Debunker-Remover (SIRDR) model to describe the propagation dynamics of rumor and rumor refutation. To conform closely to the actual scenarios of rumor dissemination, we consider a refined division of the population involved in propagation and introduce an information exposure attractiveness index and forgetting mechanisms to characterize individual spreading behavior. We then investigate the extent to which the suppression of rumor dissemination by refuting rumors at different spreading stages. The proposed model is validated by the analysis of real datasets collected from Sina Weibo and Twitter. The R-squared values of data 1 are 0.947 and 0.985, the R-squared values of data 2 are 0.993 and 0.976, while the R-squared values of data 3 are 0.948 and 0.956, indicating that our model is very consistent with the real rumor propagation process. Especially, we find out that the suppression effect of rumor refutation on rumor propagation varies greatly with its release time. Our work may offer theoretical support for rumors governance in cyberspace.

关键词：

Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Luo, Anwei Cai, Rizhao Kong, Chenqi Ju, Yakun Kang, Xiangui Huang, Jiwu Kot, Alex C. The School of Information Technology Jiangxi University of Finance and Economics Nanchang330013 China The School of Computer Science and Engineering Sun Yat-Sen University Guangzhou510006 China Lab. School of Electrical and Electronic Engineering Nanyang Technology University Singapore The Guangdong Key Laboratory of Intelligent Information Processing National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China The China-Singapore International Joint Research Institute Singapore

With the rapid progress of generative models, the current challenge in face forgery detection is how to effectively detect realistic manipulated faces from different unseen domains. Though previous studies show that pre-trained Vision Transformer (ViT) based models can achieve some promising results after fully fine-tuning on the Deepfake dataset, their generalization performances are still unsatisfactory. One possible reason is that fully fine-tuned ViT-based models may disrupt the pre-trained features [1], [2] and overfit to some data-specific patterns [3]. To alleviate this issue, we present a Forgery-aware Adaptive Vision Transformer (FA-ViT) under the adaptive learning paradigm, where the parameters in the pre-trained ViT are kept fixed while the designed adaptive modules are optimized to capture forgery features. Specifically, a global adaptive module is designed to model long-range interactions among input tokens, which takes advantage of self-attention mechanism to mine global forgery clues. To further explore essential local forgery clues, a local adaptive module is proposed to expose local inconsistencies by enhancing the local contextual association. In addition, we introduce a fine-grained adaptive learning module that emphasizes the common compact representation of genuine faces through relationship learning in fine-grained pairs, driving these proposed adaptive modules to be aware of fine-grained forgery-aware information. Extensive experiments demonstrate that our FA-ViT achieves state-of-the-arts results in the cross-dataset evaluation, and enhances the robustness against unseen perturbations. Particularly, FA-ViT achieves 93.83% and 78.32% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation. The code and trained model have been released at: https://***/LoveSiameseCat/FAViT. Copyright © 2023, The Authors. All rights reserved.

关键词： Contrastive Learning