检索结果-内蒙古大学图书馆

ReconBoost: Boosting Can Achieve Modality Reconcilement

学校读者我要写书评

暂无评论

arxiv 2024年

作者： Hua, Cong Xu, Qianqian Bao, Shilong Yang, Zhiyong Huang, Qingming Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China Institute of Information Engineering Chinese Academy of Sciences Beijing China School of Cyber Security University of Chinese Academy of Sciences Beijing China Key Laboratory of Big Data Mining and Knowledge Management Chinese Academy of Sciences Beijing China

This paper explores a novel multi-modal alternating learning paradigm pursuing a reconciliation between the exploitation of uni-modal features and the exploration of cross-modal interactions. This is motivated by the fact that current paradigms of multi-modal learning tend to explore multi-modal features simultaneously. The resulting gradient prohibits further exploitation of the features in the weak modality, leading to modality competition, where the dominant modality overpowers the learning process. To address this issue, we study the modality-alternating learning paradigm to achieve reconcilement. Specifically, we propose a new method called ReconBoost to update a fixed modality each time. Herein, the learning objective is dynamically adjusted with a reconcilement regularization against competition with the historical models. By choosing a KL-based reconcilement, we show that the proposed method resembles Friedman's Gradient-Boosting (GB) algorithm, where the updated learner can correct errors made by others and help enhance the overall performance. The major difference with the classic GB is that we only preserve the newest model for each modality to avoid overfitting caused by ensembling strong learners. Furthermore, we propose a memory consolidation scheme and a global rectification scheme to make this strategy more effective. Experiments over six multi-modal benchmarks speak to the efficacy of the method. We release the code at https://***/huacong/ReconBoost. Copyright © 2024, The Authors. All rights reserved.

关键词： Machine learning

MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection

学校读者我要写书评

暂无评论

arxiv 2024年

作者： Zhang, Yaning Wang, Tianyi Yu, Zitong Gao, Zan Shen, Linlin Chen, Shengyong Jinan250353 China Nanyang Technological University 50 Nanyang Ave Block N 4 639798 Singapore School of Computing and Information Technology Great Bay University Dongguan523000 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen518060 China Jinan250014 China The Key Laboratory of Computer Vision and System Ministry of Education Tianjin University of Technology Tianjin300384 China Computer Vision Institute College of Computer Science and Software Engineering Shenzhen University Shenzhen518060 China National Engineering Laboratory for Big Data System Computing Technology Shenzhen University Shenzhen China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen518129 China Guangdong Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen518060 China

The rapid development of photo-realistic face generation methods has raised significant concerns in society and academia, highlighting the urgent need for robust and generalizable face forgery detection (FFD) techniques. Although existing approaches mainly capture face forgery patterns using image modality, other modalities like fine-grained noises and texts are not fully explored, which limits the generalization capability of the model. In addition, most FFD methods tend to identify facial images generated by GAN, but struggle to detect unseen diffusion-synthesized ones. To address the limitations, we aim to leverage the cutting-edge foundation model, contrastive language-image pre-training (CLIP), to achieve generalizable diffusion face forgery detection (DFFD). In this paper, we propose a novel multi-modal fine-grained CLIP (MFCLIP) model, which mines comprehensive and fine-grained forgery traces across image-noise modalities via language-guided face forgery representation learning, to facilitate the advancement of DFFD. Specifically, we devise a fine-grained language encoder (FLE) that extracts fine global language features from hierarchical text prompts. We design a multi-modal vision encoder (MVE) to capture global image forgery embeddings as well as fine-grained noise forgery patterns extracted from the richest patch, and integrate them to mine general visual forgery traces. Moreover, we build an innovative plug-and-play sample pair attention (SPA) method to emphasize relevant negative pairs and suppress irrelevant ones, allowing cross-modality sample pairs to conduct more flexible alignment. Extensive experiments and visualizations show that our model outperforms the state of the arts on different settings like cross-generator, cross-forgery, and cross-dataset evaluations. Copyright © 2024, The Authors. All rights reserved.

关键词： Alignment

DNN Task Assignment in UAV Networks: A Generative AI Enhanced Multi-Agent Reinforcement Learning Approach

学校读者我要写书评

暂无评论

arxiv 2024年

作者： Tang, xin Chen, Qian Weng, Wenjie Liao, Binhan Wang, Jiacheng Cao, xianbin Li, xiaohuan Guangxi University Key Laboratory of Intelligent Networking and Scenario System School of Information and Communication Guilin University of Electronic Technology Guilin541004 China The College of Computing and Data Science Nanyang Technological University 639798 Singapore School of Architecture and Transportation Engineering Guilin University of Electronic Technology Guilin541004 China National Engineering Laboratory for Comprehensive Transportation Big Data Application Technology Guangxi Nanning530001 China College of Computing and Data Science Nanyang Technological University 639798 Singapore School of Electronic and Information Engineering The Key Laboratory of Advanced Technology of Near Space Information System Ministry of Industry and Information Technology of China Beihang University Beijing100191 China

Unmanned Aerial Vehicles (UAVs) possess high mobility and flexible deployment capabilities, prompting the development of UAVs for various application scenarios within the Internet of Things (IoT). The unique capabilities of UAVs give rise to increasingly critical and complex tasks in uncertain and potentially harsh environments. The substantial amount of data generated from these applications necessitates processing and analysis through deep neural networks (DNNs). However, UAVs encounter challenges due to their limited computing resources when managing DNN models. This paper presents a joint approach that combines multiple-agent reinforcement learning (MARL) and generative diffusion models (GDM) for assigning DNN tasks to a UAV swarm, aimed at reducing latency from task capture to result output. To address these challenges, we first consider the task size of the target area to be inspected and the shortest flying path as optimization constraints, employing a greedy algorithm to resolve the subproblem with a focus on minimizing the UAV’s flying path and the overall system cost. In the second stage, we introduce a novel DNN task assignment algorithm, termed GDM-MADDPG, which utilizes the reverse denoising process of GDM to replace the actor network in multi-agent deep deterministic policy gradient (MADDPG). This approach generates specific DNN task assignment actions based on agents’ observations in a dynamic environment. Simulation results indicate that our algorithm performs favorably compared to benchmarks in terms of path planning, Age of Information (AoI), energy consumption, and task load balancing. Copyright © 2024, The Authors. All rights reserved.

关键词： Motion planning

Multi-granularity Causal Structure Learning

学校读者我要写书评

暂无评论

arxiv 2023年

作者： Liang, Jiaxuan Wang, Jun Yu, Guoxian xia, Shuyin Wang, Guoyin School of Software Shandong University Jinan China SDU-NTU Joint Centre for AI Research Shandong University Jinan China Chongqing Key Laboratory of Computational Intelligence Chongqing Uni. of Posts and Telecom. Chongqing China MOE Key Laboratory of Big Data Intelligent Computing Chongqing Uni. of Posts and Telecom. Chongqing China

Unveil, model, and comprehend the causal mechanisms underpinning natural phenomena stand as fundamental endeavors across myriad scientific disciplines. Meanwhile, new knowledge emerges when discovering causal relationships from data. Existing causal learning algorithms predominantly focus on the isolated effects of variables, overlook the intricate interplay of multiple variables and their collective behavioral patterns. Furthermore, the ubiquity of high-dimensional data exacts a substantial temporal cost for causal algorithms. In this paper, we develop a novel method called MgCSL (Multi-granularity Causal Structure Learning), which first leverages sparse auto-encoder to explore coarse-graining strategies and causal abstractions from micro-variables to macro-ones. MgCSL then takes multi-granularity variables as inputs to train multilayer perceptrons and to delve the causality between variables. To enhance the efficacy on high-dimensional data, MgCSL introduces a simplified acyclicity constraint to adeptly search the directed acyclic graph among variables. Experimental results show that MgCSL outperforms competitive baselines, and finds out explainable causal connections on fMRI datasets. Copyright © 2023, The Authors. All rights reserved.

关键词： Learning algorithms

Realizing Emotional Interactions to Learn User Experience and Guide Energy Optimization for Mobile Architectures 22

学校读者我要写书评

暂无评论

Realizing Emotional Interactions to Learn User Experience an...

Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture

作者： Xueliang Li Zhuobin Shi Junyang Chen Yepang Liu National Engineering Laboratory for Big Data System Computing Technology Shenzhen University China College of Computer Science and Software Engineering Shenzhen University China Research Institute of Trustworthy Autonomous Systems Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation and Department of Computer Science and Engineering Southern University of Science and Technology China

ISBN: (纸本)9781665462723

In the age of AI, mobile architectures such as smartphones are still "cold machines"; machines do not feel. If the architecture is able to feel users' feelings and runtime user experience (UX), it will accordingly adapt performance/energy to find the optimal system-operating state that consumes the least energy to satisfy users. In this paper, we will utilize users' facial expressions (FEs) to learn their runtime UX. We know that FEs are the natural and direct way for humans to convey their emotions and feelings. Our study reveals that FEs also reflect UX. Our research for the first time quantifies the link between FEs and UX. Leveraging this link, the architecture will be able to use the front camera to see FEs and feel users' UX. Based on UX, the architecture can appropriately provision computing resources. We propose Vi-energy system to realize the above idea. Our evaluation shows that Vi-energy reduces energy consumption by 52.9% at maximum and secures UX.

关键词：

Fast Coding Mode Prediction for Intra Prediction in VVC SCC

学校读者我要写书评

暂无评论

Fast Coding Mode Prediction for Intra Prediction in VVC SCC

IEEE International Conference on Image Processing

作者： Dayong Wang Junyi Yu xin Lu Frederic Dufaux Hongwei Guo Hui Guo Ce Zhu Chongqing Key Laboratory on Big Data for Bio Intelligence Chongqing University of Posts and Telecommunications Guangxi Key Laboratory of Machine Vision and Intelligent Control Wuzhou University Chongqing Key Laboratory of Image Cognition Chongqing University of Posts and Telecommunications Faculty of Computing Engineering and Media (CEM) De Montfort University Université Paris-Saclay CNRS CentraleSupélec Laboratoire des signaux et systèmes School of Engineering Honghe University Mengzi Yunnan China University of Electronic Science and Technology of China

ISBN: (数字)9798350349399

ISBN: (纸本)9798350349405

Currently, screen content video applications are increasingly widespread in our daily lives. The latest Screen Content Coding (SCC) standard, known as Versatile Video Coding (VVC) SCC, employs screen content Coding Modes (CMs) selection. While VVC SCC achieves high coding efficiency, its coding complexity poses a significant obstacle to the further widespread adoption of screen content video. Hence, it is crucial to enhance the coding speed of VVC SCC. In this paper, we propose a fast mode and splitting decision for Intra prediction in VVC SCC. Specifically, we initially exploit deep learning techniques to predict content types for all CUs. Subsequently, we examine CM distributions of different content types to predict candidate CMs for CUs. We then introduce early skip and early terminate CM decisions for different content types of CUs to further eliminate unlikely CMs. Finally, we develop Block-based Differential Pulse-Code Modulation (BDPCM) early termination to improve coding speed. Experimental results demonstrate that the proposed algorithm can improve coding speed by $34.95 \%$ on average while maintaining almost the same coding efficiency.

关键词： Video coding Deep learning Image coding Modulation Prediction algorithms Complexity theory Copper

On the Fine-Grained Crowd Analysis via Passive WiFi Sensing

学校读者我要写书评

暂无评论

TechRxiv

TechRxiv 2023年

作者： Hao, Lifei Huang, Baoqi Jia, Bing Mao, Guoqiang The Engineering Research Center of Ecological Big Data Ministry of Education The Inner Mongolia A.R. Key Laboratory of Wireless Networking and Mobile Computing The College of Computer Science Inner Mongolia University Hohhot010021 China The Research Institute of Smart Transportation Xidian University Xi’an710071 China

Regarding the passive WiFi sensing based crowd analysis, this paper first theoretically investigates its limitations, and then proposes a deep learning based scheme targeted for returning fine-grained crowd states in large surveillance areas. To this end, three key challenges are coped with: to relieve the influences of the randomness and sparsity induced by passive WiFi sensing, an attention-based deep convolutional autoencoder model is designed to recover accurate crowd density maps in a way similar to image reconstruction;to combat the anonymity caused by MAC randomization, following the identification of local high-density crowds (LHDCs) with the density clustering algorithm, i.e. DM-DBSCAN, a bidirectional convolutional LSTM based model is employed to infer LHDC speeds;to overcome the absence of passive WiFi sensing datasets for model training, three semi-synthetic datasets are produced by emulating passive WiFi sensing with practical pedestrian tracking datasets. Extensive experiments confirm that, the proposed scheme significantly outperforms existing WiFi-based methods in terms of crowd density estimation and provides superior crowd speed estimation. More importantly, the scheme can also produce consistent crowd states on a real-world dataset, revealing that it has the ability to support accurate, visualized and real-time crowd monitoring in large surveillance areas. © 2023, CC BY-NC-SA.

关键词： Convolution

Dual attention mechanism object tracking algorithm based on Fully-convolutional Siamese network

学校读者我要写书评

暂无评论

Dual attention mechanism object tracking algorithm based on ...

International Conference on Networking and Network Applications (NaNA)

作者： Sugang Ma Zixian Zhang Lei Zhang Yanping Chen xiaobao Yang Lei Pu Zhiqiang Hou School of Computer Science and Technology Xi’an University of Posts and Telecommunications Xi’an Shaanxi China School of Information Engineering Chang’an University Xi’an Shaanxi China Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing Xi’an Key Laboratory of Big Data and Intelligent Computing Xi’an University of Posts and Telecommunications Xi’an Shaanxi China School of Information and Navigation Air Force Engineering University Xi’an Shaanxi China

In an effort to the problem of insufficient tracking performance of the Fully-convolutional Siamese network (SiamFC) in complex scenarios, a dual attention mechanism object tracking algorithm based on the Fully-convolutional Siamese network is proposed to improve the generalization capability of the tracker by ameliorating the robustness of the template characteristics. Firstly, a global context attention module is appended after the backbone network of SiamFC to ameliorate the power of original feature extraction from two dimensions of spatial and channel. Then, a coordinate attention module is introduced to augment the capability of feature extraction in the channel dimension. Finally, the model of the proposed algorithm is trained on the Got-10k dataset. Five related algorithms are tested on the OTB2015 dataset, the results of experiments manifest that our algorithm outperforms the baseline trackers, the success and precision rate of the proposed algorithm are improved by 3.3% and 6.3%. The average tracking speed is 145FPS, which can demand the requirement of real-time tracking.

关键词： Target tracking Feature extraction Real-time systems Robustness Object tracking Task analysis

A Survey of Route Recommendations: Methods, Applications, and Opportunities

学校读者我要写书评

暂无评论

arxiv 2024年

作者： Zhang, Shiming Luo, Zhipeng Yang, Li Teng, Fei Li, Tianrui School of Computing and Artificial Intelligence Southwest Jiaotong University China Engineering Research Center of Sustainable Urban Intelligent Transportation Ministry of Education China National Engineering Laboratory of Integrated Transportation Big Data Application Technology Southwest Jiaotong University China Manufacturing Industry Chains Collaboration and Information Support Technology Key Laboratory of Sichuan Province Southwest Jiaotong University Sichuan Chengdu611756 China

Nowadays, with advanced information technologies deployed citywide, large data volumes and powerful computational resources are intelligentizing modern city development. As an important part of intelligent transportation, route recommendation and its applications are widely used, directly influencing citizens' travel habits. Developing smart and efficient travel routes based on big data (possibly multi-modal) has become a central challenge in route recommendation research. Our survey offers a comprehensive review of route recommendation work based on urban computing. It is organized by the following three parts: 1) Methodology-wise. We categorize a large volume of classic methods and modern deep learning methods. Also, we discuss their historical relations and reveal the edge-cutting progress. 2) Application-wise. We present numerous novel applications related to route commendation within urban computing scenarios. 3) We discuss current problems and challenges and envision several promising research directions. We believe that this survey can help relevant researchers quickly familiarize themselves with the current state of route recommendation research and then direct them to future research trends. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep learning