检索结果-内蒙古大学图书馆

Fed-FA: theoretically modeling client data divergence for federated language backdoor defense 23

学校读者我要写书评

暂无评论

Fed-FA: theoretically modeling client data divergence for fe...

Proceedings of the 37th International Conference on Neural Information Processing Systems

作者： Zhiyuan Zhang Deli Chen Hao Zhou Fandong Meng Jie Zhou Xu Sun National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University and Pattern Recognition Center WeChat AI Tencent Inc. China Pattern Recognition Center WeChat AI Tencent Inc. China National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University

Federated learning algorithms enable neural network models to be trained across multiple decentralized edge devices without sharing private data. However, they are susceptible to backdoor attacks launched by malicious clients. Existing robust federated aggregation algorithms heuristically detect and exclude suspicious clients based on their parameter distances, but they are ineffective on Natural Language Processing (NLP) tasks. The main reason is that, although text backdoor patterns are obvious at the underlying dataset level, they are usually hidden at the parameter level, since injecting backdoors into texts with discrete feature space has less impact on the statistics of the model parameters. To settle this issue, we propose to identify backdoor clients by explicitly modeling the data divergence among clients in federated NLP systems. Through theoretical analysis, we derive the f-divergence indicator to estimate the client data divergence with aggregation updates and Hessians. Furthermore, we devise a dataset synthesization method with a Hessian reassignment mechanism guided by the diffusion theory to address the key challenge of inaccessible datasets in calculating clients' data Hessians. We then present the novel Federated F-Divergence-Based Aggregation (Fed-FA) algorithm, which leverages the f-divergence indicator to detect and discard suspicious clients. Extensive empirical results show that Fed-FA outperforms all the parameter distance-based methods in defending against backdoor attacks among various natural language backdoor attack scenarios.

关键词：

WDA-Net: Weakly-Supervised Domain Adaptive Segmentation of Electron Microscopy

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Qiu, Dafei Yi, Jiajin Peng, Jialin Xiamen Key Laboratory of Computer Vision and Pattern Recognition Huaqiao University China Meitu Inc. Xiamen China College of Computer Science and Technology Huaqiao University China

Accurate segmentation of organelle instances is essential for electron microscopy analysis. Despite the outstanding performance of fully supervised methods, they highly rely on sufficient per-pixel annotated data and are sensitive to domain shift. Aiming to develop a highly annotation-efficient approach with competitive performance, we focus on weakly-supervised domain adaptation (WDA) with a type of extremely sparse and weak annotation demanding minimal annotation efforts, i.e., sparse point annotations on only a small subset of object instances. To reduce performance degradation arising from domain shift, we explore multi-level transferable knowledge through conducting three complementary tasks, i.e., counting, detection, and segmentation, constituting a task pyramid with different levels of domain invariance. The intuition behind this is that after investigating a related source domain, it is much easier to spot similar objects in the target domain than to delineate their fine boundaries. Specifically, we enforce counting estimation as a global constraint to the detection with sparse supervision, which further guides the segmentation. A cross-position cut-and-paste augmentation is introduced to further compensate for the annotation sparsity. Extensive validations show that our model with only 15% point annotations can achieve comparable performance as supervised models and shows robustness to annotation selection. © 2022, CC BY-NC-ND.

关键词： Electron microscopy

Unsupervised HDR Image and Video Tone Mapping via Contrastive Learning

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Cao, Cong Yue, Huanjing Liu, Xin Yang, Jingyu School of Electrical and Information Engineering Tianjin University China Computer Vision and Pattern Recognition Laboratory School of Engineering Science Lappeenranta-Lahti University of Technology LUT Lappeenranta Finland

Capturing high dynamic range (HDR) images (videos) is attractive because it can reveal the details in both dark and bright regions. Since the mainstream screens only support low dynamic range (LDR) content, tone mapping algorithm is required to compress the dynamic range of HDR images (videos). Although image tone mapping has been widely explored, video tone mapping is lagging behind, especially for the deep-learning-based methods, due to the lack of HDR-LDR video pairs. In this work, we propose a unified framework (IVTMNet) for unsupervised image and video tone mapping. To improve unsupervised training, we propose domain and instance based contrastive learning loss. Instead of using a universal feature extractor, such as VGG to extract the features for similarity measurement, we propose a novel latent code, which is an aggregation of the brightness and contrast of extracted features, to measure the similarity of different pairs. We totally construct two negative pairs and three positive pairs to constrain the latent codes of tone mapped results. For the network structure, we propose a spatial-feature-enhanced (SFE) module to enable information exchange and transformation of nonlocal regions. For video tone mapping, we propose a temporal-feature-replaced (TFR) module to efficiently utilize the temporal correlation and improve the temporal consistency of video tone-mapped results. We construct a large-scale unpaired HDR-LDR video dataset to facilitate the unsupervised training process for video tone mapping. Experimental results demonstrate that our method outperforms state-of-the-art image and video tone mapping methods. Our code and dataset are available at https://***/caocong/UnCLTMO. Copyright © 2023, The Authors. All rights reserved.

关键词： Conformal mapping

MUSES: 3D-Controllable Image Generation via Multi-Modal Agent Collaboration

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Ding, Yanbo Zhuang, Shaobin Li, Kunchang Yue, Zhengrong Qiao, Yu Wang, Yali Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China School of Artificial Intelligence University of Chinese Academy of Sciences China Shanghai Artificial Intelligence Laboratory China Shanghai Jiao Tong University China

Despite recent advancements in text-to-image generation, most existing methods struggle to create images with multiple objects and complex spatial relationships in the 3D world. To tackle this limitation, we introduce a generic AI system, namely MUSES, for 3D-controllable image generation from user queries. Specifically, our MUSES develops a progressive workflow with three key components, including (1) Layout Manager for 2D-to-3D layout lifting, (2) Model Engineer for 3D object acquisition and calibration, (3) Image Artist for 3D-to-2D image rendering. By mimicking the collaboration of human professionals, this multi-modal agent pipeline facilitates the effective and automatic creation of images with 3D-controllable objects, through an explainable integration of top-down planning and bottom-up generation. Additionally, existing benchmarks lack detailed descriptions of complex Copyright © 2024, The Authors. All rights reserved.

关键词： 3D modeling

WDA-Net: Weakly-Supervised Domain Adaptive Segmentation of Electron Microscopy

学校读者我要写书评

暂无评论

WDA-Net: Weakly-Supervised Domain Adaptive Segmentation of E...

IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

作者： Dafei Qiu Jiajin Yi Jialin Peng Xiamen Key Laboratory of Computer Vision and Pattern Recognition Huaqiao University China Meitu Inc. Xiamen China College of Computer Science and Technology Huaqiao University China

ISBN: (数字)9781665468190

ISBN: (纸本)9781665468206

关键词： Degradation Adaptation models Annotations Biological system modeling Estimation Robustness Data models

Modeling Inter-Intra Heterogeneity for Graph Federated Learning 39

学校读者我要写书评

暂无评论

Modeling Inter-Intra Heterogeneity for Graph Federated Learn...

39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025

作者： Yu, Wentao Chen, Shuo Tong, Yongxin Gu, Tianlong Gong, Chen School of Computer Science and Engineering Nanjing University of Science and Technology China Center for Advanced Intelligence Project RIKEN Japan State Key Laboratory of Complex & Critical Software Environment Beihang University China Engineering Research Center of Trustworthy AI Ministry of Education Jinan University China Department of Automation Institute of Image Processing and Pattern Recognition Shanghai Jiao Tong University China

ISBN: (纸本)157735897X

Heterogeneity is a fundamental and challenging issue in federated learning, especially for the graph data due to the complex relationships among the graph nodes. To deal with the heterogeneity, lots of existing methods perform the weighted federation based on their calculated similarities between pairwise clients (i.e., subgraphs). However, their inter-subgraph similarities estimated with the outputs of local models are less reliable, because the final outputs of local models may not comprehensively represent the real distribution of subgraph data. In addition, they ignore the critical intra-heterogeneity which usually exists within each subgraph itself. To address these issues, we propose a novel Federated learning method by integrally modeling the Inter-Intra Heterogeneity (FedIIH). For the inter-subgraph relationship, we propose a novel hierarchical variational model to infer the whole distribution of subgraph data in a multi-level form, so that we can accurately characterize the inter-subgraph similarities with the global perspective. For the intra-heterogeneity, we disentangle the subgraph into multiple latent factors and partition the model parameters into multiple parts, where each part corresponds to a single latent factor. Our FedIIH not only properly computes the distribution similarities between subgraphs, but also learns disentangled representations that are robust to irrelevant factors within subgraphs, so that it successfully considers the inter- and intra- heterogeneity simultaneously. Extensive experiments on six homophilic and five heterophilic graph datasets in both non-overlapping and overlapping settings demonstrate the effectiveness of our method when compared with nine state-of-the-art methods. Specifically, FedIIH averagely outperforms the second-best method by a large margin of 5.79% on all heterophilic datasets. Copyright © 2025, Association for the Advancement of Artificial Intelligence (***). All rights reserved.

关键词： Federated learning

MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Ding, Yanbo Hu, Xirui Guo, Zhizhi Wang, Yali Shenzhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China China Telecom China School of Computer Science and Technology Xi’an Jiaotong University China School of Artificial Intelligence University of Chinese Academy of Sciences China Shanghai Artificial Intelligence Laboratory China

Human image animation has gained increasing attention and developed rapidly due to its broad applications in digital humans. However, existing methods rely largely on 2D-rendered pose images for motion guidance, which limits generalization and discards essential 3D information for open-world animation. To tackle this problem, we propose MTVCrafter (Motion Tokenization Video Crafter), the first framework that directly models raw 3D motion sequences (i.e., 4D motion) for human image animation. Specifically, we introduce 4DMoT (4D motion tokenizer) to quantize 3D motion sequences into 4D motion tokens. Compared to 2D-rendered pose images, 4D motion tokens offer more robust spatio-temporal cues and avoid strict pixel-level alignment between pose image and character, enabling more flexible and disentangled control. Then, we introduce MV-DiT (Motion-aware Video DiT). By designing unique motion attention with 4D positional encodings, MV-DiT can effectively leverage motion tokens as 4D compact yet expressive context for human image animation in the complex 3D world. Hence, it marks a significant step forward in this field and opens a new direction for pose-guided human video generation. Experiments show that our MTVCrafter achieves state-of-the-art results with an FID-VID of 6.98, surpassing the second-best by 65%. Powered by robust motion tokens, MTVCrafter also generalizes well to diverse open-world characters (single/multiple, full/half-body) across various styles and scenarios. Our video demos and code are on: https://***/DINGYANB/MTVCrafter. Copyright © 2025, The Authors. All rights reserved.

关键词： Three dimensional computer graphics

OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer

学校读者我要写书评

暂无评论

OSRT: Omnidirectional Image Super-Resolution with Distortion...

Conference on computer vision and pattern recognition (CVPR)

作者： Fanghua Yu Xintao Wang Mingdeng Cao Gen Li Ying Shan Chao Dong ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences ARC Tencent PCG The University of Tokyo Platform Technologies Tencent Online Video Shanghai Artificial Intelligence Laboratory

Omnidirectional images (ODIs) have obtained lots of research interest for immersive experiences. Although ODIs require extremely high resolution to capture details of the entire scene, the resolutions of most ODIs are insufficient. Previous methods attempt to solve this issue by image super-resolution (SR) on equirectangular projection (ERP) images. However, they omit geometric properties of ERP in the degradation process, and their models can hardly generalize to real ERP images. In this paper, we propose Fisheye downsampling, which mimics the real-world imaging process and synthesizes more realistic low-resolution samples. Then we design a distortion-aware Transformer (OSRT) to modulate ERP distortions continuously and self-adaptively. Without a cumbersome process, OSRT outperforms previous methods by about 0.2dB on PSNR. Moreover, we propose a convenient data augmentation strategy, which synthesizes pseudo ERP images from plain images. This simple strategy can alleviate the over-fitting problem of large networks and significantly boost the performance of ODISR. Extensive experiments have demonstrated the state-of-the-art performance of our OSRT.

关键词：

Design of surrogate models in civil engineering by neural networks

学校读者我要写书评

暂无评论

Design of surrogate models in civil engineering by neural ne...

South Eastern European Design Automation, computer Engineering, computer Networks and Social Media Conference (SEEDA-CECNSM)

作者： Vojtěch Drahý Radek Mařík Heikki Kälviäinen Department of Computer Science Czech Technical University in Prague Prague Czech Republic Department of Telecommunication Engineering Czech Technical University in Prague Prague Czech Republic Computer Vision and Pattern Recognition Laboratory Lappeenranta-Lahti University of Technology LUT Lappeenranta Finland

ISBN: (数字)9798331504489

ISBN: (纸本)9798331504496

We present a task from the critical infrastructure field in materials engineering. We created a surrogate model for the bridge construction object to determine the material parameters’ values. The work aims to use neural networks to conduct an initial investigation of the task and to find out the aspects of machine learning application. To reduce the computational complexity of the models, we designed specific neural networks whose architecture corresponds to the structure and characteristics of the processed data. Furthermore, we outcome also interpretability and justification of the model’s decision-making. The main contribution of the work is the replacement of the unknown or too complex physical, mathematical description of material objects with a neural network model.

关键词： Training Social networking (online) Computational modeling Neural networks Focusing computer architecture Machine learning Feature extraction Mathematical models Computational complexity