检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Pei, Gaozheng Lyu, Shaojie Ma, Ke Yang, Pinci Xu, Qianqian Sun, Yingfei School of Electronic Electrical and Communication Engineering UCAS Beijing China Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS Beijing China Tencent Corporate

Data-free model stealing involves replicating the functionality of a target model into a substitute model without accessing the target model’s structure, parameters, or training data. The adversary can only access the target model’s predictions for generated samples. Once the substitute model closely approximates the behavior of the target model, attackers can exploit its white-box characteristics for subsequent malicious activities, such as adversarial attacks. Existing methods within cooperative game frameworks often produce samples with high confidence for the prediction of the substitute model, which makes it difficult for the substitute model to replicate the behavior of the target model. This paper presents a new data-free model stealing approach called Query Efficient Data Generation (QEDG). We introduce two distinct loss functions to ensure the generation of sufficient samples that closely and uniformly align with the target model’s decision boundary across multiple classes. Building on the limitation of current methods, which typically yield only one piece of supervised information per query, we propose the query-free sample augmentation that enables the acquisition of additional supervised information without increasing the number of queries. Motivated by theoretical analysis, we adopt the consistency rate metric, which more accurately evaluates the similarity between the substitute and target models. We conducted extensive experiments to verify the effectiveness of our proposed method, which achieved better performance with fewer queries compared to the state-of-the-art methods on the real MLaaS scenario and five datasets. Copyright © 2024, The Authors. All rights reserved.

关键词： Structured Query Language

来源：评论

学校读者我要写书评

暂无评论

ES3: Evolving Self-Supervised Learning of Robust Audio-Visual Speech Representations

ES3: Evolving Self-Supervised Learning of Robust Audio-Visua...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Yuanhang Zhang Shuang Yang Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of Chinese Academy of Sciences Beijing China

ISBN: (数字)9798350353006

ISBN: (纸本)9798350353013

We propose a novel strategy, ES 3 , for self-supervised learning of robust audio-visual speech representations from unlabeled talking face videos. While many recent approaches for this task primarily rely on guiding the learning process using the audio modality alone to capture information shared between audio and video, we reframe the problem as the acquisition of shared, unique (modality-specific) and synergistic speech information to address the inherent asymmetry between the modalities. Based on this formulation, we propose a novel “evolving” strategy that progressively builds joint audio-visual speech representations that are strong for both uni-modal (audio & visual) and bi-modal (audio-visual) speech. First, we leverage the more easily learnable audio modality to initialize audio and visual representations by capturing audio-unique and shared speech information. Next, we incorporate video-unique speech information and bootstrap the audio-visual repre-sentations on top of the previously acquired shared knowledge. Finally, we maximize the total audio-visual speech information, including synergistic information to obtain robust and comprehensive representations. We implement ES 3 as a simple Siamese framework and experiments on both English benchmarks and a newly contributed large-scale Mandarin dataset show its effectiveness. In particular, on LRS2-BBC, our smallest model is on par with SoTA models with only 1/2 parameters and 1/8 unlabeled data (223h).

关键词： Visualization Computer vision Self-supervised learning Manuals Benchmark testing Data models Pattern recognition

来源：评论

学校读者我要写书评

暂无评论

Multi-View Consistent 3D GAN Inversion via Bidirectional Encoder

Multi-View Consistent 3D GAN Inversion via Bidirectional Enc...

引用

International Conference on Automatic Face and Gesture Recognition

作者： Haozhan Wu Hu Han Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China University of the Chinese Academy of Sciences Beijing China

ISBN: (数字)9798350394948

ISBN: (纸本)9798350394955

3D GAN inversion enables not only 3D reconstruction from a 2D image, but also novel view synthesis and image editing. Existing works ensure the novel view synthesis quality by constraining the synthesized views to conform to the real image distribution. However, most of the methods did not consider the multi-view consistency, i.e., different photos of the same 3D scene via 3D GAN inversion can be inverted to the same 3D scene. In this paper, we propose a bidirectional encoder (BiDiE) for 3D GAN inversion that can improve the multi-view consistency and alleviate the interference of camera parameter prediction errors. On the one hand, the bidirectional encoder takes real images as input, estimates the camera parameters, and performs 3D reconstruction. On the other hand, the bidirectional encoder takes randomly sampled latent code and camera parameters as input, and generates synthesized images to assist in the latent code learning process. In addition, we extend the latent space from W+ to W++ to improve its reconstruction and editing capabilities. Experiments on the FFHQ, CelebA-HQ and Multi-PIE datasets prove that our proposed method outperforms state-of-the-art methods in multi-view consistent reconstruction as well as editing capability. 1 1 Code and datasets are available at https://***/WHZMM/BiDiE

关键词： Three-dimensional displays Codes Face recognition Interference Gesture recognition Cameras Image reconstruction

来源：评论

学校读者我要写书评

暂无评论

Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement

arXiv

引用

arXiv 2024年

作者： Yuan, Zheng Zhang, Jie Wang, Yude Shan, Shiguang Chen, Xilin Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China University of Chinese Academy of Sciences Beijing100049 China

The attention mechanism has been proven effective on various visual tasks in recent years. In the semantic segmentation task, the attention mechanism is applied in various methods, including the case of both Convolution Neural Networks (CNN) and Vision Transformer (ViT) as backbones. However, we observe that the attention mechanism is vulnerable to patch-based adversarial attacks. Through the analysis of the effective receptive field, we attribute it to the fact that the wide receptive field brought by global attention may lead to the spread of the adversarial patch. To address this issue, in this paper, we propose a Robust Attention Mechanism (RAM) to improve the robustness of the semantic segmentation model, which can notably relieve the vulnerability against patch-based attacks. Compared to the vallina attention mechanism, RAM introduces two novel modules called Max Attention Suppression and Random Attention Dropout, both of which aim to refine the attention matrix and limit the influence of a single adversarial patch on the semantic segmentation results of other positions. Extensive experiments demonstrate the effectiveness of our RAM to improve the robustness of semantic segmentation models against various patch-based attack methods under different attack settings. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

ACT-MNMT: Auto-Constriction Turning for Multilingual Neural Machine Translation

arXiv

引用

arXiv 2024年

作者： Dai, Shaojie Liu, Xin Luo, Ping Yu, Yue Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences China Peng Cheng Laboratory China University of Chinese Academy of Sciences China

Large language model (LLM) has achieved promising performance in multilingual machine translation tasks through zero/few-shot prompts or prompt-tuning. However, due to the mixture of multilingual data during the pre-training of LLM, the LLM-based translation models face the off-target issue in both prompt-based methods, including a series of phenomena, namely instruction misunderstanding, translation with wrong language and over-generation. For this issue, this paper introduces an Auto-Constriction Turning mechanism for Multilingual Neural Machine Translation (ACT-MNMT), which is a novel supervised fine-tuning mechanism and orthogonal to the traditional prompt-based methods. In this method, ACT-MNMT automatically constructs a constrained template in the target side by adding trigger tokens ahead of the ground truth. Furthermore, trigger tokens can be arranged and combined freely to represent different task semantics, and they can be iteratively updated to maximize the label likelihood. Experiments are performed on WMT test sets with multiple metrics, and the experimental results demonstrate that ACT-MNMT achieves substantially improved performance across multiple translation directions and reduce the off-target phenomena in the translation. © 2024, CC BY.

关键词： Neural machine translation

来源：评论

学校读者我要写书评

暂无评论

Unified segment-to-segment framework for simultaneous sequence generation 23

Unified segment-to-segment framework for simultaneous sequen...

引用

Proceedings of the 37th International Conference on Neural information processing Systems

作者： Shaolei Zhang Yang Feng Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences (ICT/CAS) and University of Chinese Academy of Sciences

Simultaneous sequence generation is a pivotal task for real-time scenarios, such as streaming speech recognition, simultaneous machine translation and simultaneous speech translation, where the target sequence is generated while receiving the source sequence. The crux of achieving high-quality generation with low latency lies in identifying the optimal moments for generating, accomplished by learning a mapping between the source and target sequences. However, existing methods often rely on task-specific heuristics for different sequence types, limiting the model's capacity to adaptively learn the source-target mapping and hindering the exploration of multi-task learning for various simultaneous tasks. In this paper, we propose a unified segment-to-segment framework (Seg2Seg) for simultaneous sequence generation, which learns the mapping in an adaptive and unified manner. During the process of simultaneous generation, the model alternates between waiting for a source segment and generating a target segment, making the segment serve as the natural bridge between the source and target. To accomplish this, Seg2Seg introduces a latent segment as the pivot between source to target and explores all potential source-target mappings via the proposed expectation training, thereby learning the optimal moments for generating. Experiments on multiple simultaneous generation tasks demonstrate that Seg2Seg achieves state-of-the-art performance and exhibits better generality across various tasks. Code is available at: https://***/ictnlp/Seg2Seg.

关键词：

来源：评论

学校读者我要写书评

暂无评论

SKT-MOT and DyTracker: A Multiobject Tracking Dataset and a Dynamic Tracker for Speed Skating Video

引用

Scientific Programming 2023年第1期2023卷

作者： Wang, Junwu Li, Zongmin Li, Yachuan Yang, Shaobo Wang, Ben Li, Hua Qingdao266580 China Key Laboratory of Intelligent Information Processing Institute of Computing Technology Chinese Academy of Sciences Beijing100190 China

Speed skating serves as a significant application domain for multiobject tracking (MOT), presenting unique challenges such as frequent occlusion, highly similar appearances, and motion blur. To address these challenges, this paper constructs an MOT dataset called SKT-MOT for speed skating and analyzes the shortcomings of existing datasets and methods. Accordingly, we propose a dynamic MOT method called DyTracker. The method builds upon the DeepSORT baseline and enhances three key modules. At the global level, we design the track dynamic management (TDM) algorithm. In the motion branch, a novel metric is proposed to evaluate occlusion and Kalman filter dynamic update (KFDU) is implemented. In the appearance branch, we account for the difference in human posture and propose the feature dynamic selection and updating (FDSU) strategy. This makes our DyTracker flexible and efficient to achieve a multiobject tracking accuracy (MOTA) of 93.70% and identification F1 (IDF1) score of 92.39% on SKT-MOT, which is a significant advantage over existing SOTA methods. To validate the generalization of our proposed module, two dynamic update modules are inserted into other methods and validated on the public dataset MOT17, and the accuracy is generally improved by 0.2%-0.6%. © 2023 Junwu Wang et al.

关键词：

来源：评论

学校读者我要写书评

暂无评论

FLIP-80M: 80 Million Visual-Linguistic Pairs for Facial Language-Image Pre-Training 24

FLIP-80M: 80 Million Visual-Linguistic Pairs for Facial Lang...

引用

32nd ACM International Conference on Multimedia, MM 2024

作者： Li, Yudong Hou, Xianxu Dezhi, Zheng Shen, Linlin Zhao, Zhe School of Computer Science and Software Engineering Shenzhen University Shenzhen China Shenzhen Institute of Artificial Intelligence and Robotics for Society Shenzhen China School of AI and Advanced Computing Xi'an Jiaotong-Liverpool University Shenzhen China Guangdong Provincial Key Laboratory of Intelligent Information Processing Shenzhen University Shenzhen China Tencent AI Lab Beijing China

ISBN: (纸本)9798400706868

While significant progress has been made in multi-modal learning driven by large-scale image-text datasets, there is still a noticeable gap in the availability of such datasets within the facial domain. To facilitate and advance the field of facial representation learning, we present FLIP-80M, a large-scale visual-linguistic dataset comprising over 80 million face images paired with text descriptions. FLIP-80M is constructed by leveraging the large openly available image-text-pair dataset LAION-5B and a mixed-method approach to filter face-related pairs from both visual and linguistic perspectives. Our curation process involves face detection, face caption classification, text de-noising, and synthesis-based image augmentation. As a result, FLIP-80M stands as the largest face-text dataset to date. To evaluate the potential of our dataset, we fine-tune the CLIP model using the proposed FLIP-80M, to create FLIP (Facial Language-Image Pretraining) and assess its representation capabilities across various downstream tasks. Our experiments demonstrate that our FLIP model achieves state-of-the-art results in a range of face analysis tasks, including face parsing, face alignment, and face attribute classification. The dataset and models are available at https://***/ydli-ai/FLIP. © 2024 ACM.

关键词： Face recognition

来源：评论

学校读者我要写书评

暂无评论

ESNet: Evolution and Succession Network for High-Resolution Salient Object Detection 41

ESNet: Evolution and Succession Network for High-Resolution ...

引用

41st International Conference on Machine Learning, ICML 2024

作者： Liu, Hongyu Cong, Runmin Li, Hua Xu, Qianqian Huang, Qingming Zhang, Wei Institute of Information Science Beijing Jiaotong University Beijing Key Laboratory of Advanced Information Science and Network Technology Beijing China School of Control Science and Engineering Shandong University Key Laboratory of Machine Intelligence and System Control Ministry of Education Jinan China School of Computer Science and Technology Hainan University Hainan China Key Laboratory of Intelligent Information Processing Institute of Computing Technology CAS Beijing China School of Computer Science and Technology University of Chinese Academy of Sciences Beijing China

Preserving details and avoiding high computational costs are the two main challenges for the High-Resolution Salient Object Detection (HRSOD) task. In this paper, we propose a two-stage HRSOD model from the perspective of evolution and succession, including an evolution stage with Low-resolution Location Model (LrLM) and a succession stage with High-resolution Refinement Model (HrRM). The evolution stage achieves detail-preserving salient objects localization on the low-resolution image through the evolution mechanisms on supervision and feature;the succession stage utilizes the shallow high-resolution features to complement and enhance the features inherited from the first stage in a lightweight manner and generate the final high-resolution saliency prediction. Besides, a new metric named Boundary-Detail-aware Mean Absolute Error (MAEBD) is designed to evaluate the ability to detect details in high-resolution scenes. Extensive experiments on five datasets demonstrate that our network achieves superior performance at real-time speed (49 FPS) compared to state-of-the-art methods. Our code is publicly available at: https://***/rmcong/ESNet_ICML24. Copyright 2024 by the author(s)

关键词： Miocene

来源：评论

学校读者我要写书评

暂无评论

Deep Subdomain Alignment for Cross-domain Image Classification

Deep Subdomain Alignment for Cross-domain Image Classificati...

引用

IEEE Workshop on Applications of Computer Vision (WACV)

作者： Yewei Zhao Hu Han Shiguang Shan Xilin Chen Key Laboratory of Intelligent Information Processing Chinese Academy of Sciences (CAS) Institute of Computing Technology CAS Beijing China China University of the Chinese Academy of Sciences Beijing China

Unsupervised domain adaptation (UDA), which aims to transfer knowledge learned from a labeled source domain to an unlabeled target domain, is useful for various cross-domain image classification scenarios. A commonly used approach for UDA is to minimize the distribution differences between two domains, and subdomain alignment is found to be an effective method. However, most of the existing subdomain alignment methods are based on adversarial learning and focus on subdomain alignment procedures without considering the discriminability among individual subdomains, resulting in slow convergence and unsatisfactory adaptation results. To address these issues, we propose a novel deep subdomain alignment method for UDA in image classification, which consists of a Union Subdo-main Contrastive Learning (USCL) module and a Multi-view Subdomain Alignment (MvSA) strategy. USCL can create discriminative and dispersed subdomains by bringing samples from the same subdomain closer while pushing away samples from different subdomains. MvSA makes use of labeled source domain data and easy target domain data to perform target-to-source and target-to-target alignment. Experimental results on three image classifi-cation datasets (Office-31, Office-Home, Visda-17) demonstrate that our proposed method is effective for UDA and achieves promising results in several cross-domain image classification tasks. Our code will be available: https://***/zhaoyewei/DSACDIC.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：