检索结果-内蒙古大学图书馆

Proceedings of the 38th International Conference on Neural Information Processing Systems

作者： Xinyu Zhou Jinglun Li Lingyi Hong Kaixun Jiang Pinxue Guo Weifeng Ge Wenqiang Zhang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering and Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China and Shanghai Engineering Research Center of AI & Robotics Academy for Engineering and Technology Fudan University Shanghai China

ISBN: (纸本)9798331314385

Previous visual object tracking methods employ image-feature regression models or coordinate autoregression models for bounding box prediction. Image-feature regression methods heavily depend on matching results and do not utilize positional prior, while the autoregressive approach can only be trained using bounding boxes available in the training set, potentially resulting in suboptimal performance during testing with unseen data. Inspired by the diffusion model, denoising learning enhances the model's robustness to unseen data. Therefore, We introduce noise to bounding boxes, generating noisy boxes for training, thus enhancing model robustness on testing data. We propose a new paradigm to formulate the visual object tracking problem as a denoising learning process. However, tracking algorithms are usually asked to run in real-time, directly applying the diffusion model to object tracking would severely impair tracking speed. Therefore, we decompose the denoising learning process into every denoising block within a model, not by running the model multiple times, and thus we summarize the proposed paradigm as an in-model latent denoising learning process. Specifically, we propose a denoising Vision Transformer (ViT), which is composed of multiple denoising blocks. In the denoising block, template and search embeddings are projected into every denoising block as conditions. A denoising block is responsible for removing the noise in a predicted bounding box, and multiple stacked denoising blocks cooperate to accomplish the whole denoising process. Subsequently, we utilize image features and trajectory information to refine the denoised bounding box. Besides, we also utilize trajectory memory and visual memory to improve tracking stability. Experimental results validate the effectiveness of our approach, achieving competitive performance on several challenging datasets. The proposed in-model latent denoising tracker achieve real-time speed, rendering denoising learning

关键词：

来源：评论

学校读者我要写书评

暂无评论

PG-Attack: A Precision-Guided Adversarial Attack Framework Against Vision Foundation Models for Autonomous Driving

arXiv

引用

arXiv 2024年

作者： Fu, Jiyuan Chen, Zhaoyu Jiang, Kaixun Guo, Haijing Gao, Shuyong Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University China

Vision foundation models are increasingly employed in autonomous driving systems due to their advanced capabilities. However, these models are susceptible to adversarial attacks, posing significant risks to the reliability and safety of autonomous vehicles. Adversaries can exploit these vulnerabilities to manipulate the vehicle’s perception of its surroundings, leading to erroneous decisions and potentially catastrophic consequences. To address this challenge, we propose a novel Precision-Guided Adversarial Attack (PG-Attack) framework that combines two techniques: Precision Mask Perturbation Attack (PMP-Attack) and Deceptive Text Patch Attack (DTP-Attack). PMP-Attack precisely targets the attack region to minimize the overall perturbation while maximizing its impact on the target object’s representation in the model’s feature space. DTP-Attack introduces deceptive text patches that disrupt the model’s understanding of the scene, further enhancing the attack’s effectiveness. Our experiments demonstrate that PG-Attack successfully deceives a variety of advanced multi-modal large models, including GPT-4V, Qwen-VL, and imp-V1. Additionally, we won First-Place in the CVPR 2024 Workshop Challenge: Black-box Adversarial Attacks on Vision Foundation Models‡ and codes are available at https://***/fuhaha824/PG-Attack. Copyright © 2024, The Authors. All rights reserved.

关键词： Autonomous vehicles

来源：评论

学校读者我要写书评

暂无评论

Delving into Decision-based Black-box Attacks on Semantic Segmentation

arXiv

引用

arXiv 2024年

作者： Chen, Zhaoyu Shan, Zhengyang Chang, Jingwen Jiang, Kaixun Yang, Dingkang Cheng, Yiting Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University China

Semantic segmentation is a fundamental visual task that finds extensive deployment in applications with security-sensitive considerations. Nonetheless, recent work illustrates the adversarial vulnerability of semantic segmentation models to white-box attacks. However, its adversarial robustness against black-box attacks has not been fully explored. In this paper, we present the first exploration of black-box decision-based attacks on semantic segmentation. First, we analyze the challenges that semantic segmentation brings to decision-based attacks through the case study. Then, to address these challenges, we first propose a decision-based attack on semantic segmentation, called Discrete Linear Attack (DLA). Based on random search and proxy index, we utilize the discrete linear noises for perturbation exploration and calibration to achieve efficient attack efficiency. We conduct adversarial robustness evaluation on 5 models from Cityscapes and ADE20K under 8 attacks. DLA shows its formidable power on Cityscapes by dramatically reducing PSPNet’s mIoU from an impressive 77.83% to a mere 2.14% with just 50 queries. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

DeTrack: In-model Latent Denoising Learning for Visual Object Tracking

arXiv

引用

arXiv 2025年

作者： Zhou, Xinyu Li, Jinglun Hong, Lingyi Jiang, Kaixun Guo, Pinxue Ge, Weifeng Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering and Technology Fudan University Shanghai China

关键词： Regression analysis

来源：评论

学校读者我要写书评

暂无评论

Deeper Introspective SLAM: How to Avoid Tracking Failures Over Longer Routes?

Deeper Introspective SLAM: How to Avoid Tracking Failures Ov...

引用

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

作者： Kanwal Naveed Muhammad Latif Anjum Wajahat Hussain Donghwan Lee Robotics & Machine Intelligence (ROMI) Lab School of Electrical Engineering and Computer Science (SEECS) National University of Sciences and Technology Islamabad Pakistan Reinforcement Learning Research Lab School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon South Korea

ISBN: (数字)9798350377705

ISBN: (纸本)9798350377712

Large scale active exploration has recently revealed limitations of visual SLAM’s tracking ability. Active view planning methods based on reinforcement learning have been proposed to improve visual tracking *** this work, we expose the limitations of deep reinforcement learning-based visual SLAM over longer routes. We demonstrate that additional modalities (depth, scene layout) offer little improvement. Furthermore, reward shaping is not the main reason behind the shortsightedness of the state-of-the-art visual SLAM tracker. We propose a novel video vision transformer-based architecture that improves the farsightedness of the visual tracker, which results in the completion of longer routes with efficient *** of 60 challenging routes, our approach manages to complete 56 routes, which is a three-fold improvement over the state-of-the-art active view mapping (DI-SLAM) baseline. Interestingly, ORB-SLAM3 was unable to complete a single route without tracking failure. Our code is available at https://***/w935spuz.

关键词： Visualization Simultaneous localization and mapping Codes Layout Transformers Deep reinforcement learning Robustness Planning Intelligent robots

来源：评论

学校读者我要写书评

暂无评论

VideoPure: Diffusion-based Adversarial Purification for Video Recognition

arXiv

引用

arXiv 2025年

作者： Jiang, Kaixun Chen, Zhaoyu Fu, Jiyuan Hong, Lingyi Li, Jinglun Zhang, Wenqiang Shanghai Engineering Research Center of AI Robotics Academy for Engineering & Technology Fudan University Shanghai China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

—Recent work indicates that video recognition models are vulnerable to adversarial examples, posing a serious security risk to downstream applications. However, current research has primarily focused on adversarial attacks, with limited work exploring defense mechanisms. Furthermore, due to the spatial-temporal complexity of videos, existing video defense methods face issues of high cost, overfitting, and limited defense performance. Recently, diffusion-based adversarial purification methods have achieved robust defense performance in the image domain. However, due to the additional temporal dimension in videos, directly applying these diffusion-based adversarial purification methods to the video domain suffers performance and efficiency degradation. To achieve an efficient and effective video adversarial defense method, we propose the first diffusion-based video purification framework to improve video recognition models’ adversarial robustness: VideoPure. Given an adversarial example, we first employ temporal DDIM inversion to transform the input distribution into a temporally consistent and trajectory-defined distribution, covering adversarial noise while preserving more video structure. Then, during DDIM denoising, we leverage intermediate results at each denoising step and conduct guided spatial-temporal optimization, removing adversarial noise while maintaining temporal consistency. Finally, we input the list of optimized intermediate results into the video recognition model for multi-step voting to obtain the predicted class. We investigate the defense performance of our method against state-of-the-art black-box, gray-box, and adaptive attacks on benchmark datasets and models. Compared with other adversarial purification methods, our method overall demonstrates better defense performance against different attacks. Moreover, our method can be applied as a flexible defense plugin for video recognition models. Our code is available at https://***/deep-kaix

关键词： HTTP

来源：评论

学校读者我要写书评

暂无评论

ClickVOS: Click Video Object Segmentation

arXiv

引用

arXiv 2024年

作者： Guo, Pinxue Hong, Lingyi Zhou, Xinyu Gao, Shuyong Li, Wanyun Li, Jinglun Chen, Zhaoyu Li, Xiaoqiang Zhang, Wei Zhang, Wenqiang the Shanghai Engineering Research Center of AI&Robotics Academy for Engineering&Technology Fudan University Shanghai China the Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China the School of Computer Engineering and Science Shanghai University Shanghai China Engineering Research Center of AI&Robotics Ministry of Education Academy for Engineering&Technology Fudan University Shanghai China

—Video Object Segmentation (VOS) task aims to segment objects in videos. However, previous settings either require time-consuming manual masks of target objects at the first frame during inference or lack the flexibility to specify arbitrary objects of interest. To address these limitations, we propose the setting named Click Video Object Segmentation (ClickVOS) which segments objects of interest across the whole video according to a single click per object in the first frame. And we provide the extended datasets DAVIS-P and YouTubeVOS-P that with point annotations to support this task. ClickVOS is of significant practical applications and research implications due to its only 1-2 seconds interaction time for indicating an object, comparing annotating the mask of an object needs several minutes. However, ClickVOS also presents increased challenges. To address this task, we propose an end-to-end baseline approach named called Attention Before Segmentation (ABS), motivated by the attention process of humans. ABS utilizes the given point in the first frame to perceive the target object through a concise yet effective segmentation attention. Although the initial object mask is possibly inaccurate, in our ABS, as the video goes on, the initially imprecise object mask can self-heal instead of deteriorating due to error accumulation, which is attributed to our designed improvement memory that continuously records stable global object memory and updates detailed dense memory. In addition, we conduct various baseline explorations utilizing off-the-shelf algorithms from related fields, which could provide insights for the further exploration of ClickVOS. The experimental results demonstrate the superiority of the proposed ABS approach. Extended datasets and codes will be available at https://***/PinxueGuo/ClickVOS. Copyright © 2024, The Authors. All rights reserved.

关键词： Motion compensation

来源：评论

学校读者我要写书评

暂无评论

TagOOD: A Novel Approach to Out-of-Distribution Detection via Vision-Language Representations and Class Center Learning

arXiv

引用

arXiv 2024年

作者： Li, Jinglun Zhou, Xinyu Jiang, Kaixun Hong, Lingyi Guo, Pinxue Chen, Zhaoyu Ge, Weifeng Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China

Multimodal fusion, leveraging data like vision and language, is rapidly gaining traction. This enriched data representation improves performance across various tasks. Existing methods for out-of-distribution (OOD) detection, a critical area where AI models encounter unseen data in real-world scenarios, rely heavily on whole-image features. These image-level features can include irrelevant information that hinders the detection of OOD samples, ultimately limiting overall performance. In this paper, we propose TagOOD, a novel approach for OOD detection that leverages visionlanguage representations to achieve label-free object feature decoupling from whole images. This decomposition enables a more focused analysis of object semantics, enhancing OOD detection performance. Subsequently, TagOOD trains a lightweight network on the extracted object features to learn representative class centers. These centers capture the central tendencies of IND object classes, minimizing the influence of irrelevant image features during OOD detection. Finally, our approach efficiently detects OOD samples by calculating distance-based metrics as OOD scores between learned centers and test samples. We conduct extensive experiments to evaluate TagOOD on several benchmark datasets and demonstrate its superior performance compared to existing OOD detection methods. This work presents a novel perspective for further exploration of multimodal information utilization in OOD detection, with potential applications across various tasks. Code is available at: https://***/Jarvisgivemeasuit/tagood. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Hi-EF: Benchmarking Emotion Forecasting in Human-interaction

arXiv

引用

arXiv 2024年

作者： Wang, Haoran Mai, Xinji Tao, Zeng Wang, Yan Yu, Jiawen Zhou, Ziheng Tong, Xuan Yan, Shaoqi Zhao, Qing Gao, Shuyong Zhang, Wenqiang Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Engineering Research Center of Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China School of Information Science and Technology Fudan University Shanghai China

Affective Forecasting, a research direction in psychology that predicts individuals' future emotions, is often constrained by numerous external factors like social influence and temporal distance. To address this, we transform Affective Forecasting into a Deep Learning problem by designing an Emotion Forecasting paradigm based on two-party interactions. We propose a novel Emotion Forecasting (EF) task grounded in the theory that an individual's emotions are easily influenced by the emotions or other information conveyed during interactions with another person. To tackle this task, we have developed a specialized dataset, Humaninteraction- based Emotion Forecasting (Hi-EF), which contains 3069 two-party Multilayered-Contextual Interaction Samples (MCIS) with abundant affectiverelevant labels and three modalities. Hi-EF not only demonstrates the feasibility of the EF task but also highlights its potential. Additionally, we propose a methodology that establishes a foundational and referential baseline model for the EF task and extensive experiments are provided. The dataset and code is available HERE. Copyright © 2024, The Authors. All rights reserved.

关键词： Prediction models

来源：评论

学校读者我要写书评

暂无评论

P3S-Diffusion:A Selective Subject-driven Generation Framework via Point Supervision

arXiv

引用

arXiv 2024年

作者： Hu, Junjie Gao, Shuyong Hong, Lingyi Wang, Qishan Zhao, Yuzhou Wang, Yan Zhang, Wenqiang Shanghai Key Lab of Intelligent Information Processing School of Computer Science Fudan University Shanghai China Shanghai Engineering Research Center of AI & Robotics Academy for Engineering & Technology Fudan University Shanghai China Engineering Research Center of AI & Robotics Ministry of Education Academy for Engineering & Technology Fudan University Shanghai China

Recent research in subject-driven generation increasingly emphasizes the importance of selective subject features. Nevertheless, accurately selecting the content in a given reference image still poses challenges, especially when selecting the similar subjects in an image (e.g., two different dogs). Some methods attempt to use text prompts or pixel masks to isolate specific elements. However, text prompts often fall short in precisely describing specific content, and pixel masks are often expensive. To address this, we introduce P3S-Diffusion, a novel architecture designed for context-selected subject-driven generation via point supervision. P3S-Diffusion leverages minimal cost label (e.g., points) to generate subject-driven images. During fine-tuning, it can generate an expanded base mask from these points, obviating the need for additional segmentation models. The mask is employed for inpainting and aligning with subject representation. The P3S-Diffusion preserves fine features of the subjects through Multi-layers Condition Injection. Enhanced by the Attention Consistency Loss for improved training, extensive experiments demonstrate its excellent feature preservation and image generation capabilities. Copyright © 2024, The Authors. All rights reserved.

关键词： Pixels

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：