检索结果-内蒙古大学图书馆

IEEE Visual Communications and image processing (VCIP)

作者： Daocheng Ying Hua Yang Jun Sun Shanghai Key Lab of Digital Media Processing and Transmission Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University China

ISBN: (纸本)9781665475938

Human-object interaction (HOI) detection is a meaningful research topic on human activity understanding. Recent works have made significant progress by focusing on efficient triplet matching and leveraging image-wide features based on encoder-decoder architecture. However, the ability to gather relevant contextual information about human is limited and different sub-tasks in HOI detection are not differentiated by specific decoupling in previous methods. To this end, we propose a new transformer-based method for HOI detection, namely, Mask-Guided Transformer (MGT). Our model, which is composed of five parallel decoders with a shared encoder, not only emphasizes interactive regions by applying body features, but also disentangles the prediction of instance and interaction. We achieve a favorable result at 63.3 mAP on the well-known HOI detection dataset V-COCO.

关键词： Visual communication image processing Focusing Predictive models Transformers Decoding

来源：评论

学校读者我要写书评

暂无评论

Dual-Mode iterative denoiser: Tackling the weak label for anomaly detection

Dual-Mode iterative denoiser: Tackling the weak label for an...

引用

International Conference on Pattern Recognition

作者： Shuheng Lin Hua Yang Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Shanghai key lab of digital media processing and transmission Shanghai China

Crowd anomaly detection suffers from limited training data under weak supervision. In this paper, we propose a dual-mode iterative denoiser to tackle the weak label challenge for anomaly detection. First, we use a convolution autoencoder (CAE) in image space to act as a cluster for grouping similar video clips, where the spatial-temporal similarity helps the cluster metric to represent the reconstruction error. Then we use the graph convolution neural network (GCN) to explore the temporal correlation and the feature similarity between video clips within different rough labels, where the classifier can be constantly updated in the label denoising process. Without specific image-level labels, our model can predict the clip-level anomaly probabilities for videos. Extensive experiment results on two public datasets show that our approach performs favorably against the state-of-the-art methods.

关键词： Training Convolution Noise reduction Neural networks Training data Predictive models Pattern recognition

来源：评论

学校读者我要写书评

暂无评论

Vision-Language Consistency Guided Multi-modal Prompt Learning for Blind AI Generated image Quality Assessment

arXiv

引用

arXiv 2024年

作者： Fu, Jun Zhou, Wei Jiang, Qiuping Liu, Hantao Zhai, Guangtao The CAS Key Laboratory of Technology in Geo-Spatial Information Processing and Application System University of Science and Technology of China Hefei230027 China The School of Computer Science and Informatics Cardiff University CardiffCF24 4AG United Kingdom The School of Information Science and Engineering Ningbo University Ningbo315211 China The Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Shanghai200240 China

Recently, textual prompt tuning has shown inspirational performance in adapting Contrastive Language-image Pre-training (CLIP) models to natural image quality assessment. However, such uni-modal prompt learning method only tunes the language branch of CLIP models. This is not enough for adapting CLIP models to AI generated image quality assessment (AGIQA) since AGIs visually differ from natural images. In addition, the consistency between AGIs and user input text prompts, which correlates with the perceptual quality of AGIs, is not investigated to guide AGIQA. In this letter, we propose vision-language consistency guided multi-modal prompt learning for blind AGIQA, dubbed CLIP-AGIQA. Specifically, we introduce learnable textual and visual prompts in language and vision branches of CLIP models, respectively. Moreover, we design a text-to-image alignment quality prediction task, whose learned vision-language consistency knowledge is used to guide the optimization of the above multi-modal prompts. Experimental results on two public AGIQA datasets demonstrate that the proposed method outperforms state-of-the-art quality assessment models. The source code is available at https://***/JunFu1995/CLIP-AGIQA. © 2024, CC BY-NC-ND.

关键词： Learning systems

来源：评论

学校读者我要写书评

暂无评论

image Aligning and Stitching Based on Multilayer Mesh Deformation 16th

Image Aligning and Stitching Based on Multilayer Mesh Deform...

引用

16th International Forum on Digital TV and Wireless Multimedia Communication, IFTC 2019

作者： Xie, Mingfu Zhou, Jun Gu, Xiao Yang, Hua Shanghai Key Laboratory of Multi Media Processing and Transmissions Shanghai Jiaotong University Shanghai China Institute of Image Communication and Network Engineering Shanghai Jiaotong University Shanghai China

ISBN: (纸本)9789811533402

This paper aims to solve the problem of strong disparity in image stitching. By studying current mesh deformation methods based on single grid, we propose an image stitching framework based on multilayer mesh deformation for aligning regions in different layer. With development of image depth perception and semantic segmentation technology, we can get layering maps of images or photos expediently. We introduce images representation with layers and get layer corresponding by using depth or disparity information for large parallax scenarios. Registration of each layer is carried out independently. To ensure the integrity of layer synthesis results, we apply deformation with translation and scaling compensation between different layers before blending. The experiment demonstrates that our method can adequately utilize the prior information in layering maps to decouple 2D transformation between different layers, finally achieve outstanding aligning performance in all layers and naturalness in complete stitching result. © 2020, Springer Nature Singapore Pte Ltd.

关键词： Multilayers

来源：评论

学校读者我要写书评

暂无评论

Attentive Semantic Exploring for Manipulated Face Detection

Attentive Semantic Exploring for Manipulated Face Detection

引用

IEEE International Conference on Acoustics, Speech and Signal processing

作者： Zehao Chen Hua Yang Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Shanghai China Shanghai Key Lab of Digital Media Processing and Transmission Shanghai China

Face manipulation methods develop rapidly in recent years, whose potential risk to society accounts for the emerging of researches on detection methods. However, due to the diversity of manipulation methods and the high quality of fake images, detection methods suffer from a lack of generalization ability. To solve the problem, we find that segmenting images into semantic fragments could be effective, as discriminative defects and distortions are closely related to such fragments. Besides, to highlight discriminative regions in fragments and to measure contribution to the final prediction of each fragment is efficient for the improvement of generalization ability. Therefore, we propose a novel manipulated face detection method based on Multilevel Facial Semantic Segmentation and Cascade Attention Mechanism. To evaluate our method, we reconstruct two datasets: GGFI and FFMI, and also collect two open-source datasets. Experiments on four datasets verify the advantages of our approach against other state-of-the-arts, especially its generalization ability.

关键词： image segmentation Semantics Diversity methods Feature extraction Face detection Distortion measurement Speech processing

来源：评论

学校读者我要写书评

暂无评论

Large-Scale Video-Based Person Re-identification via Non-local Attention and Feature Erasing 16th

Large-Scale Video-Based Person Re-identification via Non-loc...

引用

16th International Forum on Digital TV and Wireless Multimedia Communication, IFTC 2019

作者： Yang, Zhao Chang, Zhigang Zheng, Shibao Institute of Image Communication and Network Engineering Shanghai Key Labs of Digital Media Processing and Transmission Shanghai Jiao Tong University SEIEE Buildings 5-421 Shanghai200240 China

ISBN: (纸本)9789811533402

Encoding the video tracks of person to an aggregative representation is the key for video-based person re-identification (re-ID), where average pooling or RNN methods are typically used to aggregating frame-level features. However, It is still difficult to deal with the spatial misalignment caused by occlusion, posture changes and camera views. Inspired by the success of non-local block in video analysis, we use a non-local attention block as a spatial-temporal attention mechanism to handle the spatial-temporal misalignment problem. Moreover, partial occlusion is widely occurred in video sequences. We propose a local feature branch to tackle the partial occlusion problem by using feature erasing in the frame-level feature map. Therefore, our network is composed by two-branch, the global branch via non-local attention encoding the global feature and the local feature branch grasping the local feature. In evaluation, the global feature and local feature are concatenated to obtain a more discriminative feature. We conduct extensive experiments on two challenging datasets (MARS and iLIDS-VID). The experimental results demonstrate that our method is comparable with the state-of-the-art methods in these datasets. © 2020, Springer Nature Singapore Pte Ltd.

关键词： Signal encoding

来源：评论

学校读者我要写书评

暂无评论

Adversarial semantic contour for object detection

arXiv

引用

arXiv 2021年

作者： Zhang, Yichi Zhu, Zijian Yang, Xiao Zhu, Jun Department of Computer Science and Technology Institute for AI THBI Lab Tsinghua University Beijing100084 China Institute of Image Processing and Network Engineering Shanghai Jiao Tong University Shanghai200240 China

Modern object detectors are vulnerable to adversarial examples, which brings potential risks to numerous applications, e.g., self-driving car. Among attacks regularized by lp norm, l0-attack aims to modify as few pixels as possible. Nevertheless, the problem is nontrivial since it generally requires to optimize the shape along with the texture simultaneously, which is an NP-hard problem. To address this issue, we propose a novel method of Adversarial Semantic Contour (ASC) guided by object contour as prior. With this prior, we reduce the searching space to accelerate the l0 optimization, and also introduce more semantic information which should affect the detectors more. Based on the contour, we optimize the selection of modified pixels via sampling and their colors with gradient descent alternately. Extensive experiments demonstrate that our proposed ASC outperforms the most commonly manually designed patterns (e.g., square patches and grids) on task of disappearing. By modifying no more than 5% and 3.5% of the object area respectively, our proposed ASC can successfully mislead the mainstream object detectors including the SSD512, Yolov4, Mask RCNN, Faster RCNN, etc. Copyright © 2021, The Authors. All rights reserved.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Understanding the Robustness of 3D Object Detection with Bird'View Representations in Autonomous Driving

Understanding the Robustness of 3D Object Detection with Bir...

引用

Conference on Computer Vision and Pattern Recognition (CVPR)

作者： Zijian Zhu Yichi Zhang Hai Chen Yinpeng Dong Shu Zhao Wenbo Ding Jiachen Zhong Shibao Zheng Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Dept. of Comp. Sci. and Tech. THBI Lab Institute for AI Tsinghua University BNRist Center Zhongguancun Laboratory Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education Information Materials and Intelligent Sensing Laboratory of Anhui Province School of Computer Science and Technology Anhui University SAIC Motor AI Lab

3D object detection is an essential perception task in autonomous driving to understand the environments. The Bird's-Eye-View (BEV) representations have significantly improved the performance of 3D detectors with camera inputs on popular benchmarks. However, there still lacks a systematic understanding of the robustness of these vision-dependent BEV models, which is closely related to the safety of autonomous driving systems. In this paper, we evaluate the natural and adversarial robustness of various representative models under extensive settings, to fully understand their behaviors influenced by explicit BEV features compared with those without BEV. In addition to the classic settings, we propose a 3D consistent patch attack by applying adversarial patches in the 3D space to guarantee the spatiotemporal consistency, which is more realistic for the scenario of autonomous driving. With substantial experiments, we draw several findings: 1) BEV models tend to be more stable than previous methods under different natural conditions and common corruptions due to the expressive spatial representations; 2) BEV models are more vulnerable to adversarial noises, mainly caused by the redundant BEV features; 3) Camera-LiDARfusion models have superior performance under different settings with multi-modal inputs, but BEV fusion model is still vulnerable to adversarial noises of both point cloud and image. These findings alert the safety issue in the applications of BEV detectors and could facilitate the development of more robust models.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Low-Cost and Portable Physiological Signal Monitor Using Physrate Model

SSRN

引用

SSRN 2023年

作者： Tan, Xudong Zhu, Yan Cheng, Zenghui Hu, Menghan Zhang, Xinxin Pei, Gan Yu, Chao Li, Qingli Li, Wenfang Wang, Jia Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University Shanghai200241 China Department of Radiology Ruijin Hospital Shanghai Jiao Tong University School of Medicine Shanghai200025 China Department of Emergency and Critical Care Shanghai Changzheng Hospital Shanghai200003 China Institute of Image Communication and Network Engineering Shanghai Jiao Tong University Shanghai200240 China

In this work, we propose a lightweight non-contact physiological signal monitor. The proposed system is equipped with the modified DeepPhys model and the rate estimation model PhysRate that enable non-contact physiological signal measurements without markers using the low-cost color camera. We introduce the constraint term guided by scene information into the original DeepPhys model to mitigate the overfitting problem of the original DeepPhys model. The results of validation experiments in ICU observation ward show that the modified DeepPhys model can effectively counteract the overfitting problem of the original DeepPhys model. The experimental results of PhysRate indicate that it exhibits certain performance improvements compared to other rate prediction models. In terms of system integration, the proposed physiological signal detection monitor includes Raspberry Pi 4B, RGB camera, 4G module, mobile power, and mobile terminal. With the help of above five modules, the system can analyze the captured signals at the edge and then transmit the processed data to the cloud via 4G network. The edge side is equipped with physiological signal detection model and the other data processing algorithms. On mobile devices, the users can view the relevant data through the developed Mini App. The PhysRate model code is available at https://***/ShawnTan86/PhysRate. © 2023, The Authors. All rights reserved.

关键词： Video signal processing

来源：评论

学校读者我要写书评

暂无评论

Understanding the Robustness of 3D Object Detection with Bird’s-Eye-View Representations in Autonomous Driving

arXiv

引用

arXiv 2023年

作者： Zhu, Zijian Zhang, Yichi Chen, Hai Dong, Yinpeng Zhao, Shu Ding, Wenbo Zhong, Jiachen Zheng, Shibao Institute of Image Communication and Network Engineering Shanghai Jiao Tong University China Dept. of Comp. Sci. and Tech. Institute for AI THBI Lab BNRist Center Tsinghua University China Key Laboratory of Intelligent Computing and Signal Processing Ministry of Education School of Computer Science and Technology Anhui University Information Materials and Intelligent Sensing Laboratory of Anhui Province China SAIC Motor AI Lab China Zhongguancun Laboratory China

3D object detection is an essential perception task in autonomous driving to understand the environments. The Bird’s-Eye-View (BEV) representations have significantly improved the performance of 3D detectors with camera inputs on popular benchmarks. However, there still lacks a systematic understanding of the robustness of these vision-dependent BEV models, which is closely related to the safety of autonomous driving systems. In this paper, we evaluate the natural and adversarial robustness of various representative models under extensive settings, to fully understand their behaviors influenced by explicit BEV features compared with those without BEV. In addition to the classic settings, we propose a 3D consistent patch attack by applying adversarial patches in the 3D space to guarantee the spatiotemporal consistency, which is more realistic for the scenario of autonomous driving. With substantial experiments, we draw several findings: 1) BEV models tend to be more stable than previous methods under different natural conditions and common corruptions due to the expressive spatial representations;2) BEV models are more vulnerable to adversarial noises, mainly caused by the redundant BEV features;3) Camera-LiDAR fusion models have superior performance under different settings with multi-modal inputs, but BEV fusion model is still vulnerable to adversarial noises of both point cloud and image. These findings alert the safety issue in the applications of BEV detectors and could facilitate the development of more robust models. Code available at https: //***/zzj403/BEV_Robust. Copyright © 2023, The Authors. All rights reserved.

关键词： Birds

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：