检索结果-内蒙古大学图书馆

Low-Resolution Action recognition for Tiny Actions Challenge

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Chen, Boyu Qiao, Yu Wang, Yali ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Shanghai AI Laboratory Shanghai China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society China

Tiny Actions Challenge focuses on understanding human activities in real-world surveillance. Basically, there are two main difficulties for activity recognition in this scenario. First, human activities are often recorded at a distance, and appear in a small resolution without much discriminative clue. Second, these activities are naturally distributed in a long-tailed way. It is hard to alleviate data bias for such heavy category imbalance. To tackle these problems, we propose a comprehensive recognition solution in this paper. First, we train video backbones with data balance, in order to alleviate overfitting in the challenge benchmark. Second, we design a dual-resolution distillation framework, which can effectively guide low-resolution action recognition by super-resolution knowledge. Finally, we apply model ensemble with post-processing, which can further boost performance on the long-tailed categories. Our solution ranks Top-1 on the leaderboard. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

UDC-UNet: Under-Display Camera Image Restoration via U-shape Dynamic Network

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Liu, Xina Hu, Jinfan Chen, Xiangyu Dong, Chao Shenzhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shanghai China University of Chinese Academy of Sciences Shanghai China University of Macau Shanghai China Shanghai AI Laboratory Shanghai China

Under-Display Camera (UDC) has been widely exploited to help smartphones realize full-screen displays. However, as the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. Particularly, flare and blur in UDC images could severely deteriorate the user experience in high dynamic range (HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with an estimated PSF in HDR scenes. Our network consists of three parts, including a U-shape base network to utilize multi-scale information, a condition branch to perform spatially variant modulation, and a kernel branch to leverage the prior knowledge of the PSF. According to the characteristics of HDR data, we additionally design a tone mapping loss to stabilize network optimization and achieve better visual quality. Experimental results show that the proposed UDC-UNet outperforms the state-of-the-art methods in quantitative and qualitative comparisons. Our approach won second place in the UDC image restoration track of the MIPI challenge. Codes and models are available at https://***/J-FHu/UDCUNet. © 2022, CC BY.

关键词： Image reconstruction

MorphMLP: An Efficient MLP-Like Backbone for Spatial-Temporal Representation Learning

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhang, David Junhao Li, Kunchang Wang, Yali Chen, Yunpeng Chandra, Shashwat Qiao, Yu Liu, Luoqi Shou, Mike Zheng National University of Singapore Singapore Meitu Inc China ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China University of Chinese Academy of Sciences China Shanghai AI Laboratory China

Recently, MLP-Like networks have been revived for image recognition. However, whether it is possible to build a generic MLP-Like architecture on video domain has not been explored, due to complex spatial-temporal modeling with large computation burden. To fill this gap, we present an efficient self-attention free backbone, namely MorphMLP, which flexibly leverages the concise Fully-Connected (FC) layer for video representation learning. Specifically, a MorphMLP block consists of two key layers in sequence, i.e., MorphFCs and MorphFCt, for spatial and temporal modeling respectively. MorphFCs can effectively capture core semantics in each frame, by progressive token interaction along both height and width dimensions. Alternatively, MorphFCt can adaptively learn long-term dependency over frames, by temporal token aggregation on each spatial location. With such multi-dimension and multi-scale factorization, our MorphMLP block can achieve a great accuracy-computation balance. Finally, we evaluate our MorphMLP on a number of popular video benchmarks. Compared with the recent state-of-the-art models, MorphMLP significantly reduces computation but with better accuracy, e.g., MorphMLP-S only uses 50% GFLOPs of VideoSwin-T but achieves 0.9% top-1 improvement on Kinetics400, under ImageNet1K pretraining. MorphMLP-B only uses 43% GFLOPs of MViT-B but achieves 2.4% top-1 improvement on SSV2, even though MorphMLP-B is pretrained on ImageNet1K while MViT-B is pretrained on Kinetics400. Moreover, our method adapted to the image domain outperforms previous SOTA MLP-Like architectures. Code is available at https://***/MTLab/MorphMLP. Copyright © 2021, The Authors. All rights reserved.

关键词： Image recognition

Masked Image Training for Generalizable Deep Image Denoising

学校读者我要写书评

暂无评论

Masked Image Training for Generalizable Deep Image Denoising

Conference on computer vision and pattern recognition (CVPR)

作者： Haoyu Chen Jinjin Gu Yihao Liu Salma Abdel Magid Chao Dong Qiong Wang Hanspeter Pfister Lei Zhu The Hong Kong University of Science and Technology (Guangzhou) Shanghai AI Lab The University of Sydney ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences University of Chinese Academy of Sciences Harvard University Guangdong Provincial Key Laboratory of Computer Vision and Virtual Reality Technology Shenzhen Institute of Advanced Technology Chinese Academy of Sciences The Hong Kong University of Science and Technology

When capturing and storing images, devices inevitably introduce noise. Reducing this noise is a critical task called image denoising. Deep learning has become the de facto method for image denoising, especially with the emergence of Transformer-based models that have achieved notable state-of-the-art results on various image tasks. However, deep learning-based methods often suffer from a lack of generalization ability. For example, deep models trained on Gaussian noise may perform poorly when tested on other noise distributions. To address this issue, we present a novel approach to enhance the generalization performance of denoising networks, known as masked training. Our method involves masking random pixels of the input image and reconstructing the missing information during training. We also mask out the features in the self-attention layers to avoid the impact of training-testing inconsistency. Our approach exhibits better generalization ability than other deep learning models and is directly applicable to real-world scenarios. Additionally, our interpretability analysis demonstrates the superiority of our method.

关键词：

Regional attention with architecture-rebuilt 3D network for RGB-D gesture recognition

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Zhou, Benjia Li, Yunan Wan, Jun Macau University of Science and Technology China National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing China School of Computer Science and Technology Xidian Univeristy China Xi'an Key Laboratory of Big Data and Intelligent Vision China School of Artificial Intelligence University of Chinese Academy of Sciences Beijing China

Human gesture recognition has drawn much attention in the area of computer vision. However, the performance of gesture recognition is always influenced by some gesture-irrelevant factors like the background and the clothes of performers. Therefore, focusing on the regions of hand/arm is important to the gesture recognition. Meanwhile, a more adaptive architecture-searched network structure can also perform better than the block-fixed ones like ResNet since it increases the diversity of features in different stages of the network better. In this paper, we propose a regional attention with architecture-rebuilt 3D network (RAAR3DNet) for gesture recognition. We replace the fixed Inception modules with the automatically rebuilt structure through the network via Neural Architecture Search (NAS), owing to the different shape and representation ability of features in the early, middle, and late stage of the network. It enables the network to capture different levels of feature representations at different layers more adaptively. Meanwhile, we also design a stackable regional attention module called Dynamic-Static Attention (DSA), which derives a Gaussian guidance heatmap and dynamic motion map to highlight the hand/arm regions and the motion information in the spatial and temporal domains, respectively. Extensive experiments on two recent large-scale RGB-D gesture datasets validate the effectiveness of the proposed method and show it outperforms state-of-the-art methods. The codes of our method are available at: https://***/zhoubenjia/RAAR3DNet. Copyright © 2021, The Authors. All rights reserved.

关键词： Large dataset

A new journey from SDRTV to HDRTV

学校读者我要写书评

暂无评论

arXiv 2021年

作者： Chen, Xiangyu Zhang, Zhengwen Ren, Jimmy S. Tian, Lynhoo Qiao, Yu Dong, Chao ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shanghai China SenseTime Research Shanghai China Qing Yuan Research Institute Shanghai Jiao Tong University Shanghai China Shanghai AI Laboratory Shanghai China

Nowadays modern displays are capable to render video content with high dynamic range (HDR) and wide color gamut (WCG). However, most available resources are still in standard dynamic range (SDR). Therefore, there is an urgent demand to transform existing SDR-TV contents into their HDR-TV versions. In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step solution pipeline including adaptive global color mapping, local enhancement and highlight generation. Moreover, the above analysis inspires us to present a lightweight network that utilizes global statistics as guidance to conduct image-adaptive color mapping. In addition, we construct a dataset using HDR videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results achieve state-of-the-art performance in quantitative comparisons and visual quality. The code and dataset are available at https://***/ chxy95/HDRTVNet. Copyright © 2021, The Authors. All rights reserved.

关键词： Color

Cross Domain Object Detection by Target-Perceived Dual Branch Distillation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： He, Mengzhe Wang, Yali Wu, Jiaxi Wang, Yiru Li, Hanqing Li, Bo Gan, Weihao Wu, Wei Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences China SenseTime Research University of Chinese Academy of Science China Shanghai AI Laboratory Shanghai China Beihang University China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society China

Cross domain object detection is a realistic and challenging task in the wild. It suffers from performance degradation due to large shift of data distributions and lack of instance-level annotations in the target domain. Existing approaches mainly focus on either of these two difficulties, even though they are closely coupled in cross domain object detection. To solve this problem, we propose a novel Target-perceived Dual-branch Distillation (TDD) framework. By integrating detection branches of both source and target domains in a unified teacher-student learning scheme, it can reduce domain shift and generate reliable supervision effectively. In particular, we first introduce a distinct Target Proposal Perceiver between two domains. It can adaptively enhance source detector to perceive objects in a target image, by leveraging target proposal contexts from iterative cross-attention. Afterwards, we design a concise Dual Branch Self Distillation strategy for model training, which can progressively integrate complementary object knowledge from different domains via self-distillation in two branches. Finally, we conduct extensive experiments on a number of widely-used scenarios in cross domain object detection. The results show that our TDD significantly outperforms the state-of-the-art methods on all the benchmarks. Our code and model will be available at here. Copyright © 2022, The Authors. All rights reserved.

关键词： Distillation

A comprehensive survey of oracle character recognition: challenges, benchmarks, and beyond

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Jing Chi, Xueke Wang, Qiufeng Huang, Kaizhu Wang, Da-Han Liu, Yongge Liu, Cheng-Lin College of Science & Technology Ningbo University Ningbo China School of Advanced Technology Xi’an Jiaotong-Liverpool University Suzhou China Duke Kunshan University Suzhou China Fujian Key Laboratory of Pattern Recognition and Image Understanding School of Computer and Information Engineering Xiamen University of Technology Xiamen China Anyang Normal University Anyang China State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation Chinese Academy of Sciences Beijing China

Oracle character recognition—an analysis of ancient Chinese inscriptions found on oracle bones—has become a pivotal field intersecting archaeology, paleography, and historical cultural studies. Traditional methods of oracle character recognition have relied heavily on manual interpretation by experts, which is not only labor-intensive but also limits broader accessibility to the general public. With recent breakthroughs in pattern recognition and deep learning, there is a growing movement towards the automation of oracle character recognition (OrCR), showing considerable promise in tackling the challenges inherent to these ancient scripts. However, a comprehensive understanding of OrCR still remains elusive. Therefore, this paper presents a systematic and structured survey of the current landscape of OrCR research. We commence by identifying and analyzing the key challenges of OrCR. Then, we provide an overview of the primary benchmark datasets and digital resources available for OrCR. A review of contemporary research methodologies follows, in which their respective efficacies, limitations, and applicability to the complex nature of oracle characters are critically highlighted and examined. Additionally, our review extends to ancillary tasks associated with OrCR across diverse disciplines, providing a broad-spectrum analysis of its applications. We conclude with a forward-looking perspective, proposing potential avenues for future investigations that could yield significant advancements in the field. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep learning

Dual-AI: Dual-path Actor Interaction Learning for Group Activity recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Han, Mingfei Zhang, David Junhao Wang, Yali Yan, Rui Yao, Lina Chang, Xiaojun Qiao, Yu ReLER AAII UTS United States National University of Singapore Singapore ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences China RMIT University Australia University of New South Wales Australia Shanghai AI Laboratory Shanghai China

Learning spatial-temporal relation among multiple actors is crucial for group activity recognition. Different group activities often show the diversified interactions between actors in the video. Hence, it is often difficult to model complex group activities from a single view of spatial-temporal actor evolution. To tackle this problem, we propose a distinct Dual-path Actor Interaction (Dual-AI) framework, which flexibly arranges spatial and temporal transformers in two complementary orders, enhancing actor relations by integrating merits from different spatiotemporal paths. Moreover, we introduce a novel Multi-scale Actor Contrastive Loss (MAC-Loss) between two interactive paths of Dual-AI. Via self-supervised actor consistency in both frame and video levels, MAC-Loss can effectively distinguish individual actor representations to reduce action confusion among different actors. Consequently, our Dual-AI can boost group activity recognition by fusing such discriminative features of different actors. To evaluate the proposed approach, we conduct extensive experiments on the widely used benchmarks, including Volleyball [24], Collective Activity [12], and NBA datasets [56]. The proposed Dual-AI achieves state-of-the-art performance on all these datasets. It is worth noting the proposed Dual-AI with 50% training data outperforms a number of recent approaches with 100% training data. This confirms the generalization power of Dual-AI for group activity recognition, even under the challenging scenarios of limited supervision. Copyright © 2022, The Authors. All rights reserved.

关键词： Machine learning