检索结果-内蒙古大学图书馆

Image quality assessment for perceptual image restoration: A new dataset, benchmark and metric

学校读者我要写书评

暂无评论

arXiv 2020年

作者： Gu, Jinjin Cai, Haoming Chen, Haoyu Ye, Xiaoxing Ren, Jimmy S. Dong, Chao School of Electrical and Information Engineering University of Sydney Australia Chinese University of Hong Kong Shenzhen Hong Kong ShenZhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences SenseTime Research Qing Yuan Research Institute Shanghai Jiao Tong University Shanghai China SIAT Branch Shenzhen Institute of Artificial Intelligence and Robotics for Society

Image quality assessment (IQA) is the key factor for the fast development of image restoration (IR) algorithms. The most recent perceptual IR algorithms based on generative adversarial networks (GANs) have brought in significant improvement on visual performance, but also pose great challenges for quantitative evaluation. Notably, we observe an increasing inconsistency between perceptual quality and the evaluation results. We present two questions: (1) Can existing IQA methods objectively evaluate recent IR algorithms? (2) With the focus on beating current benchmarks, are we getting better IR algorithms? To answer the questions and promote the development of IQA methods, we contribute a large-scale IQA dataset, called Perceptual Image Processing ALgorithms (PIPAL) dataset. Especially, this dataset includes the results of GAN-based IR algorithms, which are missing in previous datasets. We collect more than 1.13 million human judgements to assign subjective scores for PIPAL images using the more reliable "Elo system". Based on PIPAL, we present new benchmarks for both IQA and super-resolution methods. Our results indicate that existing IQA methods cannot fairly evaluate GAN-based IR algorithms. While using appropriate evaluation methods is important, IQA methods should also be updated along with the development of IR algorithms. At last, we shed light on how to improve the IQA performance on GAN-based distortion. Inspired by the find that the existing IQA methods have an unsatisfactory performance on the GAN-based distortion partially because of their low tolerance to spatial misalignment, we propose to improve the performance of an IQA network on GAN-based distortion by explicitly considering this misalignment. We propose the Space Warping Difference Network, which includes the novel l2 pooling layers and Space Warping Difference layers. Experiments demonstrate the effectiveness of the proposed method. © 2020, CC BY.

关键词： Image reconstruction

UniFormer: Unifying Convolution and Self-attention for Visual recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Kunchang Wang, Yali Zhang, Junhao Gao, Peng Song, Guanglu Liu, Yu Li, Hongsheng Qiao, Yu ShenZhen Key Lab of Computer Vision and Pattern Recognition Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Shenzhen518055 China University of Chinese Academy of Sciences Beijing100049 China Shanghai Artificial Intelligence Laboratory Shanghai200232 China National University of Singapore Singapore Shanghai Artificial Intelligence Laboratory China SenseTime Research China The Chinese University of Hong Kong Hong Kong

It is a challenging task to learn discriminative representation from images and videos, due to large local redundancy and complex global dependency in these visual data. Convolution neural networks (CNNs) and vision transformers (ViTs) have been two dominant frameworks in the past few years. Though CNNs can efficiently decrease local redundancy by convolution within a small neighborhood, the limited receptive field makes it hard to capture global dependency. Alternatively, ViTs can effectively capture long-range dependency via self-attention, while blind similarity comparisons among all the tokens lead to high redundancy. To resolve these problems, we propose a novel Unified transFormer (UniFormer), which can seamlessly integrate the merits of convolution and self-attention in a concise transformer format. Different from the typical transformer blocks, the relation aggregators in our UniFormer block are equipped with local and global token affinity respectively in shallow and deep layers, allowing tackling both redundancy and dependency for efficient and effective representation learning. Finally, we flexibly stack our blocks into a new powerful backbone, and adopt it for various vision tasks from image to video domain, from classification to dense prediction. Without any extra training data, our UniFormer achieves 86.3 top-1 accuracy on ImageNet-1K classification task. With only ImageNet-1K pre-training, it can simply achieve state-of-the-art performance in a broad range of downstream tasks. It obtains 82.9/84.8 top-1 accuracy on Kinetics-400/600, 60.9/71.2 top-1 accuracy on Something-Something V1/V2 video classification tasks, 53.8 box AP and 46.4 mask AP on COCO object detection task, 50.8 mIoU on ADE20K semantic segmentation task, and 77.4 AP on COCO pose estimation task. Moreover, we build an efficient UniFormer with a concise hourglass design of token shrinking and recovering, which achieves 2-4× higher throughput than the recent lightweight models. Code is av

关键词： Convolution

AIM 2020 Challenge on Image Extreme Inpainting 16th

学校读者我要写书评

暂无评论

AIM 2020 Challenge on Image Extreme Inpainting

Workshops held at the 16th European Conference on computer vision, ECCV 2020

作者： Ntavelis, Evangelos Romero, Andrés Bigdeli, Siavash Timofte, Radu Hui, Zheng Wang, Xiumei Gao, Xinbo Shin, Chajin Kim, Taeoh Son, Hanbin Lee, Sangyoun Li, Chao Li, Fu He, Dongliang Wen, Shilei Ding, Errui Bai, Mengmeng Li, Shuchen Zeng, Yu Lin, Zhe Yang, Jimei Zhang, Jianming Shechtman, Eli Lu, Huchuan Zeng, Weijian Ni, Haopeng Cai, Yiyang Li, Chenghua Xu, Dejia Wu, Haoning Han, Yu Nadim, Uddin S. M. Jang, Hae Woong Ahmed, Soikat Hasan Yoon, Jungmin Jung, Yong Ju Li, Chu-Tak Liu, Zhi-Song Wang, Li-Wen Siu, Wan-Chi Lun, Daniel P. K. Suin, Maitreya Purohit, Kuldeep Rajagopalan, A.N. Narang, Pratik Mandal, Murari Chauhan, Pranjal Singh Computer Vision Lab ETH Zürich Zürich Switzerland CSEM Neuchâtel Switzerland School of Electronic Engineering Xidian University Xi’an China Image and Video Pattern Recognition Laboratory School of Electrical and Electronic Engineering Yonsei University Seoul Korea Republic of Baidu Inc. Beijing China Beijing China Dalian University of Technology Dalian China Adobe San Jose United States Rensselaer Polytechnic Institute Troy United States Peking University Beijing China Lab Gachon University Seongnam Korea Republic of Centre for Multimedia Signal Processing Department of Electronic and Information Engineering The Hong Kong Polytechnic University Hong Kong China Indian Institute of Technology Madras Chennai India BITS Pilani Pilani India MNIT Jaipur Jaipur India

ISBN: (纸本)9783030670696

This paper reviews the AIM 2020 challenge on extreme image inpainting. This report focuses on proposed solutions and results for two different tracks on extreme image inpainting: classical image inpainting and semantically guided image inpainting. The goal of track 1 is to inpaint large part of the image with no supervision. Similarly, the goal of track 2 is to inpaint the image by having access to the entire semantic segmentation map of the input. The challenge had 88 and 74 participants, respectively. 11 and 6 teams competed in the final phase of the challenge, respectively. This report gauges current solutions and set a benchmark for future extreme image inpainting methods. © 2020, Springer Nature Switzerland AG.

关键词： Semantic Segmentation

Occlusion Boundary Prediction and Transformer Based Depth-Map Refinement From Single Image

学校读者我要写书评

暂无评论

ACM Transactions on Multimedia Computing, Communications, and Applications 1000年

作者： Praful Hambarde Gourav Wadhwa Santosh Kumar Vipparthi Subrahmanyam Murala Abhinav Dhall Computer Vision and Pattern Recognition Lab Indian Institute of Technology Ropar India ByteDance Singapore School of Computer Science and Statistics Trinity College Dublin Ireland Flinders University Adelaide Australia

Due to the numerous applications of boundary maps and occlusion orientation maps (ORI-maps) in high-level vision problems, accurate estimation of these maps is a crucial task. The existing deep networks employ a single-stream network to estimate the relation between boundary map and ORI-map estimation. However, these networks fail to explore significant individual information separately. To resolve this problem, in this paper, we propose a novel two-stream generative adversarial network (GAN) for boundary map and ORI-map estimation, named OBP-GAN. The proposed OBP-GAN consists of two streams known as BP-GAN and OR-GAN. The BP-GAN estimates the boundary map, and the OR-GAN predicts the ORI-map. The boundary and ORI-map can also be useful cues for the task of depth-map refinement from single images. Therefore, in this work, we propose a transformer-based depth-map refinement network (TRANSDMR-GAN) for refining the depth estimated from monocular images using boundary and ORI-map. We conducted extensive analyses on indoor and outdoor datasets to validate our proposed OBP-GAN and TRANSDMR-GAN. The extensive experimental analysis and ablation study demonstrate the ability of the proposed OBP-GAN to generate state-of-the-art occlusion boundary maps. Furthermore, we show that the proposed network, TRANSDMR-GAN, can generate an edge-enhanced depth map without degrading the accuracy of the initial depth map.

关键词： Depth-map Refinement Occlusion Boundary Map Occlusion Orientation Map Self-attention and Generative Adversarial Networks

Application of a genetic algorithm in triangulation of a 3-D object surface

学校读者我要写书评

暂无评论

International Journal of computers and Applications 2000年第2期22卷 73-77页

作者： Zhenyu, Chen Mbede, J.B. Yan, Zhou Dehua, Li Hanping, Hu State Commn. Res. Open Lab. Image P. Inst. Pattern Recog. Artif. Intell. Huazhong Univ. of Sci. and Technol. 430074 Wuhan China Intelligent Contr. and Robotics Lab. Dept. of Contr. Sci. and Engineering Huazhong Univ. of Sci. and Technol. 430074 Wuhan China Second Artillery Institute Huazhong Univ. of Sci. and Technol. Inst. Pattern Recog. Artif. Intell. Huazhong Univ. of Sci. and Technol. Second Artillery Institute China Wuhan University China Artificial Intelligence Department University of Edinburgh Inst. of AI and Pattern Recognition Huazhong Univ. of Sci. and Technol. China Cogn. Sci. Natl. Key Found. Res. P. Douala University Cameroon Tübingen University Germany Automatic Control Laboratory Darmstadt University Department of Service Cameroon Min. of National Education Robotics Laboratory Huazhong Univ. of Sci. and Technol. Wuhan China Assoc. Connectionnistes These-ACTH Rennes France IEEE Soc. of Robotics and Automation IEEE Society of Control Systems IASTED Assoc. Jeunes Chercheurs Robotique France Navy University of Engineering China Huazhong Univ. of Sci. and Technol. China Department of Computer Science Airforce Radar Acad. of Engineering China Inst. of P.R and AI Huazhong Univ. of Sci. and Technol.

This paper proposes a genetic-based algorithm for surface reconstruction of three-dimension (3-D) objects from a group of contours representing its section plane lines. The algorithm can optimize the triangulation of the surface of 3-D objects with a multi-objective optimization function to meet the needs of a wide range of applications. Further, a new crossover operator for triangulation and a new 3-D quadrilateral mutation operator are also introduced.

关键词： Object recognition

NTIRE 2023 Image Shadow Removal Challenge Report

学校读者我要写书评

暂无评论

NTIRE 2023 Image Shadow Removal Challenge Report

2023 IEEE/CVF Conference on computer vision and pattern recognition Workshops, CVPRW 2023

作者： Vasluianu, Florin-Alexandru Seizinger, Tim Timofte, Radu Cui, Shuhao Huang, Junshi Tian, Shuman Fan, Mingyuan Zhang, Jiaqi Zhu, Li Wei, Xiaoming Wei, Xiaolin Luo, Ziwei Gustafsson, Fredrik K. Zhao, Zheng Sjölund, Jens Schön, Thomas B. Dong, Xiaoyi Zhang, Xi Sheryl Li, Chenghua Leng, Cong Yeo, Woon-Ha Oh, Wang-Taek Lee, Yeo-Reum Ryu, Han-Cheol Luo, Jinting Jiang, Chengzhi Han, Mingyan Wu, Qi Lin, Wenjie Yu, Lei Li, Xinpeng Jiang, Ting Fan, Haoqiang Liu, Shuaicheng Xu, Shuning Song, Binbin Chen, Xiangyu Zhang, Shile Zhou, Jiantao Zhang, Zhao Zhao, Suiyi Zheng, Huan Gao, Yangcheng Wei, Yanyan Wang, Bo Ren, Jiahuan Luo, Yan Kondo, Yuki Miyata, Riku Yasue, Fuma Naruki, Taito Ukita, Norimichi Chang, Hua-En Yang, Hao-Hsiang Chen, Yi-Chung Chiang, Yuan-Chun Huang, Zhi-Kai Chen, Wei-Ting Chen, I-Hsiang Hsieh, Chia-Hsuan Kuo, Sy-Yen Xianwei, Li Fu, Huiyuan Liu, Chunlin Ma, Huadong Fu, Binglan He, Huiming Wang, Mengjia She, Wenxuan Liu, Yu Nathan, Sabari Kansal, Priya Zhang, Zhongjian Yang, Huabin Wang, Yan Zhang, Yanru Phutke, Shruti S. Kulkarni, Ashutosh Khan, Md Raqib Murala, Subrahmanyam Vipparthi, Santosh Kumar Ye, Heng Liu, Zixi Yang, Xingyi Liu, Songhua Wu, Yinwei Jing, Yongcheng Yu, Qianhao Zheng, Naishan Huang, Jie Long, Yuhang Yao, Mingde Zhao, Feng Zhao, Bowen Ye, Nan Shen, Ning Cao, Yanpeng Xiong, Tong Xia, Weiran Li, Dingwen Xia, Shuchen Computer Vision Lab Ifi Caidas University of Würzburg Germany Computer Vision Lab Eth Zürich Switzerland Meituan Group China Department of Information Technology Uppsala University Sweden Institute of Automation Chinese Academy of Sciences Beijing China Nanjing China Maicro Nanjing China Department of Artificial Intelligence Convergence Sahmyook University Seoul Korea Republic of Megvii Technology China University of Electronic Science and Technology of China China University of Macau China China Toyota Technological Institute Japan Graduate Institute of Electronics Engineering National Taiwan University Taiwan Department of Electrical Engineering National Taiwan University Taiwan Graduate Institute of Communication Engineering National Taiwan University Taiwan ServiceNow United States Beijing University of Post and Teleconmunication Beijing China Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education China Couger Inc. Computer Vision and Pattern Recognition Lab Indian Institute of Technology Ropar Punjab Rupnagar India Research Institute Singapore National University of Singapore Singapore Research Institute Singapore University of Sydney Australia Brain-Inspired Vision Laboratory Information Science and Technology Institution University of Science and Technology of China China State Key Laboratory of Fluid Power and Mechatronic Systems School of Mechanical Engineering Zhejiang University Hangzhou310027 China Key Laboratory of Advanced Manufacturing Technology of Zhejiang Province School of Mechanical Engineering Zhejiang University Hangzhou310027 China South China University of Technology China

ISBN: (纸本)9798350302493

This work reviews the results of the NTIRE 2023 Challenge on Image Shadow Removal. The described set of solutions were proposed for a novel dataset, which captures a wide range of object-light interactions. It consists of 1200 roughly pixel aligned pairs of real shadow free and shadow affected images, captured in a controlled environment. The data was captured in a white-box setup, using professional equipment for lights and data acquisition sensors. The challenge had a number of 144 participants registered, out of which 19 teams were compared in the final ranking. The proposed solutions extend the work on shadow removal, improving over the performance level describing state-of-the-art methods. © 2023 IEEE.

关键词： Data acquisition

Survey on Deep Face Restoration: From Non-blind to Blind and Beyond

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Wenjie Wang, Mei Zhang, Kai Li, Juncheng Li, Xiaoming Zhang, Yuhang Gao, Guangwei Deng, Weihong Lin, Chia-Wen The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing China The Computer Vision Lab ETH Zürich Zürich Switzerland The School of Communication and Information Engineering Shanghai University Shanghai China The Nanyang Technological University Singapore The Intelligent Visual Information Perception Laboratory Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Department of Electrical Engineering National Tsing Hua University Hsinchu Taiwan

Face restoration (FR) is a specialized field within image restoration that aims to recover low-quality (LQ) face images into high-quality (HQ) face images. Recent advances in deep learning technology have led to significant progress in FR methods. In this paper, we begin by examining the prevalent factors responsible for real-world LQ images and introduce degradation techniques used to synthesize LQ images. We also discuss notable benchmarks commonly utilized in the field. Next, we categorize FR methods based on different tasks and explain their evolution over time. Furthermore, we explore the various facial priors commonly utilized in the restoration process and discuss strategies to enhance their effectiveness. In the experimental section, we thoroughly evaluate the performance of state-of-the-art FR methods across various tasks using a unified benchmark. We analyze their performance from different perspectives. Finally, we discuss the challenges faced in the field of FR and propose potential directions for future advancements. The open-source repository corresponding to this work can be found at https://***/24wenjie-li/Awesome-Face-Restoration. Copyright © 2023, The Authors. All rights reserved.

关键词： Restoration

Group shift pointwise convolution for volumetric medical image segmentation

学校读者我要写书评

暂无评论

arXiv 2021年

作者： He, Junjun Ye, Jin Li, Cheng Song, Diping Chen, Wanli Wang, Shanshan Gu, Lixu Qiao, Yu School of Biomedical Engineering Shanghai Jiao Tong University Shanghai China Institute of Medical Robotics Shanghai Jiao Tong University Shanghai China Shenzhen Key Lab of Computer Vision and Pattern Recognition SIAT-SenseTime Joint Lab Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong Shenzhen China Shanghai AI Lab Shanghai China Paul C. Lauterbur Research Center for Biomedical Imaging Shenzhen Institute of Advanced Technology Chinese Academy of Sciences Guangdong Shenzhen China The Chinese University of Hong Kong Hong Kong Peng Cheng Laboratory Guangdong Shenzhen China Pazhou Lab Guangdong Guangzhou China

Recent studies have witnessed the effectiveness of 3D convolutions on segmenting volumetric medical images. Compared with the 2D counterparts, 3D convolutions can capture the spatial context in three dimensions. Nevertheless, models employing 3D convolutions introduce more trainable parameters and are more computationally complex, which may lead easily to model overfitting especially for medical applications with limited availab.e training data. This paper aims to improve the effectiveness and efficiency of 3D convolutions by introducing a novel Group Shift Pointwise Convolution (GSP-Conv). GSP-Conv simplifies 3D convolutions into pointwise ones with 1 × 1 × 1 kernels, which dramatically reduces the number of model parameters and FLOPs (e.g. 27× fewer than 3D convolutions with 3 × 3 × 3 kernels). Naïve pointwise convolutions with limited receptive fields cannot make full use of the spatial image context. To address this problem, we propose a parameter-free operation, Group Shift (GS), which shifts the feature maps along different spatial directions in an elegant way. With GS, pointwise convolutions can access features from different spatial locations, and the limited receptive fields of pointwise convolutions can be compensated. We evaluate the proposed methods on two datasets, PROMISE12 and BraTS18. Results show that our method, with substantially decreased model complexity, achieves comparable or even better performance than models employing 3D convolutions. Copyright © 2021, The Authors. All rights reserved.

关键词： Convolution

ChaLearn looking at people: IsoGD and ConGD large-scale RGB-D gesture recognition

学校读者我要写书评

暂无评论

arXiv 2019年

作者： Wan, Jun Lin, Chi Wen, Longyin Li, Yunan Miao, Qiguang Escalera, Sergio Anbarjafari, Gholamreza Guyon, Isabelle Guo, Guodong Li, Stan Z. National Laboratory of Pattern Recognition Institute of Automation Chinese Academy of Sciences Beijing100190 China JD Finance Mountain ViewCA United States University of Southern California Los AngelesCA90089-0911 United States School of Computer Science and Technology Xidian University & Xi'an Key Laboratory of Big Data and Intelligent Vision 2nd South Taibai Road Xi'an710071 China Universitat de Barcelona Computer Vision Center Spain iCV Lab Institute of Technology University of Tartu Estonia Faculty of Engineering Hasan Kalyoncu University Gaziantep Turkey Institute of Digital Technologies Loughborough University London United Kingdom ChaLearn United States University Paris-Saclay France institute of Deep Learning Baidu Research National Engineering Laboratory for Deep Learning Technology and Application China

The ChaLearn large-scale gesture recognition challenge has been run twice in two workshops in conjunction with the International Conference on pattern recognition (ICPR) 2016 and International Conference on computer vision (ICCV) 2017, attracting more than 200 teams around the world. This challenge has two tracks, focusing on isolated and continuous gesture recognition, respectively. This paper describes the creation of both benchmark datasets and analyzes the advances in large-scale gesture recognition based on these two datasets. We discuss the challenges of collecting large-scale ground-truth annotations of gesture recognition, and provide a detailed analysis of the current state-of-the-art methods for large-scale isolated and continuous gesture recognition based on RGB-D video sequences. In addition to recognition rate and mean jaccard index (MJI) as evaluation metrics used in our previous challenges, we also introduce the corrected segmentation rate (CSR) metric to evaluate the performance of temporal segmentation for continuous gesture recognition. Furthermore, we propose a bidirectional long short-term memory (Bi-LSTM) baseline method, determining the video division points based on the skeleton points extracted by convolutional pose machine (CPM). Experiments demonstrate that the proposed Bi-LSTM outperforms the state-of-the-art methods with an absolute improvement of 8.1% (from 0.8917 to 0.9639) of CSR. Copyright © 2019, The Authors. All rights reserved.

关键词： Long short-term memory