检索结果-内蒙古大学图书馆

Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xu, Guoan Li, Juncheng Gao, Guangwei Lu, Huimin Yang, Jian Yue, Dong The Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing210023 China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou215006 China The School of Communication and Information Engineering Shanghai University Shanghai200444 China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing210049 China The Kyushu Institute of Technology Kitakyushu804-8550 Japan The School of Computer Science and Technology Nanjing University of Science and Technology Nanjing210049 China

In the past decade, convolutional neural networks (CNNs) have shown prominence for semantic segmentation. Although CNN models have very impressive performance, the ability to capture global representation is still insufficient, which results in suboptimal results. Recently, Transformer achieved huge success in NLP tasks, demonstrating its advantages in modeling long-range dependency. Recently, Transformer has also attracted tremendous attention from computer vision researchers who reformulate the image processing tasks as a sequence-to-sequence prediction but resulted in deteriorating local feature details. In this work, we propose a lightweight real-time semantic segmentation network called LETNet. LETNet combines a U-shaped CNN with Transformer effectively in a capsule embedding style to compensate for respective deficiencies. Meanwhile, the elaborately designed Lightweight Dilated Bottleneck (LDB) module and Feature Enhancement (FE) module cultivate a positive impact on training from scratch simultaneously. Extensive experiments performed on challenging datasets demonstrate that LETNet achieves superior performances in accuracy and efficiency balance. Specifically, It only contains 0.95M parameters and 13.6G FLOPs but yields 72.8% mIoU at 120 FPS on the Cityscapes test set and 70.5% mIoU at 250 FPS on the CamVid test dataset using a single RTX 3090 GPU. Source code will be available at https://***/IVIPlab/LETNet. Copyright © 2023, The Authors. All rights reserved.

关键词： Semantic Segmentation

EWT: Efficient Wavelet-Transformer for Single image Denoising

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Juncheng Cheng, Bodong Chen, Ying Gao, Guangwei Zeng, Tieyong The School of Communication & Information Engineering Shanghai University Shanghai China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing China The School of Computer Science and Technology Xidian University Xian China The Department of Cyberspace Security Beijing Electronic Science & Technology Institute Beijing China The Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou China The Department of Mathematics The Chinese University of Hong Kong New Territories Hong Kong

Transformer-based image denoising methods have achieved encouraging results in the past year. However, it must uses linear operations to model long-range dependencies, which greatly increases model inference time and consumes GPU storage space. Compared with convolutional neural network-based methods, current Transformer-based image denoising methods cannot achieve a balance between performance improvement and resource consumption. In this paper, we propose an Efficient Wavelet Transformer (EWT) for image denoising. Specifically, we use Discrete Wavelet Transform (DWT) and Inverse Wavelet Transform (IWT) for downsampling and upsampling, respectively. This method can fully preserve the image features while reducing the image resolution, thereby greatly reducing the device resource consumption of the Transformer model. Furthermore, we propose a novel Dual-stream Feature Extraction Block (DFEB) to extract image features at different levels, which can further reduce model inference time and GPU memory usage. Experiments show that our method speeds up the original Transformer by more than 80%, reduces GPU memory usage by more than 60%, and achieves excellent denoising results. All code will be public. © 2023, CC BY.

关键词： image denoising

CTCNet: A CNN-Transformer Cooperation Network for Face image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Gao, Guangwei Xu, Zixiang Li, Juncheng Yang, Jian Zeng, Tieyong Qi, Guo-Jun The Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou China The School of Communication & Information Engineering Shanghai University Shanghai China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing China The School of Computer Science and Technology Nanjing University of Science and Technology Nanjing China The Center for Mathematical Artificial Intelligence Department of Mathematics The Chinese University of Hong Kong Hong Kong Research Center for Industries of the Future School of Engineering Westlake University OPPO Research Seattle United States

Recently, deep convolution neural networks (CNNs) steered face super-resolution methods have achieved great progress in restoring degraded facial details by joint training with facial priors. However, these methods have some obvious limitations. On the one hand, multi-task joint learning requires additional marking on the dataset, and the introduced prior network will significantly increase the computational cost of the model. On the other hand, the limited receptive field of CNN will reduce the fidelity and naturalness of the reconstructed facial images, resulting in suboptimal reconstructed images. In this work, we propose an efficient CNN-Transformer Cooperation Network (CTCNet) for face super-resolution tasks, which uses the multi-scale connected encoder-decoder architecture as the backbone. Specifically, we first devise a novel Local-Global Feature Cooperation Module (LGCM), which is composed of a Facial Structure Attention Unit (FSAU) and a Transformer block, to promote the consistency of local facial detail and global facial structure restoration simultaneously. Then, we design an efficient Feature Refinement Module (FRM) to enhance the encoded features. Finally, to further improve the restoration of fine facial details, we present a Multi-scale Feature Fusion Unit (MFFU) to adaptively fuse the features from different stages in the encoder procedure. Extensive evaluations on various datasets have assessed that the proposed CTCNet can outperform other state-of-the-art methods significantly. Source code will be available at https://***/IVIPlab/CTCNet. Copyright © 2022, The Authors. All rights reserved.

关键词： Restoration

Tigc-Net: Transformer-Improved Graph Convolution Network for Spatio-Temporal Prediction

学校读者我要写书评

暂无评论

SSRN

SSRN 2022年

作者： Chen, Kai Yang, Chunfeng Zhou, Zhengyuan Liu, Yao Ji, Tianjiao Sun, Weiya Chen, Yang School of Cyber Science and Engineering Southeast University Nanjing210096 China Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education Nanjing210096 China The College of Software Engineering Southeast University Nanjing210096 China Laboratory of Image Science and Technology The School of Computer Science and Engineering Southeast University Nanjing210096 China Jiangsu Key Laboratory of Molecular and Functional Imaging Department of Radiology Zhongda Hospital Southeast University Nanjing210009 China Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing School of Computer Science and Engineering Southeast University Nanjing210096 China NHC Key Laboratory of Medical Virology and Viral Diseases National Institute for Viral Disease Control and Prevention Chinese Center for Disease Control and Prevention Beijing China Beijing Institute of Tracking and Communication Technology Beijing100094 China

Modeling spatio-temporal sequences is an important topic yet challenging for existing neural networks. Most of the current spatio-temporal sequence prediction methods usually capture features separately in temporal and spatial dimensions or employ multiple mutually independent local spatio-temporal graphs to represent a spatio-temporal sequence. The first kind of method mentioned above is difficult to mine the complex spatio-temporal correlations, while the other is limited for the accuracy of long-term predictions. To handle these issues, this paper proposes a Transformer-Improved Graph Convolution Network for spatio-temporal prediction. Specifically, the temporal location encoding method is exploited to derive the spatio-temporal characteristics of the sequence utilizing a spatio-temporal feature fusion network. In addition, a spatio-temporal attention network is developed to enhance the spatio-temporal correlation of the sequence, and the dynamic spatial features of sequence are further extracted through the adaptive graph convolution network. A private dataset and a public dataset are employed to demonstrate the performance of the proposed TIGC-Net. The qualitative and quantitative results show that the proposed TIGC-Net can extract dynamic spatiotemporal properties more effectively, enhance the spatio-temporal correlation of sequences and improve the prediction accuracy compared with four stateof-the-art. © 2022, The Authors. All rights reserved.

关键词： Convolution

Cross-receptive Focused Inference Network for Lightweight image Super-Resolution

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Wenjie Li, Juncheng Gao, Guangwei Deng, Weihong Zhou, Jiantao Yang, Jian Qi, Guo-Jun The Intelligent Visual Information Perception Laboratory Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing210046 China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou215006 China The School of Communication and Information Engineering Shanghai University Shanghai200444 China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing210094 China The Pattern Recognition and Intelligent System Laboratory School of Artificial Intelligence Beijing University of Posts and Telecommunications Beijing100876 China The State Key Laboratory of Internet of Things for Smart City Department of Computer and Information Science Faculty of Science and Technology University of Macau 999078 China The School of Computer Science and Technology Nanjing University of Science and Technology Nanjing210094 China The Research Center for Industries of the Future The School of Engineering Westlake University Hangzhou310024 China OPPO Research SeattleWA98101 United States

Recently, Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks due to the ability of global feature extraction. However, the capabilities of Transformers that need to incorporate contextual information to extract features dynamically are neglected. To address this issue, we propose a lightweight Cross-receptive Focused Inference Network (CFIN) that consists of a cascade of CT Blocks mixed with CNN and Transformer. Specifically, in the CT block, we first propose a CNN-based Cross-Scale Information Aggregation Module (CIAM) to enable the model to better focus on potentially helpful information to improve the efficiency of the Transformer phase. Then, we design a novel Cross-receptive Field Guided Transformer (CFGT) to enable the selection of contextual information required for reconstruction by using a modulated convolutional kernel that understands the current semantic information and exploits the information interaction within different self-attention. Extensive experiments have shown that our proposed CFIN can effectively reconstruct images using contextual information, and it can strike a good balance between computational cost and model performance as an efficient model. Source codes will be available at https://***/IVIPlab/CFIN. Copyright © 2022, The Authors. All rights reserved.

关键词： Semantics

The 5th International Conference on Biomedical Engineering and Biotechnology (ICBEB 2016) Abstracts

学校读者我要写书评

暂无评论

BMC MEDICAL IMAGING 2016年第SUPPL 1期16卷 65-65页

作者： [Anonymous] Department of MRI Shandong Medical Imaging Research Institute Affiliated to Shandong University Jinan Shandong 250021 People’s Republic of China Department of Interventional Radiology Shandong Provincial Hospital Affiliated to Shandong University Jinan Shandong 250021 People’s Republic of China College of Information Science and Technology Engr. Research Center of Digitized Textile & Fashion Tech. for Ministry of Education Donghua University Shanghai 201620 China Intelligent multimedia information processing Lab College of Software Northeastern University Shenyang Liaoning Province 110004 China Institute of Biomedical and Health Engineering Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen 518055 China School of Computer Science and Technology Nanjing Normal University Nanjing China Department of Electrical Engineering The City College of New York CUNY New York USA Jiangsu Key Laboratory of 3D Printing Equipment and Manufacturing Nanjing China School of Electronic Science and Engineering Nanjing University Nanjing Jiangsu 210046 China College of Engineering Nanyang Technological University Singapore 639798 Singapore School of Electronic Information Shanghai Dianji University Shanghai China School of Natural Sciences and Mathematics Shepherd University Shepherdstown WV 25443 USA Davis College of Agriculture Natural Resources and Design West Virginia University Morgantown WV 26505 USA State Key Laboratory of Millimeter Waves Southeast University Nanjing 210096 China Center of Medical Physics and Technology Hefei Institutes of Physical Science Chinese Academy of Sciences Hefei China College of Agricultural and Life Sciences University of Florida Gainesville FL 32611 USA Courant Institute of Mathematical Sciences New York University New York NY 10012 USA Translational Imaging Division & MRI Unit Columbia University and New York State Psychiatric Institute New York NY 10032 USA Guangxi Key Laboratory of Manufacturing System & Adv