检索结果-内蒙古大学图书馆

VIDA: HOMEOSTATIC VISUAL DOMAIN ADAPTER FOR CONTINUAL TEST TIME ADAPTATION

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Liu, Jiaming Yang, Senqiao Jia, Peidong Zhang, Renrui Lu, Ming Guo, Yandong Xue, Wei Zhang, Shanghang National Key Laboratory for Multimedia Information Processing School of Computer Science Peking University China AI2Robotics China The Chinese University of Hong Kong Hong Kong Hong Kong University of Science and Technology Hong Kong

Since real-world machine systems are running in non-stationary environments, Continual Test-Time Adaptation (CTTA) task is proposed to adapt the pre-trained model to continually changing target domains. Recently, existing methods mainly focus on model-based adaptation, which aims to leverage a self-training manner to extract the target domain knowledge. However, pseudo labels can be noisy and the updated model parameters are unreliable under dynamic data distributions, leading to error accumulation and catastrophic forgetting in the continual adaptation process. To tackle these challenges and maintain the model plasticity, we design a Visual Domain Adapter (ViDA) for CTTA, explicitly handling both domain-specific and domain-shared knowledge. Specifically, we first comprehensively explore the different domain representations of the adapters with trainable high-rank or low-rank embedding spaces. Then we inject ViDAs into the pre-trained model, which leverages high-rank and low-rank features to adapt the current domain distribution and maintain the continual domain-shared knowledge, respectively. To exploit the low-rank and high-rank ViDAs more effectively, we further propose a Homeostatic Knowledge Allotment (HKA) strategy, which adaptively combines different knowledge from each ViDA. Extensive experiments conducted on four widely used benchmarks demonstrate that our proposed method achieves state-of-the-art performance in both classification and segmentation CTTA tasks. Note that, our method can be regarded as a novel transfer paradigm for large-scale models, delivering promising results in adaptation to continually changing distributions. Project page: https://***/view/iclr2.2.-vida/home Copyright © 2.2., The Authors. All rights reserved.

关键词： Domain Knowledge

Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xu, Guoan Li, Juncheng Gao, Guangwei Lu, Huimin Yang, Jian Yue, Dong The Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing210023 China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou215006 China The School of Communication and Information Engineering Shanghai University Shanghai200444 China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing210049 China The Kyushu Institute of Technology Kitakyushu804-8550 Japan The School of Computer Science and Technology Nanjing University of Science and Technology Nanjing210049 China

In the past decade, convolutional neural networks (CNNs) have shown prominence for semantic segmentation. Although CNN models have very impressive performance, the ability to capture global representation is still insufficient, which results in suboptimal results. Recently, Transformer achieved huge success in NLP tasks, demonstrating its advantages in modeling long-range dependency. Recently, Transformer has also attracted tremendous attention from computer vision researchers who reformulate the image processing tasks as a sequence-to-sequence prediction but resulted in deteriorating local feature details. In this work, we propose a lightweight real-time semantic segmentation network called LETNet. LETNet combines a U-shaped CNN with Transformer effectively in a capsule embedding style to compensate for respective deficiencies. Meanwhile, the elaborately designed Lightweight Dilated Bottleneck (LDB) module and Feature Enhancement (FE) module cultivate a positive impact on training from scratch simultaneously. Extensive experiments performed on challenging datasets demonstrate that LETNet achieves superior performances in accuracy and efficiency balance. Specifically, It only contains 0.95M parameters and 13.6G FLOPs but yields 72.8% mIoU at 12. FPS on the Cityscapes test set and 70.5% mIoU at 2.0 FPS on the CamVid test dataset using a single RTX 3090 GPU. Source code will be available at https://***/IVIPLab/LETNet. Copyright © 2.2., The Authors. All rights reserved.

关键词： Semantic Segmentation

A spectrum-enhanced attention model for semantic segmentation of remote sensing images

学校读者我要写书评

暂无评论

A spectrum-enhanced attention model for semantic segmentatio...

International Conference on Acoustics, Speech, and Signal processing (ICASSP)

作者： Xin Li Feng Xu Feifei Tao Yao Tong Xin Lyu Jianyi Zhong André Kaup College of Computer Science and Software Engineering Hohai University Nanjing China Key Laboratory of Water Big Data Technology of Ministry of Water Resources Hohai University Nanjing China School of Computer Engineering Jiangsu Ocean University Lianyungang China School of Artificial Intelligence and Information Technology Nanjing University of Chinese Medicine Nanjing China Chair of Multimedia Communications and Signal Processing Friedrich-Alexander University Erlangen-Nuremberg Erlangen Germany

ISBN: (数字)9798350368741

ISBN: (纸本)9798350368758

Semantic segmentation of remote sensing images (RSIs) is essential for applications such as environmental monitoring, urban planning, and disaster management. Convolutional Neural Networks (CNNs) and their variants struggle to capture comprehensive spectral context for learning discriminative representations. In this paper, we propose a Spectrum-Enhanced Network (SPENet) that leverages the Frequency Transformer Block (FTB) to capture rich spectral context. FTB integrates Spectrum-Enhanced Attention (SEA) with Multi-Head Frequency Self-Attention (MH-FSA), incorporating more informative contextual cues. Specifically, SEA aggregates spectral statistics through covariance matrix normalization before applying channel-wise attention. By projecting feature maps onto the frequency domain, MH-FSA provides the network with a broader context, extending beyond the low-frequency focus of standard self-attention mechanisms. Extensive experiments on the ISPRS Potsdam and LoveDA datasets show that SPENet significantly outperforms state-of-the-art methods. Besides, the proposed SEA module notably rises average F1-score/overall accuracy/mean insert over union wiht more than 2.5/2.6%/2.3%, as demonstrated by ablation study.

关键词： Semantic segmentation Frequency-domain analysis Urban planning Semantics Speech enhancement Signal processing Transformers Sensors Remote sensing Standards

Design of Spin-Orbit Controlled Plasmonic Circuits for Selectively Transporting Photons

学校读者我要写书评

暂无评论

SSRN

SSRN 2024年

作者： Cai, Zimeng Wu, Jing Zhang, Yuhong Wu, Yongfeng Li, Jinhua Ma, Yuan Ma, Youqiao School of Physics and Optoelectronic Engineering Nanjing University of Information Science and Technology Nanjing210044 China Department of Electrical and Computer Engineering Dalhousie University HalifaxNSB3J 2X4 Canada Jiangsu Key Laboratory for Optoelectronic Detection of Atmosphere and Ocean Nanjing University of Information Science & Technology Nanjing210044 China

The manipulation of spin-related photons is crucial for developing next-generation integrated circuits. However, the spin-orbit interaction of light is typically very weak at nanoscale due to the diffraction limit. Here, a dielectric-loaded plasmonic nanocircuit is demonstrated, utilizing a chiral cylindrical structure, to directionally couple and guide plasmonic waves through dual-branch waveguides. Optimized results reveal that the waveguide modes excited by circularly polarized light with different spin states can be selectively coupled to one of the two targeted branch waveguides, with a calculated unidirectional coupling efficiency as high as 0.95. The proposed structure could provide avenues for realizing on-chip integrated nanocircuits and developing advanced multifunctional nanocircuits with enhanced degrees of freedom. © 2.2., The Authors. All rights reserved.

关键词： Photons

Discrimination-Aware Domain Adversarial Neural Network

学校读者我要写书评

暂无评论

Journal of computer Science & technology 2020年第2期35卷 259-267页

作者： Yun-Yun Wang Jian-Min Gu Chao Wang Song-Can Chen Hui Xue College of Computer Science and Engineering Nanjing University of Posts and Telecommunications Nanjing 210046China Jiangsu Key Laboratory of Big Data Security and Intelligent Processing Nanjing 210046China College of Computer Science and Technology/College of Artificial Intelligence Nanjing University of Aeronautics and AstronauticsNanjing 210023China Key Laboratory of Pattern Analysis and Machine Intelligence Ministry of Industry and Information Technology Nanjing 210023China School of Computer Science and Engineering Southeast UniversityNanjing 210096China

The domain adversarial neural network(DANN)methods have been successfully proposed and attracted much attention *** DANNs,a discriminator is trained to discriminate the domain labels of features generated by a generator,whereas the generator attempts to confuse it such that the distributions between domains are *** a result,it actually encourages the whole alignment or transfer between domains,while the inter-class discriminative information across domains is not *** this paper,we present a Discrimination-Aware Domain Adversarial Neural Network(DA2.N)method to introduce the discriminative information or the discrepancy of inter-class instances across domains into deep domain ***2.N considers both the alignment within the same class and the separation among different classes across domains in knowledge transfer via multiple *** results show that DA2.N can achieve better classification performance compared with the DANN methods.

关键词： adversarial learning inter-class separation deep neural network discrimination-aware domain adaptation

Novel Personalized Multimedia Recommendation Systems Using Tensor Singular-Value-Decomposition

学校读者我要写书评

暂无评论

Novel Personalized Multimedia Recommendation Systems Using T...

2.2. IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, BMSB 2.2.

作者： Chang, Shih Yu Wu, Hsiao-Chun Yan, Kun Chen, Xinjia Wu, Yiyan San Jose State University Department of Applied Data Science San JoseCA95192 United States Louisiana State University School of Electrical Engineering and Computer Science Baton RougeLA70803 United States Guilin University of Electronic Technology Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing Guilin541004 China Guilin University of Electronic Technology Department of Information and Telecommunication Guilin541004 China Northwestern State University of Louisiana Department of Engineering Technology NatchitochesLA71497 United States Communications Research Centre Canada OttawaONK2H 8S2 Canada

ISBN: (纸本)9798350321524

Nowadays, multimedia data are often produced by various sources, e.g., Internet of Things (IoT), social media, customer databases, etc. As tremendous multimedia data are produced every day, users or E-commerce customers cannot infer information from such data by themselves and therefore recommender systems have been developed to help users to select the products or services which better fit users' preferences or requirements. Nonetheless, there exists few works on the incorporation of side information or multiple attributes about items into the design of a more robust recommender system. In this work, we propose a novel approach based on the third-order tensor singular-value-decomposition (T3-SVD) to design new personalized multimedia recommender systems (PMRSs) for internet users. A PMRS can dynamically adjust its recommendation strategy subject to a particular user's on-line transaction behavior. To evaluate the effectiveness of our proposed PMRS based on T3-SVD, we compare the performances of our proposed new PMRS and two other existing tensor-based recommendation systems over realworld data in terms of normalized root-mean-square error (NRMSE). As a result, our proposed new PMRS greatly outperforms the other two existing systems. © 2.2. IEEE.

关键词： Recommender systems

Slnspeech: Solving Extended Speech Separation Problem with the Help of Sign Language

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： Wu, Jiasong Li, Xuan Li, Taotao Meng, Fanman Kong, Youyong Yang, Guanyu Senhadji, Lotfi Shu, Huazhong Laboratory of Image Science and Technology Key Laboratory of Computer Network and Information Integration Southeast University Ministry of Education Nanjing210096 China Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing Southeast University Nanjing210096 China Univ Rennes INSERM LTSI - UMR 1099 RennesF-35000 France Univ Rennes INSERM RennesF-35000 France

Previous audio-visual speech separation methods use the synchronization of the speaker's facial movement and speech in the video to supervise the speech separation in a self-supervised way. In this paper, we propose a model to solve the speech separation problem assisted by both face and sign language, which we call the extended speech separation problem. We design a general deep learning network for learning the combination of three modalities, audio, face, and sign language information, for better solving the speech separation problem. To train the model, we introduce a large-scale dataset named the Sign Language News Speech (SLNSpeech) dataset in which three modalities of audio, face, and sign language coexist. Experiment results show that the proposed model has better performance and robustness than the usual audio-visual system. Besides, sign language modality can also be used alone to supervise speech separation task, and the introduction of sign language is helpful for hearing-impaired people to learn and communicate. Last, our model is a general speech separation framework and can achieve very competitive separation performance on two open-source audio-visual datasets. The code is available at https://***/iveveive/SLNSpeech. © 2.2., The Authors. All rights reserved.

关键词： Audition

EWT: Efficient Wavelet-Transformer for Single Image Denoising

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Juncheng Cheng, Bodong Chen, Ying Gao, Guangwei Zeng, Tieyong The School of Communication & Information Engineering Shanghai University Shanghai China Jiangsu Key Laboratory of Image and Video Understanding for Social Safety Nanjing University of Science and Technology Nanjing China The School of Computer Science and Technology Xidian University Xian China The Department of Cyberspace Security Beijing Electronic Science & Technology Institute Beijing China The Institute of Advanced Technology Nanjing University of Posts and Telecommunications Nanjing China The Provincial Key Laboratory for Computer Information Processing Technology Soochow University Suzhou China The Department of Mathematics The Chinese University of Hong Kong New Territories Hong Kong

Transformer-based image denoising methods have achieved encouraging results in the past year. However, it must uses linear operations to model long-range dependencies, which greatly increases model inference time and consumes GPU storage space. Compared with convolutional neural network-based methods, current Transformer-based image denoising methods cannot achieve a balance between performance improvement and resource consumption. In this paper, we propose an Efficient Wavelet Transformer (EWT) for image denoising. Specifically, we use Discrete Wavelet Transform (DWT) and Inverse Wavelet Transform (IWT) for downsampling and upsampling, respectively. This method can fully preserve the image features while reducing the image resolution, thereby greatly reducing the device resource consumption of the Transformer model. Furthermore, we propose a novel Dual-stream Feature Extraction Block (DFEB) to extract image features at different levels, which can further reduce model inference time and GPU memory usage. Experiments show that our method speeds up the original Transformer by more than 80%, reduces GPU memory usage by more than 60%, and achieves excellent denoising results. All code will be public. © 2.2., CC BY.

关键词： Image denoising

Efficient Gridless DoA Estimation Method of Non-uniform Linear Arrays with Applications in Automotive Radars

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Gao, Silin Zhang, Zhe Wang, Muhan Zhang, Yan Zhao, Jie Zhang, Bingchen Wang, Yue Wu, Yirong Aerospace Information Research Institute Chinese Academy of Sciences Beijing China Key Laboratory of Technology in Geo-spatial Information Processing and Application System Chinese Academy of Sciences Beijing China School of Electronic Electrical and Communication Engineering University of Chinese Academy of Sciences Beijing China Suzhou Key Laboratory of Microwave Imaging Processing and Application Technology Suzhou Aerospace Information Research Institute Jiangsu Suzhou China The Beijing Autoroad Tech Co. Ltd China Electrical and Computer Engineering Department George Mason University FairfaxVA United States

This paper focuses on the gridless direction-of-arrival (DoA) estimation for data acquired by non-uniform linear arrays (NLAs) in automotive applications. Atomic norm minimization (ANM) is a promising gridless sparse recovery algorithm under the Toeplitz model and solved by convex relaxation, thus it is only applicable to uniform linear arrays (ULAs) with array manifolds having a Vandermonde structure. In automotive applications, it is essential to apply the gridless DoA estimation to NLAs with arbitrary geometry with efficiency. In this paper, a fast ANM-based gridless DoA estimation algorithm for NLAs is proposed, which employs the array manifold separation technique and the accelerated proximal gradient (APG) technique, making it applicable to NLAs without losing of efficiency. Simulation and measurement experiments on automotive multiple-input multiple-outpt (MIMO) radars demonstrate the superiority of the proposed method. Copyright © 2.2., The Authors. All rights reserved.

关键词： Efficiency