检索结果-内蒙古大学图书馆

CLIP-guided Prototype Modulating for Few-shot Action Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xiang Zhang, Shiwei Cen, Jun Gao, Changxin Zhang, Yingya Zhao, Deli Sang, Nong Key Laboratory of Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Alibaba Group China The Hong Kong University of Science and Technology Hong Kong

Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explored on the challenging few-shot action recognition (FSAR) task. In this work, we aim to transfer the powerful multimodal knowledge of CLIP to alleviate the inaccurate prototype estimation issue due to data scarcity, which is a critical problem in low-shot regimes. To this end, we present a CLIP-guided prototype modulating framework called CLIP-FSAR, which consists of two key components: a video-text contrastive objective and a prototype modulation. Specifically, the former bridges the task discrepancy between CLIP and the few-shot video task by contrasting videos and corresponding class text descriptions. The latter leverages the transferable textual concepts from CLIP to adaptively refine visual prototypes with a temporal Transformer. By this means, CLIP-FSAR can take full advantage of the rich semantic priors in CLIP to obtain reliable prototypes and achieve accurate few-shot classification. Extensive experiments on five commonly used benchmarks demonstrate the effectiveness of our proposed method, and CLIP-FSAR significantly outperforms existing state-of-the-art methods under various settings. The source code and Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

Design and Optimization of Liquid state Machine for Handwritten Digit Recognition

学校读者我要写书评

暂无评论

Design and Optimization of Liquid State Machine for Handwrit...

2022 Workshop on Electronics Communication Engineering

作者： Hou, Aihu Liu, Jie Zhou, Kaibo Zhou, Wenli School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China School of Civil and Hydraulic Engineering Huazhong University of Science and Technology Wuhan430074 China Hubei Key Laboratory of Digital River Basin Science and Technology Huazhong University of Science and Technology Wuhan430074 China Key Laboratory of Image Processing and Intelligent Control Huazhong University of Science and Technology Ministry of Education Wuhan430074 China School of Optical and Electronic information Huazhong University of Science and Technology Wuhan430074 China

ISBN: (纸本)9781510666634

The most commonly used implementation of handwritten digit recognition based on convolutional neural networks requires equipment with high computing power, which is not suitable for edge devices. Recently, spiking neural network (SNN) has received more attentions due to its low power consumption and real-time performance, but training SNN is very difficult. As a special SNN, liquid state machine (LSM) has the advantages of simple structure and easy training, so it is very suitable for handwritten digit recognition on edge devices. But it has no advantage in recognition accuracy. In order to improve the performance of LSM, its reservoir needs to be optimized. In this paper, an efficient local optimization strategy is proposed, improving the recognition accuracy of LSM by 11.8% with less the training time. In order to reduce the runtime, auto-encoder and feature screening are used to compress the input handwritten digit image. After feature compression, the input storage is reduced by 57%, and the runtime is reduced by 30%. This work provides an effective way to realize handwritten digit recognition on the edge devices. © 2023 SPIE.

关键词： Computing power

Effective and Efficient Intracortical Brain Signal Decoding with Spiking Neural Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Haotian Zhang, Peng Yang, Song Zhang, Herui Wang, Ziwei Wu, Dongrui Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Department of Biomedical Engineering College of Life Science and Technology Huazhong University of Science and Technology Wuhan430074 China

A brain-computer interface (BCI) facilitates direct interaction between the brain and external devices. To concurrently achieve high decoding accuracy and low energy consumption in invasive BCIs, we propose a novel spiking neural network (SNN) framework incorporating local synaptic stabilization (LSS) and channel-wise attention (CA), termed LSS-CA-SNN. LSS optimizes neuronal membrane potential dynamics, boosting classification performance, while CA refines neuronal activation, effectively reducing energy consumption. Furthermore, we introduce SpikeDrop, a data augmentation strategy designed to expand the training dataset thus enhancing model generalizability. Experiments on invasive spiking datasets recorded from two rhesus macaques demonstrated that LSS-CA-SNN surpassed state-of-the-art artificial neural networks (ANNs) in both decoding accuracy and energy efficiency, achieving 0.80-3.87% performance gains and 14.78-43.86 times energy saving. This study highlights the potential of LSS-CA-SNN and SpikeDrop in advancing invasive BCI applications. © 2024, CC BY.

关键词： Neurons

Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Yizheng Pan, Zhiyu Wang, Kewei Li, Xingyi Cui, Jiahao Xiao, Liwen Lin, Guosheng Cao, Zhiguo The Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China S-Lab Nanyang Technological University Singapore639798 Singapore Nanyang Technological University Singapore639798 Singapore

Large-scale datasets with point-wise semantic and instance labels are crucial to 3D instance segmentation but also expensive. To leverage unlabeled data, previous semi-supervised 3D instance segmentation approaches have explored self-training frameworks, which rely on high-quality pseudo labels for consistency regularization. They intuitively utilize both instance and semantic pseudo labels in a joint learning manner. However, semantic pseudo labels contain numerous noise derived from the imbalanced category distribution and natural confusion of similar but distinct categories, which leads to severe collapses in self-training. Motivated by the observation that 3D instances are non-overlapping and spatially separable, we ask whether we can solely rely on instance consistency regularization for improved semi-supervised segmentation. To this end, we propose a novel self-training network InsTeacher3D to explore and exploit pure instance knowledge from unlabeled data. We first build a parallel base 3D instance segmentation model DKNet, which distinguishes each instance from the others via discriminative instance kernels without reliance on semantic segmentation. Based on DKNet, we further design a novel instance consistency regularization framework to generate and leverage high-quality instance pseudo labels. Experimental results on multiple large-scale datasets show that the InsTeacher3D significantly outperforms prior state-of-the-art semi-supervised approaches. Code is available: https://***/W1zheng/InsTeacher3D. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Jiang Wang, Xiaoping Lv, Guoqing Zeng, Zhigang The School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

Emotion Recognition in Conversation (ERC) plays a significant part in Human-Computer Interaction (HCI) systems since it can provide empathetic services. Multimodal ERC can mitigate the drawbacks of uni-modal approaches. Recently, Graph Neural Networks (GNNs) have been widely used in a variety of fields due to their superior performance in relation modeling. In multimodal ERC, GNNs are capable of extracting both long-distance contextual information and inter-modal interactive information. Unfortunately, since existing methods such as MMGCN directly fuse multiple modalities, redundant information may be generated and diverse information may be lost. In this work, we present a directed Graph based Cross-modal Feature Complementation (GraphCFC) module that can efficiently model contextual and interactive information. GraphCFC alleviates the problem of heterogeneity gap in multimodal fusion by utilizing multiple subspace extractors and Pair-wise Cross-modal Complementary (PairCC) strategy. We extract various types of edges from the constructed graph for encoding, thus enabling GNNs to extract crucial contextual and interactive information more accurately when performing message passing. Furthermore, we design a GNN structure called GAT-MLP, which can provide a new unified network framework for multimodal learning. The experimental results on two benchmark datasets show that our GraphCFC outperforms the state-of-the-art (SOTA) approaches. Copyright © 2022, The Authors. All rights reserved.

关键词： Emotion Recognition

GA2MIF: Graph and Attention Based Two-Stage Multi-Source Information Fusion for Conversational Emotion Detection

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Jiang Wang, Xiaoping Lv, Guoqing Zeng, Zhigang School of Artificial Intelligence and Automation the Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

Multimodal Emotion Recognition in Conversation (ERC) plays an influential role in the field of human-computer interaction and conversational robotics since it can motivate machines to provide empathetic services. Multimodal data modeling is an up-and-coming research area in recent years, which is inspired by human capability to integrate multiple senses. Several graph-based approaches claim to capture interactive information between modalities, but the heterogeneity of multimodal data makes these methods prohibit optimal solutions. In this work, we introduce a multimodal fusion approach named Graph and Attention based Two-stage Multi-source Information Fusion (GA2MIF) for emotion detection in conversation. Our proposed method circumvents the problem of taking heterogeneous graph as input to the model while eliminating complex redundant connections in the construction of graph. GA2MIF focuses on contextual modeling and cross-modal modeling through leveraging Multi-head Directed Graph ATtention networks (MDGATs) and Multi-head Pairwise Cross-modal ATtention networks (MPCATs), respectively. Extensive experiments on two public datasets (i.e., IEMOCAP and MELD) demonstrate that the proposed GA2MIF has the capacity to validly capture intra-modal long-range contextual information and inter-modal complementary information, as well as outperforms the prevalent state-Of-The-Art (SOTA) models by a remarkable margin. Copyright © 2022, The Authors. All rights reserved.

关键词： Directed graphs

A Novel YJQR-LSTM Model for Nonparametric Probabilistic Sustainable Agriculture Wind Power Forecasting Based on Intelligent IoT

学校读者我要写书评

暂无评论

IEEE Internet of Things Journal 2024年

作者： Wang, Jie Jiang, Junhui Chen, Xinlong Cai, Defu Wu, Yue Lu, Renzhi School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Harbin Engineering University College of Computer Science and Technology Harbin150001 China State Grid Hubei Electric Power Research Institute Hubei Wuhan430079 China School of Technology State Key Laboratory of Efficient Production of Forest Resources Key Laboratory of National Forestry and Grassland Administration on Forestry Equipment and Automation Beijing Forestry University Beijing100083 China Key Laboratory of Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Key Laboratory of Industrial Internet of Things and Networked Control Ministry of Education Chongqing400065 China State Key Laboratory of Mechanical Transmission for Advanced Equipment Chongqing University Chongqing400044 China Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems Wuhan430074 China

Energy costs associated with the consumption of non-renewable energy sources have become an important issue in improving the international competitiveness of agriculture. Wind power, as a renewable energy source, can replace non-renewable energy sources to reduce energy costs and improve the sustainability of agricultural. However, the inherent intermittency, randomness, and volatility within weather conditions and wind speed presents a substantial challenge in accurately predicting wind power generation. This work proposes a novel YJQR-LSTM algorithm that leverages Yeo-Johnson quantile regression (YJQR) with a long short-term memory (LSTM) network for nonparametric probabilistic forecasting of wind power generation via the Intelligent Internet of Things. First, an improved YJQR model based on the YJ transformation is designed to obtain a more precise characterization of wind uncertainty, providing a more flexible probability density function for wind power generation. Then, utilizing the unique structure of the LSTM network to learn the parameters of the YJQR model, temporal features can be extracted from time series data. To mitigate the impact of outliers in the raw data on accuracy and improve computational efficiency, a novel logarithmic likelihood function is developed as the loss function utilized in the training phase. The effectiveness of the proposed algorithm is validated using a real-world dataset from five wind farms from the Global Energy Forecasting Competition. Numerical results demonstrate that the algorithm provides more accurate wind power prediction results in complex wind power data environments, which is important for making full use of wind energy and thus reducing the consumption of non-renewable energy in agriculture. © 2024 IEEE.

关键词： Wind forecasting

Transmission-Constrained Consensus of Multiagent Networks

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Wang, Xiaotian Su, Housheng School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Wuhan430074 China

This paper studies the consensus problem for multiagent systems with transmission constraints. A novel model of multiagent systems is proposed where the information transmissions between agents are disturbed by irregular distortions or interferences (named transmission constraint functions), and this model is universal which can be applied in many cases, such as interval consensus and discarded consensus. In the transmission-constrained consensus problem, we obtain the necessary and sufficient condition that agents can converge to state consensus. Furthermore, a more general case is studied in which the system reaches an equilibrium. Based on some techniques of algebraic topology and stability theory, the existence, uniqueness and stability of the system equilibrium point can be proven, which means the system can reach an asymptotically stable equilibrium. Moreover, the state values of the equilibrium are only decided by the network structure and transmission constraint functions, but not the agents’ initial states. Finally, numerical simulations are presented to illustrate the proposed theorems and corollaries. © 2022, CC BY-NC-SA.

关键词： Directed graphs

Computationally Expensive High-Dimensional Multiobjective Optimization via Surrogate-Assisted Reformulation and Decomposition

学校读者我要写书评

暂无评论

IEEE Transactions on Evolutionary Computation 2024年 1-1页

作者： Pan, Linqiang Lin, Jianqing Wang, Handing He, Cheng Tan, Kay Chen Jin, Yaochu School of Artificial Intelligence and Automation Key Laboratory of Image Information Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan China School of Artificial Intelligence Xidian University Xi’an China School of Electrical and Electronic Engineering Huazhong University of Science and Technology Wuhan China Department of Computing The Hong Kong Polytechnic University Hong Kong Hong Kong School of Engineering Trustworth and General Artificial Intelligence Laboratory Westlake University Hangzhou China

In recent decades, various surrogate-assisted evolutionary algorithms (SAEAs) have been proposed to solve computationally expensive multiobjective optimization problems (EMOPs). Nevertheless, designing an SAEA to handle high-dimensional EMOPs and balance convergence, diversity, and computational complexity remains challenging. Here, we propose a two-phase SAEA (TP-SAEA), which follows the idea of convergence first and diversity second, for solving high-dimensional EMOPs. In Phase I, a surrogate-assisted problem reformulation method is proposed to fast-track the Pareto optimal set in association with some reference solutions. Specifically, the high-dimensional EMOP is reformulated into an expensive single-objective one with low-dimensional decision space. Then, the surrogate-assisted optimization is utilized to obtain well-converged solutions. In Phase II, the high-dimensional EMOP is decomposed into two subproblems to explore subregions of the decision space that can effectively promote the diversity of the solutions. The two subproblems are optimized independently via surrogate-assisted optimization, aiming to push the population towards different regions of the Pareto optimal front. Experiments are conducted on EMOPs with 100 to 500 decision variables compared with four state-of-the-art SAEAs. The proposed TP-SAEA obtains well-converged and diverse solutions with only 509 real function evaluations. Moreover, its superiority is examined in six real-world instances with up to 12,000 decision variables. IEEE

关键词： Iron