检索结果-内蒙古大学图书馆

CLIP-guided Prototype Modulating for Few-shot Action Recognition

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wang, Xiang Zhang, Shiwei Cen, Jun Gao, Changxin Zhang, Yingya Zhao, Deli Sang, Nong Key Laboratory of Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology China Alibaba Group China The Hong Kong University of Science and Technology Hong Kong

Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explored on the challenging few-shot action recognition (FSAR) task. In this work, we aim to transfer the powerful multimodal knowledge of CLIP to alleviate the inaccurate prototype estimation issue due to data scarcity, which is a critical problem in low-shot regimes. To this end, we present a CLIP-guided prototype modulating framework called CLIP-FSAR, which consists of two key components: a video-text contrastive objective and a prototype modulation. Specifically, the former bridges the task discrepancy between CLIP and the few-shot video task by contrasting videos and corresponding class text descriptions. The latter leverages the transferable textual concepts from CLIP to adaptively refine visual prototypes with a temporal Transformer. By this means, CLIP-FSAR can take full advantage of the rich semantic priors in CLIP to obtain reliable prototypes and achieve accurate few-shot classification. Extensive experiments on five commonly used benchmarks demonstrate the effectiveness of our proposed method, and CLIP-FSAR significantly outperforms existing state-of-the-art methods under various settings. The source code and Copyright © 2023, The Authors. All rights reserved.

关键词： Semantics

Active Compliance Variable Impedance control of Lower Supernumerary Robotic Limb*

学校读者我要写书评

暂无评论

Active Compliance Variable Impedance Control of Lower Supern...

International Conference on Advanced Robotics and Mechatronics (ICARM)

作者： Jun Huo Xinyi Wang Bo Yang Qingyang Yan Zhaofan Yuan Jian Huang Hubei Key Laboratory of Brain-inspired Intelligent Systems and the Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan China China Telecom Research Institute Guangdong China China Ship Development and Design Centre Wuhan China

ISBN: (数字)9798350385724

ISBN: (纸本)9798350385731

In human-robot systems, ensuring user safety amidst external disturbances remains paramount. An efficient variable impedance control (VIC) technique is designed to enhance human-robot interaction, particularly in scenarios with external force disturbances. By dynamically adapting impedance parameters in response to detected changes in contact force, VIC achieves heightened system flexibility. The robust stability of the VIC system is rigorously established using the Lyapunov approach, with a parameter update law tailored for a lower limb supernumerary robotic limb robot. Through simulations and experiments, the efficacy of the system in maintaining a specific safety margin is demonstrated. Moreover, the technique offers a pragmatic solution to accommodate individual variations in walking gait and cross-user disparities in interaction. The proposed method contributes to advancing the safety and adaptability of cross-user human-robot systems.

关键词： Legged locomotion Adaptation models Mechatronics Robust stability Force Human-robot interaction Safety Impedance Robots Pragmatics

Boosting Explore-Exploit Behavior for Navigating Vehicle by Maintaining Informative Topological Frontier

学校读者我要写书评

暂无评论

IEEE Transactions on Intelligent Vehicles

IEEE Transactions on Intelligent Vehicles 2024年 1-11页

作者： Liu, Qiming Cui, Xinru Liu, Zhe Wang, Hesheng Department of Automation Shanghai Jiao Tong University Shanghai China MoE Key Lab of Artificial Intelligence AI Institute Shanghai Jiao Tong University Shanghai China Department of Automation Key Laboratory of System Control and Information Processing of Ministry of Education Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education Shanghai Engineering Research Center of Intelligent Control and Management Shanghai Jiao Tong University Shanghai China

Target search in unknown environments places high demands not only on an autonomous vehicle's ability to perceive and interpret target cues, but also on its conscious of collecting these cues by active exploration. While existing navigation methods have successfully built target-driven policies by maintaining memory of explored areas, there has been a lack of focus on facilitating target-aware exploration-the informative frontier information at unexplored yet visible areas is often overlooked. In this paper, we introduce a novel topology-based memory structure, Frontier-enhanced Topological Memory (FTM), and a Hierarchical Topology Encoding and Extraction (HTEE) module, fostering the autonomous vehicle's awareness of both environmental exploration and target approach. Specifically, FTM innovatively incorporates informative ghost nodes on traditional topological map to represent unexplored yet visible regions. We leverage an online-trained implicit scene representation to estimate the positions and generate features of these ghost nodes. The HTEE then employs implicit graph convolutions and attention mechanisms to extract cognitive information from FTM, taking into account the hierarchical memory structure, target cues, and current state. Our design bolsters cognitive navigation decisions. The experiments in the high-fidelity environments, including performance tests, visualizations, and interpretability experiments, validate the effectiveness of our approach in enhancing the vehicle's exploratory behavior. The improved exploration awareness for target cue collection, in turn, enhances the success rate and path efficiency of target search. Furthermore, we demonstrate the adaptability of our algorithm in real-world physical environments. IEEE

关键词： Data mining

Fully Circuit Implementation of a two-layer Memristive Neural Network for Pattern Recognition 11

学校读者我要写书评

暂无评论

Fully Circuit Implementation of a two-layer Memristive Neura...

11th International Conference on Information Science and Technology, ICIST 2021

作者： Li, Mian Wang, Xiaoping Chen, Zhanfei Huazhong University of Science and Technology School of Artificial Intelligence and Automation Wuhan China Huazhong University of Science and Technology School of Artificial Intelligence and Automation Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Wuhan China

ISBN: (纸本)9781665412667

In this paper, a fully circuit implementation of Memristive Neural Network (MNN) is proposed. The forward calculation of the network is based on winner-take-all (WTA) mechanism. The weight updating is achieved through the difference of pre-spike and post-spike, which is more close to the biological weight adjustment mechanism. The network is implemented in a full circuit without additional control units. The designed circuit consists of four modules. The memristive crossbar array module with one-memristor (1M) unit structure can effectively calculate the vector-matrix multiplication with only one step. The switch S is replaced by the transistor in the designed leaky-integrate-and-fire (LIF) module, which can control the integration and leakage of the membrane voltage and realize the lateral inhibition between output neurons. Connecting the integrated monostable trigger and the difference circuit, the post-spike generating module can output the required post-spike. The signal switch module realizes the switching of signals connected to memristors by using voltage-controlled switches. The combination of two modules validly realizes weight updating. The functions of four modules are verified separately. We performed a simulation experiment of 5×3 pixels image classification based on the designed circuit in PSPICE. The circuit output results and the high classification accuracy prove the circuit can be effectively applied in pattern recognition. The noise experiment shows the robustness of the designed circuit. © 2021 IEEE.

关键词： image classification

Effective and Efficient Intracortical Brain Signal Decoding with Spiking Neural Networks

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Fu, Haotian Zhang, Peng Yang, Song Zhang, Herui Wang, Ziwei Wu, Dongrui Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China Department of Biomedical Engineering College of Life Science and Technology Huazhong University of Science and Technology Wuhan430074 China

A brain-computer interface (BCI) facilitates direct interaction between the brain and external devices. To concurrently achieve high decoding accuracy and low energy consumption in invasive BCIs, we propose a novel spiking neural network (SNN) framework incorporating local synaptic stabilization (LSS) and channel-wise attention (CA), termed LSS-CA-SNN. LSS optimizes neuronal membrane potential dynamics, boosting classification performance, while CA refines neuronal activation, effectively reducing energy consumption. Furthermore, we introduce SpikeDrop, a data augmentation strategy designed to expand the training dataset thus enhancing model generalizability. Experiments on invasive spiking datasets recorded from two rhesus macaques demonstrated that LSS-CA-SNN surpassed state-of-the-art artificial neural networks (ANNs) in both decoding accuracy and energy efficiency, achieving 0.80-3.87% performance gains and 14.78-43.86 times energy saving. This study highlights the potential of LSS-CA-SNN and SpikeDrop in advancing invasive BCI applications. © 2024, CC BY.

关键词： Neurons

Instance Consistency Regularization for Semi-Supervised 3D Instance Segmentation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Wu, Yizheng Pan, Zhiyu Wang, Kewei Li, Xingyi Cui, Jiahao Xiao, Liwen Lin, Guosheng Cao, Zhiguo The Key Laboratory of Image Processing and Intelligent Control Ministry of Education School of Artificial Intelligence and Automation Huazhong University of Science and Technology Wuhan430074 China S-Lab Nanyang Technological University Singapore639798 Singapore Nanyang Technological University Singapore639798 Singapore

Large-scale datasets with point-wise semantic and instance labels are crucial to 3D instance segmentation but also expensive. To leverage unlabeled data, previous semi-supervised 3D instance segmentation approaches have explored self-training frameworks, which rely on high-quality pseudo labels for consistency regularization. They intuitively utilize both instance and semantic pseudo labels in a joint learning manner. However, semantic pseudo labels contain numerous noise derived from the imbalanced category distribution and natural confusion of similar but distinct categories, which leads to severe collapses in self-training. Motivated by the observation that 3D instances are non-overlapping and spatially separable, we ask whether we can solely rely on instance consistency regularization for improved semi-supervised segmentation. To this end, we propose a novel self-training network InsTeacher3D to explore and exploit pure instance knowledge from unlabeled data. We first build a parallel base 3D instance segmentation model DKNet, which distinguishes each instance from the others via discriminative instance kernels without reliance on semantic segmentation. Based on DKNet, we further design a novel instance consistency regularization framework to generate and leverage high-quality instance pseudo labels. Experimental results on multiple large-scale datasets show that the InsTeacher3D significantly outperforms prior state-of-the-art semi-supervised approaches. Code is available: https://***/W1zheng/InsTeacher3D. Copyright © 2024, The Authors. All rights reserved.

关键词： Semantics

Implicit Tracking-Based Distributed Constraint-Coupled Optimization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Jingwang Su, Housheng The School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

A class of distributed optimization problem with a globally coupled equality constraint and local constrained sets is studied in this paper. For its special case where local constrained sets are absent, an augmented primal-dual gradient dynamics is proposed and analyzed, but it cannot be implemented distributedly since the violation of the coupled constraint needs to be used. Benefiting from the brand-new comprehending of a classical distributed unconstrained optimization algorithm, the novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the implicit tracking-based distributed augmented primal-dual gradient dynamics (IDEA). A projected variant of IDEA, i.e., Proj-IDEA, is further designed to deal with the general case where local constrained sets exist. With the aid of the Lyapunov stability theory, the convergences of IDEA and Proj-IDEA over undigraphs and digraphs are analyzed respectively. As far as we know, Proj-IDEA is the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions. Besides, if local cost functions are strongly convex and smooth, IDEA can achieve exponential convergence with a weaker condition about the coupled constraint. Finally, numerical experiments are taken to corroborate our theoretical results. © 2022, CC BY.

关键词： Cost functions

NPGA: A Unified Algorithmic Framework for Decentralized Constraint-Coupled Optimization

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Jingwang Su, Housheng School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

This work focuses on a class of general decentralized constraint-coupled optimization problems. We propose a novel nested primal-dual gradient algorithm (NPGA), which can achieve linear convergence under the weakest known condition, and its theoretical convergence rate surpasses all known results. More importantly, NPGA serves not only as an algorithm but also as a unified algorithmic framework, encompassing various existing algorithms as special cases. By designing different network matrices, we can derive numerous versions of NPGA and analyze their convergences by leveraging the convergence results of NPGA conveniently, thereby enabling the design of more efficient algorithms. Finally, we conduct numerical experiments to compare the convergence rates of NPGA and existing algorithms, providing empirical evidence for the superior performance of NPGA. Copyright © 2022, The Authors. All rights reserved.

关键词： Consensus algorithm

GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Jiang Wang, Xiaoping Lv, Guoqing Zeng, Zhigang The School of Artificial Intelligence and Automation The Key Laboratory of Image Processing and Intelligent Control of Education Ministry of China Huazhong University of Science and Technology Wuhan430074 China

Emotion Recognition in Conversation (ERC) plays a significant part in Human-Computer Interaction (HCI) systems since it can provide empathetic services. Multimodal ERC can mitigate the drawbacks of uni-modal approaches. Recently, Graph Neural Networks (GNNs) have been widely used in a variety of fields due to their superior performance in relation modeling. In multimodal ERC, GNNs are capable of extracting both long-distance contextual information and inter-modal interactive information. Unfortunately, since existing methods such as MMGCN directly fuse multiple modalities, redundant information may be generated and diverse information may be lost. In this work, we present a directed Graph based Cross-modal Feature Complementation (GraphCFC) module that can efficiently model contextual and interactive information. GraphCFC alleviates the problem of heterogeneity gap in multimodal fusion by utilizing multiple subspace extractors and Pair-wise Cross-modal Complementary (PairCC) strategy. We extract various types of edges from the constructed graph for encoding, thus enabling GNNs to extract crucial contextual and interactive information more accurately when performing message passing. Furthermore, we design a GNN structure called GAT-MLP, which can provide a new unified network framework for multimodal learning. The experimental results on two benchmark datasets show that our GraphCFC outperforms the state-of-the-art (SOTA) approaches. Copyright © 2022, The Authors. All rights reserved.

关键词： Emotion Recognition