检索结果-内蒙古大学图书馆

Benchmarking Neural Decoding Backbones towards Enhanced On-edge iBCI Applications

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Zhou, Zhou He, Guohang Zhang, Zheng Leng, Luziwei Guo, Qinghai Liao, Jianxing Song, Xuan Cheng, Ran Southern University of Science and Technology Shenzhen518055 China Advanced Computing and Storage Lab Huawei Technologies Co. Ltd. Shenzhen518055 China

Traditional invasive Brain-Computer Interfaces (iBCIs) typically depend on neural decoding processes conducted on workstations within laboratory settings, which prevents their everyday usage. Implementing these decoding processes on edge devices, such as the wearables, introduces considerable challenges related to computational demands, processing speed, and maintaining accuracy. This study seeks to identify an optimal neural decoding backbone that boasts robust performance and swift inference capabilities suitable for edge deployment. We executed a series of neural decoding experiments involving nonhuman primates engaged in random reaching tasks, evaluating four prospective models, Gated Recurrent Unit (GRU), Transformer, Receptance Weighted Key Value (RWKV), and Selective State Space model (Mamba), across several metrics: single-session decoding, multi-session decoding, new session fine-tuning, inference speed, calibration speed, and scalability. The findings indicate that although the GRU model delivers sufficient accuracy, the RWKV and Mamba models are preferable due to their superior inference and calibration speeds. Additionally, RWKV and Mamba comply with the scaling law, demonstrating improved performance with larger data sets and increased model sizes, whereas GRU shows less pronounced scalability, and the Transformer model requires computational resources that scale prohibitively. This paper presents a thorough comparative analysis of the four models in various scenarios. The results are pivotal in pinpointing an optimal backbone that can handle increasing data volumes and is viable for edge implementation. This analysis provides essential insights for ongoing research and practical applications in the field. Copyright © 2024, The Authors. All rights reserved.

关键词： Scalability

Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Li, Boyan Leng, Luziwei Shen, Shuaijie Zhang, Kaixuan Zhang, Jianguo Liao, Jianxing Cheng, Ran The Department of Computer Science and Engineering Southern University of Science and Technology Shenzhen518055 China The Advanced Computing and Storage Lab Huawei Technologies Co. Ltd. Shenzhen518055 China

Advancements in adapting deep convolution architectures for Spiking Neural Networks (SNNs) have significantly enhanced image classification performance and reduced computational burdens. However, the inability of Multiplication-Free Inference (MFI) to align with attention and transformer mechanisms, which are critical to superior performance on high-resolution vision tasks, imposing limitations on these gains. To address this, our research explores a new pathway, drawing inspiration from the progress made in Multi-Layer Perceptrons (MLPs). We propose an innovative spiking MLP architecture that uses batch normalization to retain MFI compatibility and introducing a spiking patch encoding layer to enhance local feature extraction capabilities. As a result, we establish an efficient multi-stage spiking MLP network that blends effectively global receptive fields with local feature extraction for comprehensive spike-based computation. Without relying on pre-training or sophisticated SNN training techniques, our network secures a top-1 accuracy of 66.39% on the ImageNet-1K dataset, surpassing the directly trained spiking ResNet-34 by 2.67%. Furthermore, we curtail computational costs, model parameters, and simulation steps. An expanded version of our network compares with the performance of the spiking VGG-16 network with a 71.64% top-1 accuracy, all while operating with a model capacity 2.1 times smaller. Our findings highlight the potential of our deep SNN architecture in effectively integrating global and local learning abilities. Interestingly, the trained receptive field in our network mirrors the activity patterns of cortical cells. Source codes are publicly accessible at https://***/EMI-Group/mixer-snn. Copyright © 2023, The Authors. All rights reserved.

关键词： Image classification

Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptation

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Tang, Yushun Chen, Shuoshuo Kan, Zhehan Zhang, Yi Guo, Qinghai He, Zhihai The Department of Electrical and Electronic Engineering Southern University of Science and Technology Shenzhen China The Pengcheng Lab Shenzhen China The Advanced Computing and Storage Laboratory Huawei Technologies Co. LTD China

Fully test-time adaptation aims to adapt the network model based on sequential analysis of input samples during the inference stage to address the cross-domain performance degradation problem of deep neural networks. This work is based on the following interesting finding: in transformer-based image classification, the class token at the first transformer encoder layer can be learned to capture the domain-specific characteristics of target samples during test-time adaptation. This learned token, when combined with input image patch embeddings, is able to gradually remove the domain-specific information from the feature representations of input samples during the transformer encoding process, thereby significantly improving the test-time adaptation performance of the source model across different domains. We refer to this class token as visual conditioning token (VCT). To successfully learn the VCT, we propose a bi-level learning approach to capture the long-term variations of domain-specific characteristics while accommodating local variations of instance-specific characteristics. Experimental results on the benchmark datasets demonstrate that our proposed bi-level visual conditioning token learning method is able to achieve significantly improved test-time adaptation performance by up to 1.9%. Copyright © 2024, The Authors. All rights reserved.

关键词： Deep neural networks

Test-Time Training-Free Domain Adaptation 48

学校读者我要写书评

暂无评论

Test-Time Training-Free Domain Adaptation

48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

作者： Feng, Yongxiang He, Weihua You, Kaichao Liu, Bing Zhang, Ziyang Wang, Yaoyuan Li, Minglei Lou, Yihang Li, Jiawei Li, Guoqi Liao, Jianxing Huawei Technologies Co. Ltd. Advanced Computing and Storage Lab China Peking University School of Electronics Engineering and Computer Science China Huawei Technologies Co. Ltd. Language & Speech Innovation Lab China Huawei Technologies Co. Ltd. GoTen AI Lab Department of Intelligent Vision China Huawei Technologies Co. Ltd. Department of Production Automation Development China Chinese Academy of Sciences Institute of Automation China

ISBN: (纸本)9781728163277

Deploying deep learning models to new environments is very challenging. Domain adaptation (DA) is a promising paradigm to solve the problem by collecting and adapting to unlabeled data in new environments. Though research efforts have led to steady performance improvement over the past decade, DA algorithms are still hard to deploy, as training on unlabeled new data makes tuning difficult and not feasible for inference-only devices. To make DA practical, in this paper we study a new problem named Test-time Training-Free Domain Adaptation (TTDA), where trained models must adapt to a single input (mimicking the test-time scenario) without training. By exploiting spatial activation that was previously overlooked and simply averaged out, we propose a simple method based on Feature Statistics Transformation (FST) on-the-fly for each test example. The proposed algorithm is tested in the TTDA setting on two standard DA benchmarks. Surprisingly, it surpasses or performs on par with state-of-the-art DA methods, even though they require additional training. We envision that this training-free paradigm has the potential to bring DA to embedded devices and would be of interest to audience of community. © 2023 IEEE.

关键词： Deep learning

MDNet: Learning Monaural Speech Enhancement from Deep Prior Gradient

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Li, Andong Zheng, Chengshi Zhang, Ziyang Li, Xiaodong Key Laboratory Of Noise And Vibration Research Institute Of Acoustics Chinese Academy Of Sciences Beijing China University Of Chinese Academy Of Sciences Beijing China Advanced Computing And Storage Lab Huawei Technologies Co. Ltd. Beijing China

While traditional statistical signal processing model-based methods can derive the optimal estimators relying on specific statistical assumptions, current learning-based methods further promote the performance upper bound via deep neural networks but at the expense of high encapsulation and lack adequate interpretability. Standing upon the intersection between traditional model-based methods and learning-based methods, we propose a model-driven approach based on the maximum a posteriori (MAP) framework, termed as MDNet, for single-channel speech enhancement. Specifically, the original problem is formulated into the joint posterior estimation w.r.t. speech and noise components. Different from the manual assumption toward the prior terms, we propose to model the prior distribution via networks and thus can learn from training data. The framework takes the unfolding structure and in each step, the target parameters can be progressively estimated through explicit gradient descent operations. Besides, another network serves as the fusion module to further refine the previous speech estimation. The experiments are conducted on the WSJ0-SI84 and Interspeech2020 DNS-Challenge datasets, and quantitative results show that the proposed approach outshines previous state-of-the-art baselines. Copyright © 2022, The Authors. All rights reserved.

关键词： Deep neural networks

Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs

学校读者我要写书评

暂无评论

arXiv 2021年

作者： You, Kaichao Liu, Yong Zhang, Ziyang Wang, Jianmin Jordan, Michael I. Long, Mingsheng School of Software BNRist Tsinghua University Beijing100084 China Advanced Computing and Storage Lab Huawei Technologies Co. Ltd China Division of Computer Science Department of Statistics UC Berkeley CA94720-1776 United States

Model hubs with many pre-trained models (PTMs) have become a cornerstone of deep learning. Although built at a high cost, they remain under-exploited-practitioners usually pick one PTM from the provided model hub by popularity and then fine-tune the PTM to solve the target task. This naïve but common practice poses two obstacles to full exploitation of pre-trained model hubs: first, the PTM selection by popularity has no optimality guarantee, and second, only one PTM is used while the remaining PTMs are ignored. An alternative might be to consider all possible combinations of PTMs and extensively fine-tune each combination, but this would not only be prohibitive computationally but may also lead to statistical over-fitting. In this paper, we propose a new paradigm for exploiting model hubs that is intermediate between these extremes. The paradigm is characterized by two aspects: (1) We use an evidence maximization procedure to estimate the maximum value of label evidence given features extracted by pre-trained models. This procedure can rank all the PTMs in a model hub for various types of PTMs and tasks before fine-tuning. (2) The best ranked PTM can either be fine-tuned and deployed if we have no preference for the model's architecture or the target PTM can be tuned by the top K ranked PTMs via a Bayesian procedure that we propose. This procedure, which we refer to as B-Tuning, not only improves upon specialized methods designed for tuning homogeneous PTMs, but also applies to the challenging problem of tuning heterogeneous PTMs where it yields a new level of benchmark performance. © 2021, CC BY-NC-ND.

关键词： Benchmarking

TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation

学校读者我要写书评

暂无评论

arXiv 2022年

作者： He, Weihua You, Kaichao Qiao, Zhendong Jia, Xu Zhang, Ziyang Wang, Wenhui Lu, Huchuan Wang, Yaoyuan Liao, Jianxing Department of Precision Instrument Tsinghua University China Advanced Computing and Storage Lab. Huawei Technologies Co. Ltd. China School of Artificial Intelligence Dalian University of Technology China School of Software Tsinghua University China Peng Cheng Laboratory China

Recording fast motion in a high FPS (frame-per-second) requires expensive high-speed cameras. As an alternative, interpolating low-FPS videos from commodity cameras has attracted significant attention. If only low-FPS videos are available, motion assumptions (linear or quadratic) are necessary to infer intermediate frames, which fail to model complex motions. Event camera, a new camera with pixels producing events of brightness change at the temporal resolution of µs (10-6 second), is a game-changing device to enable video interpolation at the presence of arbitrarily complex motion. Since event camera is a novel sensor, its potential has not been fulfilled due to the lack of processing algorithms. The pioneering work Time Lens introduced event cameras to video interpolation by designing optical devices to collect a large amount of paired training data of high-speed frames and events, which is too costly to scale. To fully unlock the potential of event cameras, this paper proposes a novel TimeReplayer algorithm to interpolate videos captured by commodity cameras with events. It is trained in an unsupervised cycle-consistent style, canceling the necessity of high-speed training data and bringing the additional ability of video extrapolation. Its state-of-the-art results and demo videos in supplementary reveal the promising future of event-based vision. © 2022, CC BY.

关键词： High speed cameras

Continuously Controllable Facial Expression Editing in Talking Face Videos

学校读者我要写书评

暂无评论

arXiv 2022年

作者： Sun, Zhiyao Wen, Yu-Hui Lv, Tian Sun, Yanan Zhang, Ziyang Wang, Yaoyuan Liu, Yong-Jin BNRist the Department of Computer Science and Technology MOE-Key Laboratory of Pervasive Computing Tsinghua University Beijing100084 China Beijing Key Laboratory of Traffic Data Analysis and Mining School of Computer and Information Technology Beijing Jiaotong University Beijing100044 China Advanced Computing and Storage Lab Huawei Technologies Company Ltd. Shenzhen518129 China

Recently audio-driven talking face video generation has attracted considerable attention. However, very few researches address the issue of emotional editing of these talking face videos with continuously controllable expressions, which is a strong demand in the industry. The challenge is that speech-related expressions and emotion-related expressions are often highly coupled. Meanwhile, traditional image-to-image translation methods cannot work well in our application due to the coupling of expressions with other attributes such as poses, i.e., translating the expression of the character in each frame may simultaneously change the head pose due to the bias of the training data distribution. In this paper, we propose a high-quality facial expression editing method for talking face videos, allowing the user to control the target emotion in the edited video continuously. We present a new perspective for this task as a special case of motion information editing, where we use a 3DMM to capture major facial movements and an associated texture map modeled by a StyleGAN to capture appearance details. Both representations (3DMM and texture map) contain emotional information and can be continuously modified by neural networks and easily smoothed by averaging in coefficient/latent spaces, making our method simple yet effective. We also introduce a mouth shape preservation loss to control the trade-off between lip synchronization and the degree of exaggeration of the edited expression. Extensive experiments and a user study show that our method achieves state-of-the-art performance across various evaluation criteria. Copyright © 2022, The Authors. All rights reserved.

关键词： Quality control

Koopmanlab: machine learning for solving complex physics equations

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Xiong, Wei Ma, Muyuan Huang, Xiaomeng Zhang, Ziyang Sun, Pei Tian, Yang Department of Earth System Science Tsinghua University Beijing100084 China Laboratory of Advanced Computing and Storage Central Research Institute 2012 Laboratories Huawei Technologies Co. Ltd. Beijing100084 China Department of Psychology & Tsinghua Brain and Intelligence Lab Tsinghua University Beijing100084 China

Numerous physics theories are rooted in partial differential equations (PDEs). However, the increasingly intricate physics equations, especially those that lack analytic solutions or closed forms, have impeded the further development of physics. Computationally solving PDEs by classic numerical approaches suffers from the trade-off between accuracy and efficiency and is not applicable to the empirical data generated by unknown latent PDEs. To overcome this challenge, we present Koopmanlab, an efficient module of the Koopman neural operator family, for learning PDEs without analytic solutions or closed forms. Our module consists of multiple variants of the Koopman neural operator (KNO), a kind of mesh-independent neural-network-based PDE solvers developed following dynamic system theory. The compact variants of KNO can accurately solve PDEs with small model sizes while the large variants of KNO are more competitive in predicting highly complicated dynamic systems govern by unknown, high-dimensional, and non-linear PDEs. All variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman–Burgers equation in fluid mechanics) and ERA5 (i.e., one of the largest high-resolution global-scale climate data sets in earth physics). These demonstrations suggest the potential of Koopmanlab to be a fundamental tool in diverse physics studies related to equations or dynamic systems. © 2023, CC BY.

关键词： Dynamics