Federated learning is widely accepted as a privacy-preserving paradigm for training a shared global model across multiple client devices in a collaborative fashion. However, in practice, the significantly limited comp...
详细信息
Federated learning is widely accepted as a privacy-preserving paradigm for training a shared global model across multiple client devices in a collaborative fashion. However, in practice, the significantly limited computational power on client devices has been a major barrier when we wish to train large models with potentially hundreds of millions of parameters. In this paper, we propose a new architecture, referred to as Infocomm, that incorporates locally supervised learning in federated learning. With locally supervised learning, the disadvantages of split learning can be avoided by using a more flexible way to offload training from resource constrained clients to a more capable server. Infocomm enables parallel training of different modules of the neural network in both the server and clients in a gradient-isolated fashion. The efficacy in reducing both training time and communication time is supported by our theoretical analysis and empirical results. In the scenario involving larger models and fewer available local data, Infocomm has been observed to reduce the elapsed time per round by over 37% without sacrificing accuracy compared to both conventional federated learning or directly combining federated learning and split learning, which showcases the advantages of Infocomm under power-constrained IoT scenarios. IEEE
This article formulates interactive adversarial differential graphical games for synchronization control of multiagent systems(MASs) subject to adversarial inputs interacting with the systems through topology communic...
详细信息
This article formulates interactive adversarial differential graphical games for synchronization control of multiagent systems(MASs) subject to adversarial inputs interacting with the systems through topology communications. Local control and interactive adversarial inputs affect each agent's local synchronization error via local networks. The distributed global Nash equilibrium(NE) solutions are guaranteed in the games by solving the optimal control input of each agent and the worst-case adversarial input based solely on local states and communications. The asymptotic stability of the local synchronization error dynamics and the NE are guaranteed. Furthermore, the authors devise a data-driven online reinforcement learning(RL) algorithm that only computes the distributed Nash control online using system trajectory data, eliminating the need for explicit system dynamics. A simulation-based example validates the game and algorithm.
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text ***,BERT’s ...
详细信息
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text ***,BERT’s size and computational demands limit its practicality,especially in resource-constrained *** research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization *** Bengali being the sixth most spoken language globally,NLP research in this area is *** approach addresses this gap by creating an efficient BERT-based model for Bengali *** have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory *** best results demonstrate significant improvements in both speed and *** instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 *** results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.
The perception in most existing vision-based reinforcement learning(RL) models for robotic manipulation relies heavily on static third-person or hand-mounted first-person cameras. In scenarios with occlusions and limi...
详细信息
The perception in most existing vision-based reinforcement learning(RL) models for robotic manipulation relies heavily on static third-person or hand-mounted first-person cameras. In scenarios with occlusions and limited maneuvering space, these carefully positioned cameras often struggle to provide effective visual observations during manipulation. Taking inspiration from human capabilities, we introduce a novel RL-based dual-arm active visual-guided manipulation model(DAVMM), which simultaneously infers “eye” actions and “hand” actions for two separate robotic arms(referred to as the vision-arm and the worker-arm) based on current observations, empowering the robot with the ability to actively perceive and interact with its environment. To handle the extensive redundant observation-action space, we propose a decouplable target-centric reward paradigm to offer stable guidance for the training process. For making fine-grained manipulation action decisions, alongside a global scene image encoder, we utilize an independent encoder to extract local target texture features,enabling the simultaneous acquisition of both global and detailed local information. Additionally, we employ residual-RL and curriculum learning techniques to further enhance our model's sample efficiency and training stability. We conducted comparative experiments and analyses of DAVMM against a set of strong baselines on three occluded and narrow-space manipulation tasks. DAVMM notably improves the success rates across all manipulation tasks and showcases rapid learning capabilities.
The counterflow burner is a combustion device used for research on *** utilizing deep convolutional models to identify the combustion state of a counter flow burner through visible flame images,it facilitates the opti...
详细信息
The counterflow burner is a combustion device used for research on *** utilizing deep convolutional models to identify the combustion state of a counter flow burner through visible flame images,it facilitates the optimization of the combustion process and enhances combustion *** existing deep convolutional models,InceptionNeXt is a deep learning architecture that integrates the ideas of the Inception series and *** has garnered significant attention for its computational efficiency,remarkable model accuracy,and exceptional feature extraction ***,since this model still has limitations in the combustion state recognition task,we propose a Triple-Scale Multi-Stage InceptionNeXt(TSMS-InceptionNeXt)combustion state recognitionmethod based on feature extraction ***,to address the InceptionNeXt model’s limited ability to capture dynamic features in flame images,we introduce Triplet Attention,which applies attention to the width,height,and Red Green Blue(RGB)dimensions of the flame images to enhance its ability to model dynamic ***,to address the issue of key information loss in the Inception deep convolution layers,we propose a Similarity-based Feature Concentration(SimC)mechanism to enhance the model’s capability to concentrate on critical ***,to address the insufficient receptive field of the model,we propose a Multi-Scale Dilated Channel Parallel Integration(MDCPI)mechanism to enhance the model’s ability to extract multi-scale contextual ***,to address the issue of the model’s Multi-Layer Perceptron Head(MlpHead)neglecting channel interactions,we propose a Channel Shuffle-Guided Channel-Spatial Attention(ShuffleCS)mechanism,which integrates information from different channels to further enhance the representational power of the input *** validate the effectiveness of the method,experiments are conducted on the counterflow burner flame visible light image datase
Direct training of Spiking Neural Networks (SNNs) is a challenging task because of their inherent temporality. Added to it, the vanilla Back-propagation based methods are not applicable either, due to the non-differen...
详细信息
Direct training of Spiking Neural Networks (SNNs) is a challenging task because of their inherent temporality. Added to it, the vanilla Back-propagation based methods are not applicable either, due to the non-differentiability of the spikes in SNNs. Surrogate-Derivative based methods with Backpropagation Through Time (BPTT) address these direct training challenges quite well;however, such methods are not neuromorphic-hardware friendly for the On-chip training of SNNs. Recently formalized Three-Factor based Rules (TFR) for direct local-training of SNNs are neuromorphic-hardware friendly;however, they do not effectively leverage the depth of the SNN architectures (we show it empirically here), thus, are limited. In this work, we present an improved version of a conventional three-factor rule, for local learning in SNNs which effectively leverages depth - in the context of learning features hierarchically. Taking inspiration from the Back-propagation algorithm, we theoretically derive our improved, local, three-factor based learning method, named DALTON (Deep LocAl Learning via local WeighTs and SurrOgate-Derivative TraNsfer), which employs weights and surrogate-derivative transfer from the local layers. Along the lines of TFR, our proposed method DALTON is also amenable to the neuromorphic-hardware implementation. Through extensive experiments on static (MNIST, FMNIST, & CIFAR10) and event-based (N-MNIST, DVS128-Gesture, & DVSCIFAR10) datasets, we show that our proposed local-learning method DALTON makes effective use of the depth in Convolutional SNNs, compared to the vanilla TFR implementation. IEEE
Video captioning is the process of automatically generating natural language descriptions of video content. Historically, most video captioning methods have relied on extending Sequence-to-Sequence (Seq2Seq) models. H...
详细信息
Large number of antennas and higher bandwidth usage in massive multiple-input-multipleoutput(MIMO)systems create immense burden on receiver in terms of higher power *** power consumption at the receiver radio frequenc...
详细信息
Large number of antennas and higher bandwidth usage in massive multiple-input-multipleoutput(MIMO)systems create immense burden on receiver in terms of higher power *** power consumption at the receiver radio frequency(RF)circuits can be significantly reduced by the application of analog-to-digital converter(ADC)of low *** this paper we investigate bandwidth efficiency(BE)of massive MIMO with perfect channel state information(CSI)by applying low resolution ADCs with Rician *** start our analysis by deriving the additive quantization noise model,which helps to understand the effects of ADC resolution on BE by keeping the power constraint at the receiver in *** also investigate deeply the effects of using higher bit rates and the number of BS antennas on bandwidth efficiency(BE)of the *** emphasize that good bandwidth efficiency can be achieved by even using low resolution ADC by using regularized zero-forcing(RZF)combining *** also provide a generic analysis of energy efficiency(EE)with different options of bits by calculating the energy efficiencies(EE)using the achievable *** emphasize that satisfactory BE can be achieved by even using low-resolution ADC/DAC in massive MIMO.
In this paper we show a polar coding scheme for the deletion channel with a probability of error that decays roughly like 2-√Λ, where Λ is the length of the codeword. That is, the same decay rate as that of seminal...
详细信息
This study proposes a real-time integrated framework for LiDAR-based object tracking in autonomous driving environments. Advancements in LiDAR sensors are increasing point cloud data collection, leading to a demand fo...
详细信息
暂无评论