Federated learning is widely accepted as a privacy-preserving paradigm for training a shared global model across multiple client devices in a collaborative fashion. However, in practice, the significantly limited comp...
详细信息
Federated learning is widely accepted as a privacy-preserving paradigm for training a shared global model across multiple client devices in a collaborative fashion. However, in practice, the significantly limited computational power on client devices has been a major barrier when we wish to train large models with potentially hundreds of millions of parameters. In this paper, we propose a new architecture, referred to as Infocomm, that incorporates locally supervised learning in federated learning. With locally supervised learning, the disadvantages of split learning can be avoided by using a more flexible way to offload training from resource constrained clients to a more capable server. Infocomm enables parallel training of different modules of the neural network in both the server and clients in a gradient-isolated fashion. The efficacy in reducing both training time and communication time is supported by our theoretical analysis and empirical results. In the scenario involving larger models and fewer available local data, Infocomm has been observed to reduce the elapsed time per round by over 37% without sacrificing accuracy compared to both conventional federated learning or directly combining federated learning and split learning, which showcases the advantages of Infocomm under power-constrained IoT scenarios. IEEE
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text ***,BERT’s ...
详细信息
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text ***,BERT’s size and computational demands limit its practicality,especially in resource-constrained *** research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization *** Bengali being the sixth most spoken language globally,NLP research in this area is *** approach addresses this gap by creating an efficient BERT-based model for Bengali *** have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory *** best results demonstrate significant improvements in both speed and *** instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 *** results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.
The perception in most existing vision-based reinforcement learning(RL) models for robotic manipulation relies heavily on static third-person or hand-mounted first-person cameras. In scenarios with occlusions and limi...
详细信息
The perception in most existing vision-based reinforcement learning(RL) models for robotic manipulation relies heavily on static third-person or hand-mounted first-person cameras. In scenarios with occlusions and limited maneuvering space, these carefully positioned cameras often struggle to provide effective visual observations during manipulation. Taking inspiration from human capabilities, we introduce a novel RL-based dual-arm active visual-guided manipulation model(DAVMM), which simultaneously infers “eye” actions and “hand” actions for two separate robotic arms(referred to as the vision-arm and the worker-arm) based on current observations, empowering the robot with the ability to actively perceive and interact with its environment. To handle the extensive redundant observation-action space, we propose a decouplable target-centric reward paradigm to offer stable guidance for the training process. For making fine-grained manipulation action decisions, alongside a global scene image encoder, we utilize an independent encoder to extract local target texture features,enabling the simultaneous acquisition of both global and detailed local information. Additionally, we employ residual-RL and curriculum learning techniques to further enhance our model's sample efficiency and training stability. We conducted comparative experiments and analyses of DAVMM against a set of strong baselines on three occluded and narrow-space manipulation tasks. DAVMM notably improves the success rates across all manipulation tasks and showcases rapid learning capabilities.
In this paper we show a polar coding scheme for the deletion channel with a probability of error that decays roughly like 2-√Λ, where Λ is the length of the codeword. That is, the same decay rate as that of seminal...
详细信息
Video captioning is the process of automatically generating natural language descriptions of video content. Historically, most video captioning methods have relied on extending Sequence-to-Sequence (Seq2Seq) models. H...
详细信息
This study proposes a real-time integrated framework for LiDAR-based object tracking in autonomous driving environments. Advancements in LiDAR sensors are increasing point cloud data collection, leading to a demand fo...
详细信息
The global navigation satellite system-based technology has inherent limitations due to its reliance on radio signals. In contrast, visual localization operates independently of radio communication, presenting a viabl...
详细信息
Direct training of Spiking Neural Networks (SNNs) is a challenging task because of their inherent temporality. Added to it, the vanilla Back-propagation based methods are not applicable either, due to the non-differen...
详细信息
Direct training of Spiking Neural Networks (SNNs) is a challenging task because of their inherent temporality. Added to it, the vanilla Back-propagation based methods are not applicable either, due to the non-differentiability of the spikes in SNNs. Surrogate-Derivative based methods with Backpropagation Through Time (BPTT) address these direct training challenges quite well;however, such methods are not neuromorphic-hardware friendly for the On-chip training of SNNs. Recently formalized Three-Factor based Rules (TFR) for direct local-training of SNNs are neuromorphic-hardware friendly;however, they do not effectively leverage the depth of the SNN architectures (we show it empirically here), thus, are limited. In this work, we present an improved version of a conventional three-factor rule, for local learning in SNNs which effectively leverages depth - in the context of learning features hierarchically. Taking inspiration from the Back-propagation algorithm, we theoretically derive our improved, local, three-factor based learning method, named DALTON (Deep LocAl Learning via local WeighTs and SurrOgate-Derivative TraNsfer), which employs weights and surrogate-derivative transfer from the local layers. Along the lines of TFR, our proposed method DALTON is also amenable to the neuromorphic-hardware implementation. Through extensive experiments on static (MNIST, FMNIST, & CIFAR10) and event-based (N-MNIST, DVS128-Gesture, & DVSCIFAR10) datasets, we show that our proposed local-learning method DALTON makes effective use of the depth in Convolutional SNNs, compared to the vanilla TFR implementation. IEEE
Early attack detection is essential to ensure the security of complex networks,especially those in critical *** is particularly crucial in networks with multi-stage attacks,where multiple nodes are connected to extern...
详细信息
Early attack detection is essential to ensure the security of complex networks,especially those in critical *** is particularly crucial in networks with multi-stage attacks,where multiple nodes are connected to external sources,through which attacks could enter and quickly spread to other network *** attack graphs(BAGs)are powerful models for security risk assessment and mitigation in complex networks,which provide the probabilistic model of attackers’behavior and attack progression in the *** attack detection techniques developed for BAGs rely on the assumption that network compromises will be detected through routine monitoring,which is unrealistic given the ever-growing complexity of *** paper derives the optimal minimum mean square error(MMSE)attack detection and monitoring policy for the most general form of *** exploiting the structure of BAGs and their partial and imperfect monitoring capacity,the proposed detection policy achieves the MMSE optimality possible only for linear-Gaussian state space models using Kalman *** adaptive resource monitoring policy is also introduced for monitoring nodes if the expected predictive error exceeds a user-defined *** and efficient matrix-form computations of the proposed policies are provided,and their high performance is demonstrated in terms of the accuracy of attack detection and the most efficient use of available resources using synthetic Bayesian attack graphs with different topologies.
Converters rely on passive filtering as a crucial element due to the high-frequency operational characteristics of power *** filtering methods involve a dual inductor-capacitor(LC)cell or an inductor-capacitor-inducto...
详细信息
Converters rely on passive filtering as a crucial element due to the high-frequency operational characteristics of power *** filtering methods involve a dual inductor-capacitor(LC)cell or an inductor-capacitor-inductor(LCL)***,capacitors are susceptible to wear-out mechanisms and failure ***,the necessity for monitoring and regular replacement adds to an elevated cost of ownership for such *** utilization of an active output power filter can be used to diminish the dimensions of the LC filter and the electrolytic dc-link capacitor,even though the inclusion of capacitors remains an indispensable part of the *** paper introduces capacitorless solid-state power filter(SSPF)for single-phase dc-ac *** proposed configuration is capable of generating a sinusoidal ac voltage without relying on *** proposed filter,composed of a planar transformer and an H-bridge converter operating at high frequency,injects voltage harmonics to attain a sinusoidal output *** design parameters of the planar transformer are incorporated,and the impact of magnetizing and leakage inductances on the operation of the SSPF is *** analysis,supported by simulation and experimental results,are provided for a design example for a single-phase *** total harmonic distortion observed in the output voltage is well below the IEEE 519 *** system operation is experimentally tested under both steady-state and dynamic conditions.A comparison with existing technology is presented,demonstrating that the proposed topology reduces the passive components used for filtering.
暂无评论