Distributed Learning is pivotal for training extensive deep neural networks across multiple nodes, leveraging parallel computation to hasten the learning process. However, it faces challenges in communication efficien...
详细信息
ISBN:
(纸本)9798350359329;9798350359312
Distributed Learning is pivotal for training extensive deep neural networks across multiple nodes, leveraging parallel computation to hasten the learning process. However, it faces challenges in communication efficiency and resource utilization. Asynchronous Quantized stochastic Gradient Descent (AQSGD) addresses communication bottlenecks by updating quantized model parameters, thereby expediting training and reducing bandwidth usage. Yet, current stochastic quantization methods may inadequately capture varied gradient distributions, leading to accumulated biases and amplified quantization errors. These issues are amplified as the number of distributed nodes grows. This study proposes a novel stochasticquantization with Multivariate Gaussians (SQMG) for distributed machine learning. SQMG employs a multivariate Gaussian model to represent the relationships in the gradient updates for quantization. The SQMG approach allows for constructing an optimized quantization target space, coupled with an iterative mapping scheme that effectively projects the parameters onto this space while minimizing quantization errors. Experiments on DNN and CNN models for MNIST and CIFAR-10 show that SQMG increases accuracy by 0.92% and 1.54% for DNN and CNN models, respectively, compared to conventional quantizationmethods. The results validate SQMG's ability to reduce quantization errors and improve model accuracy in distributed learning systems.
The exact and analytic Green functions for spinning relativistic particles in interaction with a gravitational plane wave field are obtained within the stochastic quantization method of Parisi and Wu. We have separate...
详细信息
The exact and analytic Green functions for spinning relativistic particles in interaction with a gravitational plane wave field are obtained within the stochastic quantization method of Parisi and Wu. We have separated the classical calculations from those related to the quantum fluctuations. The problem has been solved by using a perturbative treatment via the Langevin equation relying on phase and configuration spaces formulation.
We present a class of nonlinear Schrodinger equations (NLSEs) describing, in the mean field approximation, systems of interacting particles. This class of NLSEs is obtained generalizing expediently the approach propos...
详细信息
We present a class of nonlinear Schrodinger equations (NLSEs) describing, in the mean field approximation, systems of interacting particles. This class of NLSEs is obtained generalizing expediently the approach proposed in [G. K., Phys. Rev. A 55, 941 (1997)], where a classical system obeying to an exclusion-inclusion principle is quantized using the Nelson stochasticquantization. The new class of NLSEs is obtained starting from the most general nonlinear classical kinetics compatible with a constant diffusion coefficient D = h/2m. Finally, in the case of s-stationary states, we propose a transformation which linearizes the NLSEs here proposed.
暂无评论