Since its creation, the ImageNet-1k benchmark set has played a significant role as a benchmark for ascertaining the accuracy of different deep neural net (DNN) models on the image classification problem. Moreover, in ...
详细信息
Since its creation, the ImageNet-1k benchmark set has played a significant role as a benchmark for ascertaining the accuracy of different deep neural net (DNN) models on the image classification problem. Moreover, in recent years it has also served as the principal benchmark for assessing different approaches to DNN training. Finishing a 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIA M40 GPU takes 14 days. This training requires 10(18) single precision operations in total. On the other hand, the world's current fastest supercomputer can finish 3 x 10(17) single precision operations per second (according to the Nov 2018 Top 500 results). If we can make full use of the computing capability of the fastest supercomputer, we should be able to finish the training in several seconds. Over the last two years, researchers have focused on closing this significant performance gap through scaling DNN training to larger numbers of processors. Most successful approaches to scaling ImageNet training have used the synchronous mini-batch stochastic gradient descent (SGD). However, to scale synchronous SGD one must also increase the batch size used in each iteration. Thus, for many researchers, the focus on scaling DNN training has translated into a focus on developing training algorithms that enable increasing the batch size in data-parallel synchronous SGD without losing accuracy over a fixed number of epochs. In this paper, we investigate supercomputers' capability of speeding up DNN training. Our approach is to use a large batch size, powered by the Layer-wise Adaptive Rate Scaling (LARS) algorithm, for efficient usage of massive computing resources. Our approach is generic, as we empirically evaluate the effectiveness on five neural networks: AlexNet, AlexNet-BN, GNMT, ResNet-50, and ResNet-50-v2 trained with large datasets while preserving the state-of-the-art test accuracy. Compared to the baseline of a previous study from Goyal et al. [1] , our approach shows higher
In recent years, thousands or even hundreds of thousands players interact with each other in the MMOG (Massively Multi-player Online Game). Therefore MMOG servers have the problem with scalability. To overcome this pr...
详细信息
ISBN:
(纸本)9783540730101
In recent years, thousands or even hundreds of thousands players interact with each other in the MMOG (Massively Multi-player Online Game). Therefore MMOG servers have the problem with scalability. To overcome this problem, we propose a new method for MMOG distributed server, denoted as 2Layer-Cell method. 2Layer-Cell method constructed with Upper-Layer and Down-Layer. The Upper-Layer includes important aggregated information of game objects such as virtual space, users, monster, etc. And the Down-layer includes real data of game objects. This paper makes the following contributions. First, it captures these problems of high storage cost and slow processing time for previous methods. Second, it proposes parallelprocessing strategies that aim to reduce process time. Third, it proposes an efficient partitioning algorithm for distributed servers. Our experiment results show that our method is better scalable than existing methods.
暂无评论