This paper studies the distributed flexible flow shop scheduling problem (DFFSP), where the transportation time between different factories needs to be considered and each machine has a different startup time. A mixed...
详细信息
We demonstrate that efficient meta-learning can be achieved via end-to-end training of deep neuralnetworks with memory distributed across layers. The persistent state of this memory assumes the entire burden of guidi...
详细信息
ISBN:
(纸本)9781713845393
We demonstrate that efficient meta-learning can be achieved via end-to-end training of deep neuralnetworks with memory distributed across layers. The persistent state of this memory assumes the entire burden of guiding task adaptation. Moreover, its distributed nature is instrumental in orchestrating adaptation. Ablation experiments demonstrate that providing relevant feedback to memory units distributed across the depth of the network enables them to guide adaptation throughout the entire network. Our results show that this is a successful strategy for simplifying meta-learning - often cast as a bi-level optimization problem - to standard end-to-end training, while outperforming gradient-based, prototype-based, and other memory-based meta-learning strategies. Additionally, our adaptation strategy naturally handles online learning scenarios with a significant delay between observing a sample and its corresponding label - a setting in which other approaches struggle. Adaptation via distributed memory is effective across a wide range of learning tasks, ranging from classification to online few-shot semantic segmentation.
A key step in medical image-based diagnosis is image segmentation. A common use case for medical image segmentation is the identification of single structures of an elliptical shape. Most organs like the heart and kid...
详细信息
A key step in medical image-based diagnosis is image segmentation. A common use case for medical image segmentation is the identification of single structures of an elliptical shape. Most organs like the heart and kidneys fall into this category, as well as skin lesions, polyps, and other types of abnormalities. neuralnetworks have dramatically improved medical image segmentation results, but still require large amounts of training data and long training times to converge. In this paper, we propose a general way to improve neuralnetwork segmentation performance and data efficiency on medical imaging segmentation tasks where the goal is to segment a single roughly elliptically distributed object. We propose training a neuralnetwork on polar transformations of the original dataset, such that the polar origin for the transformation is the center point of the object. This results in a reduction of dimensionality as well as a separation of segmentation and localization tasks, allowing the network to more easily converge. Additionally, we propose two different approaches to obtaining an optimal polar origin: (1) estimation via a segmentation trained on non-polar images and (2) estimation via a model trained to predict the optimal origin. We evaluate our method on the tasks of liver, polyp, skin lesion, and epicardial adipose tissue segmentation. We show that our method produces state-of-the-art results for lesion, liver, and polyp segmentation and performs better than most common neuralnetwork architectures for biomedical image segmentation. Additionally, when used as a pre-processing step, our method generally improves data efficiency across datasets and neuralnetwork architectures.
Nowadays, there are a large number of bilingual translated texts on the Internet. It is a crucial problem to build a practical bilingual corpus through the processing of translated texts. Based on NN(neuralnetwork) e...
详细信息
A hardware-based computational-efficient biophysical model of a neuralnetwork that considers the inferior olivary nucleus is presented. The implementation uses a multi-FPGA system based on the PlasticNet interconnect...
详细信息
ISBN:
(纸本)9798350306132
A hardware-based computational-efficient biophysical model of a neuralnetwork that considers the inferior olivary nucleus is presented. The implementation uses a multi-FPGA system based on the PlasticNet interconnection framework, which enables the implementation of a cost-effective and scalable model able to handle over ten thousand neurons with five FPGA evaluation boards. With the same setup, it was possible to simulate one thousand neurons in real time.
This paper describes the methods of organization of the data center protection strategy, presented as a networkdistributed infrastructure, against potential external threats. This work indicates the advantages of usi...
详细信息
Due to the complex and dynamic service demands involved in digital twin services, traditional resource matching methods often fail to meet the requirements. To address this challenge, we leverage Graph neuralnetworks...
详细信息
We study the asynchronous stochastic gradient descent algorithm for distributed training over n workers which have varying computation and communication frequency over time. In this algorithm, workers compute stochast...
详细信息
ISBN:
(纸本)9781713871088
We study the asynchronous stochastic gradient descent algorithm for distributed training over n workers which have varying computation and communication frequency over time. In this algorithm, workers compute stochastic gradients in parallel at their own pace and return those to the server without any synchronization. Existing convergence rates for this algorithm for non-convex smooth objectives depend on the maximum gradient delay tau(max) and show that an epsilon-stationary point is reached after O (sigma(2) epsilon(-2) + tau(max) epsilon(-1) ) iterations, where sigma denotes the variance of stochastic gradients. In this work we obtain (i) a tighter convergence rate of O( sigma(2) epsilon(-2) + root tau(max) tau(avg) epsilon(-1) ) without any change in the algorithm, where tau(avg) is the average delay, which can be significantly smaller than tau(max). We also provide (ii) a simple delay-adaptive learning rate scheme, under which asynchronous SGD achieves a convergence rate of O (sigma(2) epsilon(-2) + tau(avg) epsilon(-1) ), and does not require any extra hyperparameter tuning nor extra communications. Our result allows to show for the first time that asynchronous SGD is always faster than mini-batch SGD. In addition, (iii) we consider the case of heterogeneous functions motivated by federated learning applications and improve the convergence rate by proving a weaker dependence on the maximum delay compared to prior works. In particular, we show that the heterogeneity term in convergence rate is only affected by the average delay within each worker.
Newly developed machine learning technology is promising to profoundly impact high-performance computing, with the potential to significantly accelerate scientific discoveries. However, scientific machine learning per...
详细信息
ISBN:
(纸本)9781665481069
Newly developed machine learning technology is promising to profoundly impact high-performance computing, with the potential to significantly accelerate scientific discoveries. However, scientific machine learning performance is often constrained by data movement overheads, particularly on existing and emerging hardware-accelerated systems. In this work, we focus on optimizing the data movement across storage and memory systems, by developing domain-specific data encoder/decoders. These plugins have the dual benefit of significantly reducing communication while enabling efficient decoding on the accelerated hardware. We explore detailed performance analysis for two important scientific learning workloads from cosmology and climate analytics, CosmoFlow and DeepCAM, on the GPU-enabled Summit and Cori supercomputers. Results demonstrate that our optimizations can significantly improve overall performance by up to 10x compared with the default baseline, while preserving convergence behavior. Overall, this methodology can be applied to various machine learning domains and emerging AI technologies.
This paper proposes an end-to-end learning approach for coherent optical orthogonal frequency-division multiplexing (CO-OFDM) fiber communication transmission to mitigate laser phase noise. The approach is based on th...
详细信息
暂无评论