Training Deep Learning (DL) models are becoming more time-consuming, thus interruptions to the training processes are inevitable. Existing fault-tolerant work adopted checkpoint/recovery mechanism from traditional HPC...
详细信息
Training Deep Learning (DL) models are becoming more time-consuming, thus interruptions to the training processes are inevitable. Existing fault-tolerant work adopted checkpoint/recovery mechanism from traditional HPC platforms that makes periodical persistent copies of model states to save execution progress. These checkpoints have fixed time intervals that they are evenly placed across the training. We can obtain an optimal checkpointing interval for an HPC job with the precondition that the progress of a job is proportional to its execution time. Unfortunately, it is not the case in DL model training where a DL training job yields diminishing returns across its lifetime. It makes the early progress of a DL training job more valuable than the later ones. Even placement of checkpoints would either increase the risks in the early stages or waste resources overprotecting the latter stages. Meanwhile, the issue can get amplified for exploratory training jobs, where early terminations are common. Moreover, in data parallelism, the state-of-art quality-driven scheduling strategies allocate more resources for the early stages of a job than the later ones to accelerate the training progress which further amplifies the issue. This paper introduces a novel checkpointing interval problem for exploratory DL training jobs based on model convergence progress. We present COCI, an approach to compute optimal checkpointing configuration for a DL training job, minimizing the fault-tolerant overhead, including checkpointing cost and recovery cost. We implement COCI based on state-of-art iteration-level checkpointing mechanism, as a pluggable module compatible with PyTorch. COCI requires no extra user input. We conduct comprehensive evaluations with real DL application setups. The experimental results show that COCI reduces up to 40.18% fault-tolerant overhead compared to existing state-of-the-art DL fault-tolerant methods in serial scenario, 57.26% in data parallel scenario and 63.8
This paper is concerned with non-radiating elastic sources in inhomogeneous elastic media. We demonstrate that the value of non-radiating elastic sources must vanish at convex corners of their support, provided the so...
详细信息
In this paper, we design a hybrid (semi-direct) approach to simultaneous localization and mapping (SLAM) for monocular cameras and apply it to augmented reality (AR) for monocular cameras. We combine the advantagesof ...
详细信息
As a popular social code hosting platform, GitHub encourages developers to discuss and leave opinions on issues for better repository development and closer team collaboration. However, popular issues can be bloated o...
详细信息
As a popular social code hosting platform, GitHub encourages developers to discuss and leave opinions on issues for better repository development and closer team collaboration. However, popular issues can be bloated over the time, in particular, the linear format of GitHub issue discussions makes it difficult for developers to organize and extract useful information. For a better understanding of GitHub issue discussions, we first conduct an empirical study. Among the 14 most-starred repositories, we notice that 16,740 issues contain more than 10 comments. Then, more commented issue discussions refer to additional repository contributions and draw more developer attention. Hence, we narrow our perspective to the issue discussions with more than 10 comments. For 50.29% of these popular discussions, the topics of content are subject to change explicitly, and more than 36% of consecutive comment pairs do not host response relationships. In addition, just 40% of the comments on GitHub use the @ or quoting functions when commenting on other people's comments, but these functions are not only used for responding, but also for referencing, informing others, etc. Based on these results, these popular discussions suffer from intertwined content of various topics and ambiguous response linkage. To mitigate the situation, we propose a new approach IRA to automatically re-organize GitHub issue discussions, aiming at converting an issue discussion with the linear structure into a discussion tree with key information. The experimental results show that our approach outperforms other baselines, and achieves an average improvement of 19.97%, 14,25% on metrics of ACC and F1-score in the task of predicting response relationships, as well as gets 15.78%, 51.72%, 26.92%, 21.03%, 22.08%, 25.59% improvement in terms of parent accuracy, Variation of Information, One-to-One Overlap, and all Exact Match metrics in the re-organizing task. To investigate human perspectives on our re-organized
In recent years, the unlabeled augmented reality system has been gradually applied to various mobile devices, among which stable, accurate, and fast registration is the key to realizing this function. For this techniq...
详细信息
The Swin transformer has recently attracted attention in medical image analysis due to its computational efficiency and long-range modeling capability. Owing to these properties, the Swin Transformer is suitable for e...
详细信息
Lymphoma is a malignant tumor, and diffuse large B-cell lymphoma (DLBCL) is the most common type of non-Hodgkin's lymphoma. Due to its biological characteristics, surgical treatment is difficult. The main treatmen...
详细信息
Mobile CrowdSensing (MCS) is a data sensing paradigm that recruits users carrying mobile terminals to collect data. As its variant, Sparse MCS has been further proposed for large-scale and fine-grained sensing task wi...
Mobile CrowdSensing (MCS) is a data sensing paradigm that recruits users carrying mobile terminals to collect data. As its variant, Sparse MCS has been further proposed for large-scale and fine-grained sensing task with the advantage of collecting only a few data to infer unsensed data. However, in many real-world scenarios, such as early prevention of epidemic, people are interested in not only the data at the current, but also in the future or even long-term future, and the latter may be more important. Long-term prediction not only reduces sensing cost, but also identifies trends or other characteristics of the data. In this paper, we propose a spatiotemporal model based on Transformer to infer and predict the data with sparse sensed data by utilizing spatiotemporal relationships. We design a spatiotemporal feature embedding to embed the prior spatiotemporal information of sensing map into the model to guide model learning. Moreover, we also design a novel multi-head spatiotemporal attention mechanism to dynamically capture spatiotemporal relationships among data. Extensive experiments have been conducted on three types of typical urban sensing tasks, which verify the effectiveness of our proposed algorithms in improving the inference and long-term prediction accuracy with the sparse sensed data.
Existing fake news detection methods aim to classify a piece of news as true or false and provide veracity explanations, achieving remarkable performances. However, they often tailor automated solutions on manual fact...
详细信息
Video saliency prediction is an important task in the field of computer vision. Most of the existing video saliency prediction methods only focus on image information, and the audio information is often ignored. This ...
Video saliency prediction is an important task in the field of computer vision. Most of the existing video saliency prediction methods only focus on image information, and the audio information is often ignored. This leads to an incomplete perception mode, which makes it difficult to achieve optimal performance. SENet is an excellent attention mechanism-based network. It significantly enhances the performance of 2D convolutional networks. However, whether the 3D convolutional network can be applied to this attention mechanism network remains to be studied. In order to solve the above problems, we propose a saliency prediction network for audio-visual fusion to extract and predict various information in videos. At the same time, we improve the traditional SENet to make it applicable in 3D convolutional neural networks and discuss its role. Compared with the state-of-the-art methods, our model has strong competitiveness in multiple data sets.
暂无评论