With the continuous evolution of data-driven networks, storage optimization has emerged as a critical challenge for achieving efficient data management and real-time queries. Storage optimization, employing techniques...
详细信息
The paper proposes a classification of the main groups of measurement information compression methods. We carried out an analysis of existing gaps in the range of algorithms and approaches used in practice. The paper ...
详细信息
This paper analyzes various prediction methods of ECG signal compression and its hardware implementation. The comparative study of various prediction techniques of ECG compression both for single channel and multichan...
详细信息
Distributed sensor fusion requires the transmission of intermediate fusion results, consisting of point estimates and associated error covariance or information matrices. Bandwidth constraints necessitate data compres...
详细信息
ISBN:
(纸本)9798350371420;9781737749769
Distributed sensor fusion requires the transmission of intermediate fusion results, consisting of point estimates and associated error covariance or information matrices. Bandwidth constraints necessitate datacompression techniques for error covariance and information matrices, which typically dominate data volume. To ensure the safe use of the fusion results for decision-making, these techniques must be conservative, i.e., not lead to the compressed error covariance or information matrices underestimating the true estimate error. This work introduces a novel approach for the conservative compressed transmission of information matrices, that builds on a previous event-based method for covariance matrices. The proposed method allows the entire sensor fusion pipeline to operate in 'information space', facilitating efficient fusion operations without the need to compute corresponding covariance matrices. Contributions include an event-trigger for information matrices and a robust-optimization-based bounding mechanism ensuring conservativeness. The proposed approach is evaluated in the context of transmitting error information matrices generated by extended information filter SLAM to a receiver for further processing.
Error-bounded lossy compression turns more and more important for the data-moving intensive applications to deal with big datasets efficiently in HPC environments, which often requires knowing the compressibility of t...
详细信息
ISBN:
(纸本)9798350307924
Error-bounded lossy compression turns more and more important for the data-moving intensive applications to deal with big datasets efficiently in HPC environments, which often requires knowing the compressibility of the datasets before performing the compression. However, the off-the-shelf state-of-the-art lossy compressors are often driven by error bounds, so the compression ratios cannot be forecasted until the completion of the compression operation. In this paper, we propose a lightweight, robust, easy-to-train model that estimates the compressibility of datasets for different lossy compressors accurately. Our approach combines novel predictors that measure various notions of spatial correlation and smoothness exploited by lossy compressors that are implemented efficiently on the GPU in a framework and that uses mixture model regression to improve robustness with conformal prediction to provide bounds on the estimates. We then use these models with a detailed analysis of speedup to understand the tradeoffs between high speed, consistent speed, and accuracy of the methods on real applications. We evaluate our approach in the context of 3 key applications where compression ratio estimation is highly required.
This paper proposes the Gzip algorithm for image and document compression and decompression. Gzip is a hybrid algorithm that combines Lz77 and Huffman. In document management and communication systems, picture and doc...
详细信息
To address the pressing issue of wild animal incursions in rural areas, we propose an inventive solution to safeguard local communities, the project integrates fog computing and smart video compression. The system rec...
详细信息
With the development of convolutional neural networks, the scale of activation data has increased dramatically, posing significant challenges to model storage and computation. The insight indicates that the sparsity o...
详细信息
Sensing and onboard-processing capabilities of next-generation spacecraft continue to evolve. Enabled by advances in avionic systems, large amounts of data can be collected and stored on orbit. Nevertheless, loss of s...
详细信息
ISBN:
(纸本)9798350384543;9798350384536
Sensing and onboard-processing capabilities of next-generation spacecraft continue to evolve. Enabled by advances in avionic systems, large amounts of data can be collected and stored on orbit. Nevertheless, loss of signal, communication delays, and limited downlink rates remain a bottleneck for delivering data to ground stations or between satellites. This research investigates a multistage image-processing pipeline and demonstrates rapid collection, detection, and transmission of data using the Space Test Program - Houston 7 - Configurable and Autonomous Sensor Processing Research (STP-H7-CASPR) experiment aboard the International Space Station as a case study. Machine-learning (ML) models are leveraged to perform intelligent processing and compression of data prior to downlink to maximize available bandwidth. Moreover, to ensure accuracy and preserve data integrity, a fault-tolerant ML framework is employed to increase pipeline reliability. This pipeline fuses the fault-tolerant Resilient TensorFlow framework with ML-based tile classification and the CNNJPEG compression algorithm. This research shows that the imaging pipeline is able to alleviate the impact of limited communication bandwidth by using reliable, autonomous data processing and compression techniques to achieve reduced transfer sizes of essential data. The results highlight the benefits provided by resilient classification and compression including minimized storage use and reduced downlink time. Furthermore, the findings of this research are used to assess the feasibility of such a system for future space missions. The combination of these approaches enables the system to achieve up to a 98.67% reduction in data size and downlink time as well as the capacity to capture imagery up to 75.19x longer time period for a given storage size, respectively, while maintaining reconstruction quality and data integrity.
Federated Learning (FL) has emerged as a promising decentralized machine learning (ML) paradigm where distributed clients collaboratively train models without sharing their private data. However, the heterogeneous pro...
详细信息
暂无评论