The numerous applications of urban traffic detection technology in road traffic data acquisition bring new challenges for transportation and storage of road traffic big data. The travel demand and travel time of trave...
详细信息
The numerous applications of urban traffic detection technology in road traffic data acquisition bring new challenges for transportation and storage of road traffic big data. The travel demand and travel time of travel participants present certain specific regularity;thus, a compression algorithm for road traffic data in time series based on temporal correlation was proposed in this study. First, the temporal correlation of the road traffic data in time series was analysed. Second, the reference sequences of road traffic characteristics were constructed to acquire the base data under different modes. Third, the training data under the same mode were extracted to acquire the difference data between training and base data. Then the optimal threshold of the difference data was trained. Fourth, the optimal threshold was introduced into the difference data between real-time and base data in time series, combining with Lempel-Ziv-Welch (LZW) encoding to achieve the compression of difference data. Finally, the reconstruction of real-time road traffic data in time series was accomplished based on LZW decoding technology. Six typical road segments in Beijing were adopted for case studies. The final results prove the feasibility of the algorithm, and that the reconstructed data can achieve high accuracy.
This paper introduces a methodology for computing expected values associated with compression networks resulting from the application of compression algorithms to independent and identically distributed random time se...
详细信息
This paper introduces a methodology for computing expected values associated with compression networks resulting from the application of compression algorithms to independent and identically distributed random time series. Our analysis establishes a robust correspondence between the calculated expected values and empirically derived results obtained from constructing networks using nondeterministic time series. Notably, the ratio of the average indegree of a network to the computed expected indegree for stochastic time series serves as a versatile metric. It enables the assessment of inherent randomness in time series and facilitates the distinction between nondeterministic and chaotic systems. The metric demonstrates high sensitivity to nondeterminism in both synthetic and real-world datasets, highlighting its capacity to detect subtle disturbances and high-frequency noise, even in series characterized by a deficient sample rate. Our results extend and confirm previous findings in the field.
Background: As Next-Generation Sequencing data becomes available, existing hardware environments do not provide sufficient storage space and computational power to store and process the data due to their enormous size...
详细信息
Background: As Next-Generation Sequencing data becomes available, existing hardware environments do not provide sufficient storage space and computational power to store and process the data due to their enormous size. This is and will be a frequent problem that is encountered everyday by researchers who are working on genetic data. There are some options available for compressing and storing such data, such as general-purpose compression software, PBAT/PLINK binary format, etc. However, these currently available methods either do not offer sufficient compression rates, or require a great amount of CPU time for decompression and loading every time the data is accessed. Results: Here, we propose a novel and simple algorithm for storing such sequencing data. We show that, the compression factor of the algorithm ranges from 16 to several hundreds, which potentially allows SNP data of hundreds of Gigabytes to be stored in hundreds of Megabytes. We provide a C++ implementation of the algorithm, which supports direct loading and parallel loading of the compressed format without requiring extra time for decompression. By applying the algorithm to simulated and real datasets, we show that the algorithm gives greater compression rate than the commonly used compression methods, and the data-loading process takes less time. Also, The C++ library provides direct-data-retrieving functions, which allows the compressed information to be easily accessed by other C++ programs. Conclusions: The SpeedGene algorithm enables the storage and the analysis of next generation sequencing data in current hardware environment, making system upgrades unnecessary.
The oil-gas gathering and transportation station is the key to the oil and gas gathering and transportation in the whole onshore oil and gas field, and also the high risk. Therefore, it is very important to evaluate t...
详细信息
The oil-gas gathering and transportation station is the key to the oil and gas gathering and transportation in the whole onshore oil and gas field, and also the high risk. Therefore, it is very important to evaluate the risk of oil and gas gathering and transportation station and take corresponding measures according to the result. Based on the bowtie model, the left-wing is replaced by Bayesian network, and the uncertainty factor is considered. In the process of Bayesian network to the right event tree, the bowtie barrier is added to construct the Net-bowtie improved model. The dynamic change model is used to consider the real-time changes of basic events into Bayesian networks. Based on the improved compression algorithm, the conditional probability table is encoded and compressed so as to store the data. Finally, the practical application of the dehydration and degassing station of the Net-bowtie model is given, and the data storage space saving of the improved compression algorithm is validated. This model has been used in the field of dehydration and degassing stations of the DND Gas Field and the corrosion reliability of the northeastern Sichuan gas mine.
Real-time database applied in process industry requests the performance of mass data and high speed So the process data in database must be compressed effectively and reliably. According to process data characteristic...
详细信息
ISBN:
(纸本)1424403316
Real-time database applied in process industry requests the performance of mass data and high speed So the process data in database must be compressed effectively and reliably. According to process data characteristic, a lossless compression algorithm was designed based on LZW algorithm and RLE algorithm. The compression algorithm first classified process data by characteristic, and then different compression methods were designed for all kinds of data. In order to increase compression radio, pretreatment approaches were implemented before compression. The compression algorithm solved the difficulties of low compression radio and compression speed. Performance test shows that this algorithm obviously improved the real-time performance and efficiency when accessing the process database. It will be widely applied in MES and PCS.
This paper presents a new efficient DCT-IV-based ECG compression algorithm with a higher Quality Score (QS) and a better Compressing Ratio (CR). The ECG signals sourced from MIT-BIT arrhythmia database with a sampling...
详细信息
ISBN:
(纸本)9781467357623;9781467357609
This paper presents a new efficient DCT-IV-based ECG compression algorithm with a higher Quality Score (QS) and a better Compressing Ratio (CR). The ECG signals sourced from MIT-BIT arrhythmia database with a sampling rate of 360 Hz are employed to be the test patterns for the evaluating the proposed compression algorithm. The simulation results show that the averages of CR, Percent RMS Difference (PRD), and QS are, respectively, 5.267, 0.187, and 28.223 for all 48 lead-V1 patterns of MIT-BIH database. Compared with Lee et al.'s algorithm, the QS value of the proposed method has a great improvement by 25.1%. Additionally, we use DCT-IV to be a unified transform kernel for ECG signal encoding and decoding because the formula of forward DCT-IV is same to its inverse. Also, we realize it to be a compact hardware accelerator with a fewer hardware resources. Therefore, it would be a better choice for realizing the ECG compressor in the future.
Unmanned aerial vehicle(UAV) often flies in close proximity to terrain. The expansion in the number of low-altitude flight operations, along with increased operator workload, have made controlled flight into terrain t...
详细信息
ISBN:
(纸本)9780769549354
Unmanned aerial vehicle(UAV) often flies in close proximity to terrain. The expansion in the number of low-altitude flight operations, along with increased operator workload, have made controlled flight into terrain the number one cause of loss of UAV. This paper is to give an overview of the Automatic Ground Collision Avoidance System (Auto-GCAS) program attempt to provide a system to eliminate that cause of UAV loss. The Auto-GCAS utilizes a digital terrain system with a terrain referenced navigation algorithm to locate the aircraft spatially with respect to the terrain. The terrain database around the aircraft is scanned, and a terrain profile is created. This paper presents a new compression algorithm called bubble terrain compression algorithm for Auto-GCAS of UAV. In this paper, optimal trajectory is considered as a minimax optimal control problem, which is solved using direct transcription of the continuous optimal control problem. Within a very general framework for solving such problems, we transform the non-smooth cost function into a constrained nonlinear programming problem. In the formulation, we solve for optimal collision avoidance manoeuvres. To ensure smooth derivatives of general two dimensional terrain, it is approximated using the optimal trajectories with disturbances in the initial conditions. Simulation results show that the UAV with the proposed methodology successfully achieves autonomous recovery maneuvers in MATLAB environment.
Image mapping is widely applied in various fields of multi-camera video, panoramic display, medical imaging, and AR augmented reality. Generally speaking, its software implementation is a computing-intensive task and ...
详细信息
ISBN:
(纸本)9798350352153;9798350352160
Image mapping is widely applied in various fields of multi-camera video, panoramic display, medical imaging, and AR augmented reality. Generally speaking, its software implementation is a computing-intensive task and its hardware implementation needs a lot of both hardware computing resources and storage space. In order to attack such disadvantages, an efficient and hardware-friendly image mapping parameter matrix compression algorithm based on bilinear interpolation has been proposed. The proposed algorithm can theoretically achieve compression ratio with any scale and any image size, which largely benefits its circuit acceleration. The proposed algorithm has been evaluated on image quality from both subjective and objective perspectives based on both USIS-D dataset and a self-shot data, meanwhile it has been verified on a Xilinx Kintex7-325T FPGA board. Experimental results demonstrated that the proposed algorithm significantly cuts down computation and storage resource consumption under the premise of losing little image quality.
PCI Express (PCIe) has been widely used as an 1/0 bus connecting CPU and GPUs. In order to resolve the limitation of the number of PCIe ports, NEC Corporation developed ExpEther for expanding PCIe to Ethernet. Since E...
详细信息
ISBN:
(纸本)9781538691847
PCI Express (PCIe) has been widely used as an 1/0 bus connecting CPU and GPUs. In order to resolve the limitation of the number of PCIe ports, NEC Corporation developed ExpEther for expanding PCIe to Ethernet. Since Ethernet often becomes a bottleneck of communication, a conventional research proposed to implement a compression/decompression mechanism by using existing data compression mechanisms to reduce the size of data transferring on Ethernet. However, data compression mechanisms used in the research were only efficient fin" a limited input data pattern. In this paper, we proposed a novel data compression algorithm called C4, and implemented it on Xilinx Virtex-7 FPGA as an experimental environment of ExpEther. As a result, the proposed method can reduce the transfer time by 52.5%, superior to 49.7% with the conventional method. According to the evaluation of the hardware resource utilization rate, we showed that the proposed algorithm can be implemented in the FPGA used in the ExpEther NIC.
Data compression has become a commodity feature for space efficiency and performance by reducing reading and writing traffic and space capacity demand. This technology is particularly valuable for a file system to man...
详细信息
ISBN:
(纸本)9781450366069
Data compression has become a commodity feature for space efficiency and performance by reducing reading and writing traffic and space capacity demand. This technology is particularly valuable for a file system to manage and server the big data processing tasks. However, the fixed data compression scheme cannot fit all the big data workloads and data-set which have complex internal data structure and compressibility. This paper investigates a dynamic and smart data compression algorithm selection scheme for different big data processing cases in the local file system. To this end, we propose a dynamic algorithm selection module in the Linux ZFS which is an open source file system. This module will select a high compression ratio algorithm for high compressibility data, and select a fast compression algorithm for low compressibility data, and skip all data compression process for incompressibility data. The comprehensive evaluations validate that dynamic algorithm selection module can achieve up to 2.69x response time improvement for reading and writing operation in file system and reduce about 32.12% storage space for a large amount data-set.
暂无评论