Real-world networks, like social networks or the internet infrastructure, have structural properties such as large clustering coefficients that can best be described in terms of an underlying geometry. This is why the...
详细信息
Real-world networks, like social networks or the internet infrastructure, have structural properties such as large clustering coefficients that can best be described in terms of an underlying geometry. This is why the focus of the literature on theoretical models for real-world networks shifted from classic models without geometry, such as Chung-Lu random graphs, to modern geometry-based models, such as hyperbolic random graphs. With this paper we contribute to the theoretical analysis of these modern, more realistic random graph models. Instead of studying directly hyperbolic random graphs, we use a generalization that we call geometric inhomogeneous random graphs (GIRGs). Since we ignore constant factors in the edge probabilities, GIRGs are technically simpler (specifically, we avoid hyperbolic cosines), while preserving the qualitative behavior of hyperbolic random graphs, and we suggest to replace hyperbolic random graphs by this new model in future theoretical studies. We prove the following fundamental structural and algorithmic results on GIRGs. (1) We provide a sampling algorithm that generates a random graph from our model in expected linear time, improving the best-known sampling algorithm for hyperbolic random graphs by a substantial factor O (root n). (2) We establish that GIRGs have clustering coefficients in Omega(1), (3) we prove that GIRGs have small separators, i.e., it suffices to delete a sublinear number of edges to break the giant component into two large pieces, and (4) we show how to compress GIRGs using an expected linear number of bits. (C) 2018 Elsevier B.V. All rights reserved.
Despite the large number of research papers and compression algorithms proposed for com- pressing FASTQ genomic data generated by sequencing machines, by far the most commonly used compression algorithm in the industr...
详细信息
ISBN:
(纸本)9781538648834
Despite the large number of research papers and compression algorithms proposed for com- pressing FASTQ genomic data generated by sequencing machines, by far the most commonly used compression algorithm in the industry for FASTQ data is gzip. The main drawback of the proposed alternative special-purpose compression algorithms is the slow speed of either compression or de- compression or both, and also their brittleness by making various limiting assumptions about the input FASTQ format (for example, the structure of the headers or fixed lengths of the records [5]) in order to further improve their specialized compression. In this paper we propose using a simple modeling improvement combined with fast general purpose encoders, achieving 4X-40X speed improvements and 6%-11% more efficienct compression ratios for the compression and decompression of FASTQ genomic data compared to gzip and also being 5X faster than special-purpose FASTQ compressors like DSRC v2 and slimfastq.
Data compression techniques can alleviate low-bandwidth problems in multigigabit networks, and are especially useful when combined with encryption. The Titan II is a hardware compressor/decompressor core capable of sp...
详细信息
Data compression techniques can alleviate low-bandwidth problems in multigigabit networks, and are especially useful when combined with encryption. The Titan II is a hardware compressor/decompressor core capable of speeds of up to 10 gigabits per second. Its compression algorithm is a variation of the Lempel-Ziv LZ) algorithm that uses part of the previous input stream as the dictionary.
The quality of lossy compressed images is often characterized by signal-to-noise ratios, informal tests of subjective quality, or receiver operating characteristic (ROC) curves that include subjective appraisals of th...
详细信息
The quality of lossy compressed images is often characterized by signal-to-noise ratios, informal tests of subjective quality, or receiver operating characteristic (ROC) curves that include subjective appraisals of the value of an image for a particular application. We believe that for medical applications, lossy compressed images should be judged by a more natural and fundamental aspect of relative image quality: their use in making accurate diagnoses. We apply a lossy compression algorithm to medical images, and quantify the quality of the images by the diagnostic performance of radiologists, as well as by traditional signal-to-noise ratios and subjective ratings. Our study is unlike previous studies of the effects of lossy compression in that we consider non-binary detection tasks, simulate actual diagnostic practice instead of using paired tests or confidence rankings, use statistical methods that are more appropriate for non-binary clinical data than are the popular ROC curves, and use low-complexity predictive tree-structured vector quantization for compression rather than DCT-based transform codes combined with entropy coding. Our diagnostic tasks are the identification of nodules (tumors) in the lungs and lymphadenopathy in the mediastinum from computerized tomography (CT) chest scans. Radiologists read both uncompressed and lossy compressed versions of images. For the image modality, compression algorithm, and diagnostic tasks we consider, the original 12 bit per pixel (bpp) CT image can be compressed to between 1 bpp and 2 bpp with no significant changes in diagnostic accuracy. The techniques presented in this paper for evaluating image quality do not depend on the specific compression algorithm and are useful new methods for evaluating the benefits of any lossy image processing technique.
This paper provides a technical overview of the prioritization and transport in the Advanced Digital Television system (ADTV). ADTV is the all-digital terrestrial simulcast system developed by the Advanced Television ...
详细信息
This paper provides a technical overview of the prioritization and transport in the Advanced Digital Television system (ADTV). ADTV is the all-digital terrestrial simulcast system developed by the Advanced Television Research Consortium, ATRC, (Thomson Consumer Electronics, Philips North America, NBC, compression Labs, and the David Sarnoff Research Center.) ADTV incorporates an efficient MPEG-compatible compression algorithm at its central core, with application-specific data prioritization and transport features added as separable layers. The compression process is based on a 1440x960 (1050-line 2:1 interlaced) HDTV format, producing a selectable bit-rate in the region of 15-20 Mbps. The data prioritization layer of ADTV achieves robust delivery over an appropriate two-tier modem by separating compressed video data into high- and standard-priority bitstreams with appropriate bit-rates. This prioritized data is then formatted into fixed length "cells" (packets) with appropriate data-link level and service-specific adaptation level headers, designed to provide capabilities such as flexible service multiplexing, priority handling, efficient cell packing, error detection and graceful recovery from errors. (Issues related to the ADTV compression coding, receiver error recovery, and transmission are discussed in separate papers.[1,2,3])
This paper introduces a novel approach to visual data compression. The approach, named dynamic coding, consists of an effective competition between several representation models used for describing data portions, The ...
详细信息
This paper introduces a novel approach to visual data compression. The approach, named dynamic coding, consists of an effective competition between several representation models used for describing data portions, The image data is represented as the union of several regions each approximated by a representation model locally appropriate, The dynamic coding concept leads to attractive features such as genericness, flexibility, and openness and is therefore particularly suited to a multimedia environment in which many types of applications are involved. Dynamic coding is a general proposal to visual data compression and many variations on the same theme may be designed. They differ by the particular procedure by which the data is segmented into objects and the local representation model selected, As an illustrative example, a video compression scheme based on the principles of dynamic coding is presented, This compression algorithm performs a joint optimization of the segmentation (restricted to a so-called generalized quadtree partition) together with the representation models associated with each data segment. Four representation models are competing namely, fractal, motion compensation, text and graphics, and background modes, Optimality is defined with respect to a rate-distortion tradeoff and the optimization procedure leads to a multicriterion segmentation.
Despite the potential of neural scene representations to effectively compress 3D scalar fields at high reconstruction quality, the computational complexity of the training and data reconstruction step using scene repr...
详细信息
Despite the potential of neural scene representations to effectively compress 3D scalar fields at high reconstruction quality, the computational complexity of the training and data reconstruction step using scene representation networks limits their use in practical applications. In this paper, we analyse whether scene representation networks can be modified to reduce these limitations and whether such architectures can also be used for temporal reconstruction tasks. We propose a novel design of scene representation networks using GPU tensor cores to integrate the reconstruction seamlessly into on-chip raytracing kernels, and compare the quality and performance of this network to alternative network- and non-network-based compression schemes. The results indicate competitive quality of our design at high compression rates, and significantly faster decoding times and lower memory consumption during data reconstruction. We investigate how density gradients can be computed using the network and show an extension where density, gradient and curvature are predicted jointly. As an alternative to spatial super-resolution approaches for time-varying fields, we propose a solution that builds upon latent-space interpolation to enable random access reconstruction at arbitrary granularity. We summarize our findings in the form of an assessment of the strengths and limitations of scene representation networks for compression domain volume rendering, and outline future research directions. Source code:
Convolutional neural networks (CNNs) have found great success in many artificial intelligence applications. At the heart of a CNN is the operation of convolution between multi-channel data and learned kernels. It is u...
详细信息
Convolutional neural networks (CNNs) have found great success in many artificial intelligence applications. At the heart of a CNN is the operation of convolution between multi-channel data and learned kernels. It is usually implemented as a floating-point large matrix multiplication, a major bottleneck of computational speed and memory usage. In this paper, we propose a binary outer product expansion (BOPE) method to represent a kernel matrix or tensor as a weighted sum of the outer products between binary vectors of values +1 or -1. This allows for network compression and simplified computation at the same time. Our theoretical analysis shows such a decomposition converges to the original matrix given a sufficient number of binary vectors. We present computational methods to estimate outer product weights using either optimized or random binary base vectors. Significant data compression can be achieved for a highly redundant matrix, since weights and binary vectors require less storage than array elements. In addition, most floating-point multiplication in matrix convolution can be replaced by addition and binary XOR, lessening computation and memory requirements. We propose a compact convolutional layer in which highly redundant convolutional kernels are projected onto binary vectors, and represented as a weighted sum of outer products. It is shown that the number of weights in AlexNet, VGG-19 and ResNet-50 can be reduced 3.45, 6.87 and 2.95 times respectively with less than a 1% loss in the top-1 and top-5 classification accuracy on ImageNet, and MobileNetV2 can be reduced 2.31 times with a 2% loss. Compared to the standard CNN, this compact convolutional network has fewer trainable weights, is better regularized, and is easier to train from fewer training samples. Therefore, it is particularly suited for devices with limited computation, memory, and battery power.
A very useful real time ocean monitor system has been developing. In the ocean-satetellite-ground transmission subsystem of the system, suitable compression methods, right compression algorithms and efficient transmis...
详细信息
ISBN:
(纸本)0780379292
A very useful real time ocean monitor system has been developing. In the ocean-satetellite-ground transmission subsystem of the system, suitable compression methods, right compression algorithms and efficient transmission methods have to be developed because of the real time transmission and the huge video imagery data as well as the limiting satellite channel bandwidth. In the real time ocean monitor system, there are three developed technologies: One is the ocean-satellite-ground transmission subsystem configuration and data transmission method. Another is the improved video image compression algorithm and the comparing method with other compression algorithms. Third is the data splitting and compounding method. Finally, some experiment results will be given.
Large-scale Machine Learning (ML) algorithms are often iterative, using repeated read-only data access and I/O-bound matrix-vector multiplications. Hence, it is crucial for performance to fit the data into single-node...
详细信息
Large-scale Machine Learning (ML) algorithms are often iterative, using repeated read-only data access and I/O-bound matrix-vector multiplications. Hence, it is crucial for performance to fit the data into single-node or distributed main memory to enable fast matrix-vector operations. General-purpose compression struggles to achieve both good compression ratios and fast decompression for block-wise uncompressed operations. Therefore, we introduce Compressed Linear Algebra (CIA) for lossless matrix compression. CIA encodes matrices with lightweight, value-based compression techniques and executes linear algebra operations directly on the compressed representations. We contribute effective column compression schemes, cache-conscious operations, and an efficient sampling-based compression algorithm. Our experiments show good compression ratios and operations performance close to the uncompressed case, which enables fitting larger datasets into available memory. We thereby obtain significant end-to-end performance improvements.
暂无评论