检索结果-内蒙古大学图书馆

2nd IEEE International Conference on Software Services Engineering (SSE)

作者： Chen, Jiarui Wuhan Univ Sch Comp Sci Wuhan Peoples R China

ISBN: (纸本)9798350368529;9798350368512

logs play a crucial role in monitoring system behaviors, providing observability of system events at various levels. However, large-scale software systems generate massive amounts of logs daily, leading to tremendous storage overhead and management burdens. compression techniques are employed to reduce the storage pressure of logs by eliminating redundancy. Traditional compression methods achieve moderate compression ratios but fail to fully exploit the structural attributes of log data. Several efforts have been made to compress logs based on their structural or temporal attributes, but they struggle with identifying word-level patterns in log contents. Additionally, numerical values, such as timestamps, in log headers and parameters pose challenges in compression. To address these limitations, we propose logCompact, a novel log compressor that recognizes redundancy at both the word and numerical levels. Meanwhile, logCompact incorporates a word encoder based on Huffman trees to reduce the encoding length of high-frequency words and a two-dimensional differential encoder to simplify the representation of numerical values. Experimental results on five datasets demonstrate the effectiveness of logCompact in achieving higher compression rates compared to baseline methods.

关键词： logs log compression log parse

来源：评论

学校读者我要写书评

暂无评论

Unlocking the Power of Numbers: log compression via Numeric Token Parsing 24

Unlocking the Power of Numbers: Log Compression via Numeric ...

引用

39th ACM/IEEE International Conference on Automated Software Engineering (ASE)

作者： Yu, Siyu Wu, Yifan Li, Ying He, Pinjia Chinese Univ Hong Kong Shenzhen CUHK Shenzhen Shenzhen Peoples R China Peking Univ Beijing Peoples R China Shenzhen Res Inst Big Data Shenzhen Peoples R China

ISBN: (数字)9798400712487

ISBN: (纸本)9798400712487

Parser-based log compressors have been widely explored in recent years because the explosive growth of log volumes makes the compression performance of general-purpose compressors unsatisfactory. These parser-based compressors preprocess logs by grouping the logs based on the parsing result and then feed the preprocessed files into a general-purpose compressor. However, parser-based compressors have their limitations. First, the goals of parsing and compression are misaligned, so the inherent characteristics of logs were not fully utilized. In addition, the performance of parser-based compressors depends on the sample logs and thus it is very unstable. Moreover, parser-based compressors often incur a long processing time. To address these limitations, we propose Denum, a simple, general log compressor with high compression ratio and speed. The core insight is that a majority of the tokens in logs are numeric tokens (i.e. pure numbers, tokens with only numbers and special characters, and numeric variables) and effective compression of them is critical for log compression. Specifically, Denum contains a Numeric Token Parsing module, which extracts all numeric tokens and applies tailored processing methods (e.g. store the differences of incremental numbers like timestamps), and a String Processing module, which processes the remaining log content without numbers. The processed files of the two modules are then fed as input to a general-purpose compressor and it outputs the final compression results. Denum has been evaluated on 16 log datasets and it achieves an 8.7% - 434.7% higher average compression ratio and 2.6x - 37.7x faster average compression speed (i.e. 26.2 MB/S) compared to the baselines. Moreover, integrating Denum's Numeric Token Parsing module into existing log compressors can provide a 11.8% improvement in their average compression ratio and achieve 37% faster average compression speed.

关键词： Data compression log compression log Analysis

来源：评论

学校读者我要写书评

暂无评论

On the Application of log compression and Enhanced Denoising in Contrast Enhancement of Digital Radiography Images 8th

On the Application of Log Compression and Enhanced Denoising...

引用

8th International Conference on Computer Vision and Image Processing (CVIP)

作者： Asif, M. S. Panicker, Mahesh Raveendranatha Indian Inst Technol Palakkad Ctr Computat Imaging Palakkad India Indian Inst Technol Palakkad Dept Elect Engn Palakkad India

ISBN: (纸本)9783031581731;9783031581748

Digital radiography (DR) is becoming popular for the point of care imaging in the recent past. To reduce the radiation exposure, controlled radiation based on as low as reasonably achievable (ALARA) principle is employed and this results in low contrast images. To address this issue, post-processing algorithms such as the Multiscale Image Contrast Amplification (MUSICA) algorithm can be used to enhance the contrast of DR images even with a low radiation dose. In this study, a modification of the MUSICA algorithm is investigated to determine the potential for further contrast improvement specifically for DR images. The conclusion is that combining log compression and its inverse at the appropriate stage with a multi-stage MUSICA and denoising is very promising. The proposed method resulted in an average of 66.5% increase in the mean contrast-to-noise ratio (CNR) for the test images considered.

关键词： Low Dose DR Image enhancement log compression Multi-stage MUSICA

来源：评论

学校读者我要写书评

暂无评论

logDAC: A Universal Efficient Parser-based log compression Approach

LogDAC: A Universal Efficient Parser-based Log Compression A...

引用

IEEE International Conference on Communications (ICC)

作者： Tian, Ran Diao, Zulong Jiang, Haiyang Xie, Gaogang Univ Chinese Acad Sci Beijing Peoples R China Chinese Acad Sci lnst Comp Technol Beijing Peoples R China Chinese Acad Sci Comp Network Informat Ctr Beijing Peoples R China Purple Mt Labs Nanjing Peoples R China

ISBN: (数字)9781538683477

ISBN: (纸本)9781538683477

A large volume of logs provides a reliable data source for online services. For instance, 5G, Big 5G or Beyong 5G cloud core network and its User Plane Functions require analysis built on massive log storage. The storage cost of log data remains a problem for the industry. Compressing the logs before archives remains the most popular way to reduce it. However, structured logs, which usually present in tabular format, and unstructured logs, which build on variable templates, have to use different compression strategies. Using log parsers can eliminate this difference by converting unstructured logs into structured ones. Based on this idea, we introduce our universal efficient parser-based log compression approach logDAC. It also uses a divide-and-conquer preprocess strategy to improve the compatibility with successive dictionary-based general compressors. The evaluation against both structured and unstructured public datasets show a solid improvement up to 45.4% and 489.53% of compression ratio respectively, compared with existing parser-based compressors.

关键词： log compression logs log management

来源：评论

学校读者我要写书评

暂无评论

logzip: extracting hidden structures via iterative clustering for log compression 19

Logzip: extracting hidden structures via iterative clusterin...

引用

Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering

作者： Jinyang Liu Jieming Zhu Shilin He Pinjia He Zibin Zheng Michael R. Lyu Sun Yat-Sen University Guangzhou China and The Chinese University of Hong Kong Hong Kong China Huawei Noah's Ark Lab Shenzhen China The Chinese University of Hong Kong Hong Kong China ETH Zurich Switzerland Sun Yat-Sen University Guangzhou China

ISBN: (纸本)9781728125084

System logs record detailed runtime information of software systems and are used as the main data source for many tasks around software engineering. As modern software systems are evolving into large scale and complex structures, logs have become one type of fast-growing big data in industry. In particular, such logs often need to be stored for a long time in practice (e.g., a year), in order to analyze recurrent problems or track security issues. However, archiving logs consumes a large amount of storage space and computing resources, which in turn incurs high operational cost. Data compression is essential to reduce the cost of log storage. Traditional compression tools (e.g., gzip) work well for general texts, but are not tailed for system logs. In this paper, we propose a novel and effective log compression method, namely logzip. logzip is capable of extracting hidden structures from raw logs via fast iterative clustering and further generating coherent intermediate representations that allow for more effective compression. We evaluate logzip on five large log datasets of different system types, with a total of 63.6 GB in size. The results show that logzip can save about half of the storage space on average over traditional compression tools. Meanwhile, the design of logzip is highly parallel and only incurs negligible overhead. In addition, we share our industrial experience of applying logzip to Huawei's real products.

关键词： iterative clustering log management structure extraction log compression logs

来源：评论

学校读者我要写书评

暂无评论

Smart industrial information integration: a lightweight privacy protection model in an intelligent manufacturing architecture

引用

JOURNAL OF ENGINEERING DESIGN 2024年第11期35卷 1339-1366页

作者： Yang, Miaomiao Huang, Guosheng Liao, Yongyi Wang, Lan Wang, Qixu Chen, Xingshu Shen, Changxiang Sichuan Univ Sch Cyber Sci & Engn Chengdu 610065 Peoples R China Sichuan Univ Minist Educ Key Lab Data Protect & Intelligent Management Chengdu 610065 Peoples R China Sichuan Univ Cyber Sci Res Inst Chengdu 610065 Peoples R China Nucl Power Inst China Sci & Technol Reactor Syst Design Technol Lab Chengdu Peoples R China Univ Technol Sch Comp Sci Beijing Peoples R China

In the industrial ecosystem, both the manufacturing and edge clouds collaborate via virtualisation technology, playing a crucial role in the efficient aggregation and processing of vast data from endpoints, offering computational and analytical support for enhanced decision-making in all industrial operations. Consequently, ensuring the reliability of virtualised manufacturing environments is paramount. Remote attestation offers a viable solution, however, current schemes are plagued by inefficiencies and privacy leakage issues. To address these concerns, we propose an efficient remote attestation approach. The approach actively conceals sensitive industrial information in measurement logs by using a hash algorithm, thereby preventing privacy breaches. Furthermore, it streamlines log entries with a log compression algorithm, enhancing remote verification efficiency in the industrial information architecture. Extensive experiments were conducted in an authentic cloud environment by applying this approach across both mesh and hierarchical remote verification structures, reducing time overhead by approximately 81.32%. Additionally, it marginally impacts server performance with only a 1.49% reduction. The results affirm the proposed approach's exceptional efficacy in decreasing verification durations and mitigating remote verification attacks, demonstrating its promising utility and feasibility in the intelligent industrial platform.

关键词： Virtualized manufacturing environments remote attestation log compression industrial sensitive information protection manufacturing trustworthiness

来源：评论

学校读者我要写书评

暂无评论

Turbolog: A Turbocharged Lossless compression Method for System logs via Transformer

TurboLog: A Turbocharged Lossless Compression Method for Sys...

引用

International Joint Conference on Neural Networks (IJCNN)

作者： Chang, Baoming Wang, Zhaoyang Li, Shuai Zhou, Fengxi Wen, Yu Zhang, Boyang Chinese Acad Sci Key Lab Cyberspace Secur Def Inst Informat Engn Beijing Peoples R China Univ Chinese Acad Sci Sch Cyber Secur Beijing Peoples R China

ISBN: (纸本)9798350359329;9798350359312

System logs provide valuable information for intrusion detection and forensic analysis. To counter cyber attacks, enterprises widely deploy monitoring software and record fine-grained system events within the operating system through system logs. However, system logs can pile up in large quantities as the complex nature of modern computer systems and the covert nature of cyber attacks. Storing system logs efficiently is an important and challenging task. Lossless compression techniques provide an intuitive idea of reducing the size of system logs. Recently, researchers have applied Deep Neural Networks (DNNs) in designing compression strategies, achieving remarkable outcomes. Unfortunately, general-purpose lossless compression methods fail to achieve effective compression due to their inadequate extraction of structural redundancy from system logs. Certain methods opt to add a few preprocessing steps before log compression to enhance compression efficiency. Nevertheless, their RNN-based models face challenges in efficiently utilizing GPU resources and can not handle parallel computation. Therefore, their methods still demand a significant compression time. In this paper, we propose Turbolog, a novel transformer-based compression method specifically designed for system logs. To address the above deficiencies, Turbolog initiates by executing a series of preprocessing steps to reduce the structural redundancy and numerical redundancy within raw logs. Subsequently, Turbolog uses a transformer-based probability estimator to model log data in parallel. Finally, Turbolog combines the probability estimation of log data with arithmetic coding to accomplish log compression. We use a large public dataset to evaluate Turbolog. The results show that Turbolog can achieve the highest compression ratio of 65.25x. Furthermore, compared to the state-of-the-art method, Turbolog decreases compression time by 75% while concurrently enhancing the compression ratio by 20%. In conclusion,

关键词： system log log compression deep learning

来源：评论

学校读者我要写书评

暂无评论

logAssist: Assisting log Analysis Through log Summarization

引用

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING 2022年第9期48卷 3227-3241页

作者： Locke, Steven Li, Heng Chen, Tse-Hsun (Peter) Shang, Weiyi Liu, Wei Concordia Univ Software PErformance Anal & Reliabil SPEAR Lab Montreal PQ H3G 1M8 Canada Concordia Univ Dept Comp Sci & Software Engn Montreal PQ H3G 1M8 Canada Polytech Montreal Dept Comp Engn & Software Engn Montreal PQ H3T 1J4 Canada

logs contain valuable information about the runtime behaviors of software systems. Thus, practitioners rely on logs for various tasks such as debugging, system comprehension, and anomaly detection. However, logs are difficult to analyze due to their unstructured nature and large size. In this paper, we propose a novel approach called logAssist that assists practitioners with log analysis. logAssist provides an organized and concise view of logs by first grouping logs into event sequences (i.e., workflows), which better illustrate the system runtime execution paths. Then, logAssist compresses the log events in workflows by hiding consecutive events and applying n-gram modeling to identify common event sequences. We evaluated logAssist on logs generated by one enterprise and two open source systems. We find that logAssist can reduce the number of log events that practitioners need to investigate by up to 99 percent. Through a user study with 19 participants, we find that logAssist can assist practitioners by reducing the time required for log analysis tasks by an average of 40 percent. The participants also rated logAssist an average of 4.53 out of 5 for improving their experiences of performing log analysis. Finally, we document our experiences and lessons learned from developing and adopting logAssist in practice. We believe that logAssist and our reported experiences may lay the basis for future analysis and interactive exploration on logs.

关键词： Task analysis Runtime Tools Testing Faces Anomaly detection Software systems log analysis log compression n-gram modeling log abstraction workflow characterization log reduction

来源：评论

学校读者我要写书评

暂无评论

Nonsubsampledcontourlet transform with cross-guided bilateral filter for despeckling of medical ultrasound images

引用

INTERNATIONAL JOURNAL OF IMAGING SYSTEMS AND TECHNOlogY 2021年第2期31卷 763-777页

作者： Joel, Thapasimuthan Sivakumar, Rajagopal RMK Engn Coll Dept ECE Kavaraipettai 601206 Tamil Nadu India

This paper aims to enhance the image quality in ultrasound images. The significant difficulties in the ultrasound image are the presence of speckle noise. Speckle is the granular noise, and this kind of noise produces a lot of challenges during medical diagnosis. Also, these kinds of problems degrade image quality. The proposed work overcomes this kind of issue with nonsubsampled contourlet domain-based cross-guided bilateral filtering. Initially, additive speckle noise is introduced with the log-transformed Rayleigh distribution. This additive speckle noise component is to be decomposed through the non-subsampled contourlet domain transform. Then the decomposed image is despeckled by the cross-guided bilateral filter, and the cost function of this filter is minimized through adaptive galactic swarm optimization. The proposed filtering technique despeckles the additive noise in the noisy image. Finally, the decomposed image is reconstructed using the inverse transform of the nonsubsampled decomposition. The implementation of the proposed methodology is carried out in MATLAB simulation platform. Also, the performance of the proposed work is measured by few metrics such as structural similarity index matrix, mean structural similarity index matrix, mean square error, peak signal to noise ratio, Bhattacharyya coefficient, and speckle index. The result of cost function optimization provides less distortion when compared to other techniques.

关键词： decomposition despeckling filter log compression multiplicative noise optimization Rayleigh distribution ultrasound images

来源：评论

学校读者我要写书评

暂无评论

A study of the performance of general compressors on log files

引用

EMPIRICAL SOFTWARE ENGINEERING 2020年第5期25卷 3043-3085页

作者： Yao, Kundi Li, Heng Shang, Weiyi Hassan, Ahmed E. Queens Univ Kingston Software Anal & Intelligence Lab Kingston ON K7L 3N6 Canada Concordia Univ Montreal Dept Comp Sci & Software Engn Montreal PQ H3G 1M8 Canada

Large-scale software systems and cloud services continue to produce a large amount of log data. Such log data is usually preserved for a long time (e.g., for auditing purposes). General compressors, like theLZ77compressor used ingzip, are usually used in practice to compress log data to reduce the cost of long-term storage. However, such general compressors do not consider the unique nature of log data. In this paper, we study the performance of general compressors on compressing log data relative to their performance on compressing natural language data. We used 12 widely used general compressors to compress nine log files that are collected based on surveying prior literature on text compression, log compression and log analysis. We observe that log data is more repetitive than natural language data, and that log data can be compressed and decompressed faster with higher compression ratios. Besides, the compressor with the highest compression ratio for natural language data is rarely the one for log data. Nevertheless, the compressors with the highest compression ratio for log data are rarely adopted in practice by current logging libraries and log management tools. We also observe that the peak compression and decompression speeds of general compressors on log data is often achieved with a small data size, while such size may not be used by log management tools. Finally, we observe that the optimal compression performance (measured by a combined compression performance score) of log data usually requires the compression level to be configured higher than the default level. Our findings call for careful consideration of choosing general compressors and their associated compression levels for log data in practice. In addition, our findings shed lights on the opportunities for future research on compressors that better suit the characteristics of log data.

关键词： log compression Software logging log management Language model

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：