检索结果-内蒙古大学图书馆

TCMS: A Multi-sequence Log Parsing Method Based on Token Conversion

IEEE Transactions on Dependable and Secure Computing 2024年第03期22卷 3028-3045页

作者： Wei, Mingkuan Wen, Jigang He, Shiming Xie, Kun Liang, Wei Xie, Gaogang Li, Kenli Zhu, Ziyu Hunan University The Ministry of Education Key Laboratory of 'Fusion Computing of Supercomputing and Artificial Intelligence' College of Computer Science and Electronics Engineering Changsha410012 China Hunan University of Science and Technology School of Computer Science and Engineering Xiangtan411201 China Changsha University of Science and Technology School of Computer and Communication Engineering The Hunan Provincial Key Laboratory of Intelligent Processing of Big Data on Transportation Changsha410114 China Beijing100045 China Columbia University Electrical Engineering Newyork10027 United States

Detailed system operations are recorded in logs. To ensure system reliability, developers can detect system anomalies through log anomaly detection. Log parsing, which converts semi-structured log messages into structured data, is a crucial step in log anomaly detection and advanced program analysis and verification. Despite the availability of various log parsing tools, they generally suffer from low parsing accuracy and slow efficiency due to the ignorance of variable characteristics and the use of costly pairwise comparison methods. In this paper, we propose a TCMS framework to parse logs, consisting of two main technologies. First, by studying 16 public log datasets, we find that most log variable tokens are structured variable tokens. Based on this discovery, we propose a token conversion algorithm to improve parsing accuracy. This algorithm converts the changed parts in structured variable tokens into wildcards (''), preventing these tokens from being directly identified as constant tokens. Second, to improve efficiency, we propose the LogMLCS algorithm, which intelligently constructs a graph to facilitate the extraction of common parts from multiple log messages at once, instead of using pairwise comparisons. Comprehensive experiments conducted on 16 log datasets reveal that our TCMS outperforms seven other parsing methods, achieving the highest parsing accuracy at the fastest speed. Furthermore, experimental results from running a log anomaly detection algorithm in conjunction with different log parsing methods demonstrate that TCMS significantly boosts detection accuracy. For instance, on the OpenStack dataset, our TCMS-facilitated log anomaly detection algorithm achieves a perfect F1-score, precision, and recall of 100% each, surpassing the best peer method by 32.2, 0.8, and 19.5 percentage points, respectively. © 2004-2012 IEEE.

关键词： Accuracy Anomaly Detection Clustering Algorithms Peer To Peer Computing Electronic Mail Source Coding Particle Separators Codes Tokenization Itemsets Log Parsing Log Analysis Token Conversion Log MLCS Log Parsing Pairwise Comparisons Low Accuracy Public Datasets Anomaly Detection Common Part Multiple Messages Long Short Term Memory point Source Target Domain Logarithm Of The Number Captive Animals Multi Objective Optimization Problem Regular Expressions Biological Sequences Rule Based Methods point in the graph Kinds Of Datasets Template Generation Number Of Templates Similar Template Parse Tree Syntax Errors Harmonic Average Of Precision Log Length Earliest Methods Common Subsequence Precision And Recall Leaf Node Part Of The Message

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：