检索结果-内蒙古大学图书馆

arXiv 2024年

作者： Kuang, Jinxi Liu, Jinyang Huang, Junjie Zhong, Renyi Gu, Jiazhen Yu, Lan Tan, Rui Yang, Zengyin Lyu, Michael R. The Chinese University of Hong Kong Hong Kong Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd China

Due to the scale and complexity of cloud systems, a system failure would trigger an "alert storm", i.e., massive correlated alerts. Although these alerts can be traced back to a few root causes, the overwhelming number makes it infeasible for manual handling. Alert aggregation is thus critical to help engineers concentrate on the root cause and facilitate failure resolution. Existing methods typically utilize semantic similarity-based methods or statistical methods to aggregate alerts. However, semantic similarity-based methods overlook the causal rationale of alerts, while statistical methods can hardly handle infrequent alerts. To tackle these limitations, we introduce leveraging external knowledge, i.e., Standard Operation Procedure (SOP) of alerts as a supplement. We propose COLA, a novel hybrid approach based on correlation mining and LLM (Large Language Model) reasoning for online alert aggregation. The correlation mining module effectively captures the temporal and spatial relations between alerts, measuring their correlations in an efficient manner. Subsequently, only uncertain pairs with low confidence are forwarded to the LLM reasoning module for detailed analysis. This hybrid design harnesses both statistical evidence for frequent alerts and the reasoning capabilities of computationally intensive LLMs, ensuring the overall efficiency of COLA in handling large volumes of alerts in practical scenarios. We evaluate COLA on three datasets collected from the production environment of a large-scale cloud platform. The experimental results show COLA achieves F1-scores from 0.901 to 0.930, outperforming state-of-the-art methods and achieving comparable efficiency. We also share our experience in deploying COLA in our real-world cloud system, Cloud X1 Copyright © 2024, The Authors. All rights reserved.

关键词： Large datasets

来源：评论

学校读者我要写书评

暂无评论

Knowledge-aware Alert Aggregation in Large-scale Cloud Systems: a Hybrid Approach

Knowledge-aware Alert Aggregation in Large-scale Cloud Syste...

引用

IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP)

作者： Jinxi Kuang Jinyang Liu Junjie Huang Renyi Zhong Jiazhen Gu Lan Yu Rui Tan Zengyin Yang Michael R. Lyu The Chinese University of Hong Kong Hong Kong SAR China Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd China

ISBN: (数字)9798400705014

ISBN: (纸本)9798350352177

Due to the scale and complexity of cloud systems, a system fail-ure would trigger an “alert storm”, i.e., massive correlated alerts. Although these alerts can be traced back to a few root causes, the overwhelming number makes it infeasible for manual handling. Alert aggregation is thus critical to help engineers concentrate on the root cause and facilitate failure resolution. Existing methods typically utilize semantic similarity-based methods or statistical methods to aggregate alerts. However, semantic similarity-based methods overlook the causal rationale of alerts, while statistical methods can hardly handle infrequent alerts. To tackle these limitations, we introduce leveraging external knowledge, i.e., Standard Operation Procedure (SOP) of alerts as a supplement. We propose COLA, a novel hybrid approach based on correlation mining and LLM (Large Language Model) reasoning for online alert aggregation. The correlation mining module effectively captures the temporal and spatial relations between alerts, measuring their correlations in an efficient manner. Subsequently, only uncertain pairs with low confidence are forwarded to the LLM rea-soning module for detailed analysis. This hybrid design harnesses both statistical evidence for frequent alerts and the reasoning ca-pabilities of computationally intensive LLMs, ensuring the overall efficiency of COLA in handling large volumes of alerts in practical scenarios. We evaluate COLA on three datasets collected from the production environment of a large-scale cloud platform. The exper-imental results show COLA achieves F1-scores from 0.901 to 0.930, outperforming state-of-the-art methods and achieving comparable efficiency. We also share our experience in deploying COLA in our real-world cloud system, Cloud x 1 1 Due to the company policy, we anonymize the name as Cloud x..

关键词： Correlation Statistical analysis Storms Aggregates Semantics Production Cognition

来源：评论

学校读者我要写书评

暂无评论

FaultProfIT: Hierarchical Fault Profiling of Incident Tickets in Large-Scale Cloud Systems

FaultProfIT: Hierarchical Fault Profiling of Incident Ticket...

引用

IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP)

作者： Junjie Huang Jinyang Liu Zhuangbin Chen Zhihan Jiang Yichen Li Jiazhen Gu Cong Feng Zengyin Yang Yongqiang Yang Michael R. Lyu The Chinese University of Hong Kong Hong Kong China Sun Yat-sen University China Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd

ISBN: (数字)9798400705014

ISBN: (纸本)9798350352177

Postmortem analysis is essential in the management of incidents within cloud systems, which provides valuable insights to improve system's reliability and robustness. At CloudA 1 1 Due to the company policy, we anonymize the name as CloudA., fault pattern profiling is performed during the postmortem phase, which involves the classification of incidents' faults into unique categories, referred to as fault pattern. By aggregating and analyzing these fault patterns, engineers can discern common faults, vulnerable components and emerging fault trends. However, this process is currently conducted by manual labeling, which has inherent drawbacks. On the one hand, the sheer volume of incidents means only the most severe ones are analyzed, causing a skewed overview of fault patterns. On the other hand, the complexity of the task demands extensive domain knowledge, which leads to errors and inconsistencies. To address these limitations, we propose an automated approach, named FaultProflT, for Fault pattern Profiling of Incident Tickets. It leverages hierarchy-guided contrastive learning to train a hierarchy-aware incident encoder and predicts fault patterns with enhanced incident representations. We evaluate FaultProflT using the production incidents from CloudA. The results demonstrate that Fault-Prof IT outperforms state-of-the-art methods. Our ablation study and analysis also verify the effectiveness of hierarchy-guided contrastive learning. Additionally, we have deployed FaultProf IT at CloudA for six months. To date, FaultProflT has analyzed 10,000+ incidents from 30+ cloud services, successfully revealing several fault trends that have informed system improvements.

关键词： Text categorization Taxonomy Production System improvement Manuals Predictive models Market research

来源：评论

学校读者我要写书评

暂无评论

FaultProfIT: Hierarchical Fault Profiling of Incident Tickets in Large-scale Cloud Systems

arXiv

引用

arXiv 2024年

作者： Jiang, Zhihan Li, Yichen Gu, Jiazhen Feng, Cong Yang, Zengyin Yang, Yongqiang Lyu, Michael R. Huang, Junjie Liu, Jinyang Chen, Zhuangbin The Chinese University of Hong Kong Hong Kong Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd. China Sun Yat-sen University China

Postmortem analysis is essential in the management of incidents within cloud systems, which provides valuable insights to improve system's reliability and robustness. At CloudA1, fault pattern profiling is performed during the postmortem phase, which involves the classification of incidents' faults into unique categories, referred to as fault pattern. By aggregating and analyzing these fault patterns, engineers can discern common faults, vulnerable components and emerging fault trends. However, this process is currently conducted by manual labeling, which has inherent drawbacks. On the one hand, the sheer volume of incidents means only the most severe ones are analyzed, causing a skewed overview of fault patterns. On the other hand, the complexity of the task demands extensive domain knowledge, which leads to errors and inconsistencies. To address these limitations, we propose an automated approach, named FaultProfIT, for Fault pattern Profiling of Incident Tickets. It leverages hierarchy-guided contrastive learning to train a hierarchy-aware incident encoder and predicts fault patterns with enhanced incident representations. We evaluate FaultProfIT using the production incidents from CloudA. The results demonstrate that FaultProfIT outperforms state-of-the-art methods. Our ablation study and analysis also verify the effectiveness of hierarchy-guided contrastive learning. Additionally, we have deployed FaultProfIT at CloudA for six months. To date, FaultProfIT has analyzed 10,000+ incidents from 30+ cloud services, successfully revealing several fault trends that have informed system improvements. © 2024, CC BY-NC-ND.

关键词： Domain Knowledge

来源：评论

学校读者我要写书评

暂无评论

FEDGE: An Interference-Aware QoS Prediction Framework for Black-Box Scenario in IaaS Clouds with Domain Generalization

FEDGE: An Interference-Aware QoS Prediction Framework for Bl...

引用

International Symposium on Parallel and Distributed Processing (IPDPS)

作者： Yunlong Cheng Xiuqi Huang Zifeng Liu Jiadong Chen Xiaofeng Gao Zhen Fang Yongqiang Yang Department of Computer Science and Engineering MoE Key Lab of Artificial Intelligence Shanghai Jiao Tong University Shanghai China Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd China

ISBN: (数字)9798350387117

ISBN: (纸本)9798350387124

Public cloud providers embrace multi-tenancy as a strategy to enhance the utilization and efficiency of resources. However, co-located virtual machines (VMs) suffer from qualityof-service (QoS) degradation caused by shared resource interference. Existing solutions for predicting QoS degradation often rely on the assumption of online access to application-level information. However, in a production environment, this assumption proves invalid as the VMs are black boxes to the providers. This intrinsic characteristic of the IaaS cloud necessitates the prediction model to generalize to unfamiliar applications and imposes specific criteria on the monitorable *** meet the black-box scenario under Infrastructure as a Service (IaaS) cloud computing, we present a novel framework, FEDGE, that can predict interference-aware QoS (IA-QoS) of co-located VMs using only low-level monitorable metrics before migration. Specifically, FEDGE utilizes a stochastic gates layer to select the most informative features from the high-dimensional resource and hardware metrics, which helps to reduce the monitoring overhead. Furthermore, we design a multi-domain MMD-based adversarial denoising autoencoder to regularize the learned hidden representations and prevent over-fitting on the source domains. Next, we employ a multi-layer perceptron (MLP) to accurately predict complex QoS degradation using the learned representations with domain generalization. Experimental results demonstrate that FEDGE outperforms other state-of-the-art methods in terms of both generalizability and effectiveness.

关键词： Measurement Degradation Cloud computing Closed box Stochastic processes Quality of service Production

来源：评论

学校读者我要写书评

暂无评论

Prism: Revealing Hidden Functional Clusters from Massive Instances in Cloud Systems 23

Prism: Revealing Hidden Functional Clusters from Massive Ins...

引用

Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering

作者： Jinyang Liu Zhihan Jiang Jiazhen Gu Junjie Huang Zhuangbin Chen Cong Feng Zengyin Yang Yongqiang Yang Michael R. Lyu The Chinese University of Hong Kong Hong Kong SAR China School of Software Engineering Sun Yat-sen University Zhuhai China Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd China

ISBN: (纸本)9798350329964

Ensuring the reliability of cloud systems is critical for both cloud vendors and customers. Cloud systems often rely on virtualization techniques to create instances of hardware resources, such as virtual machines. However, virtualization hinders the observability of cloud systems, making it challenging to diagnose platform-level issues. To improve system observability, we propose to infer functional clusters of instances, i.e., groups of instances having similar functionalities. We first conduct a pilot study on a large-scale cloud system, i.e., Huawei Cloud, demonstrating that instances having similar functionalities share similar communication and resource usage patterns. Motivated by these findings, we formulate the identification of functional clusters as a clustering problem and propose a non-intrusive solution called Prism. Prism adopts a coarse-to-fine clustering strategy. It first partitions instances into coarse-grained chunks based on communication patterns. Within each chunk, Prism further groups instances with similar resource usage patterns to produce fine-grained functional clusters. Such a design reduces noises in the data and allows Prism to process massive instances efficiently. We evaluate Prism on two datasets collected from the real-world production environment of Huawei Cloud. Our experiments show that Prism achieves a v-measure of ~0.95, surpassing existing state-of-the-art solutions. Additionally, we illustrate the integration of Prism within monitoring systems for enhanced cloud reliability through two real-world use cases.

关键词： functional clusters

来源：评论

学校读者我要写书评

暂无评论

Practical Anomaly Detection over Multivariate Monitoring Metrics for Online Services

Practical Anomaly Detection over Multivariate Monitoring Met...

引用

International Symposium on Software Reliability Engineering (ISSRE)

作者： Jinyang Liu Tianyi Yang Zhuangbin Chen Yuxin Su Cong Feng Zengyin Yang Michael R. Lyu The Chinese University of Hong China School of Software Engineering Sun Yat-sen University Zhuhai China Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd China

As modern software systems continue to grow in terms of complexity and volume, anomaly detection on multivariate monitoring metrics, which profile systems’ health status, becomes more and more critical and challenging. In particular, the dependency between different metrics and their historical patterns plays a critical role in pursuing prompt and accurate anomaly detection. Existing approaches fall short of industrial needs for being unable to capture such information efficiently. To fill this significant gap, in this paper, we propose CMAnomaly, an anomaly detection framework on multivariate monitoring metrics based on collaborative machine. The proposed collaborative machine is a mechanism to capture the pairwise interactions along with feature and temporal dimensions with linear time complexity. Cost-effective models can then be employed to leverage both the dependency between monitoring metrics and their historical patterns for anomaly detection. The proposed framework is extensively evaluated with both public data and industrial data collected from a large-scale online service system of Huawei Cloud. The experimental results demonstrate that compared with state-of-the-art baseline models, CMAnomaly achieves an average F1 score of 0.9494, outperforming baselines by 6.77% ~ 10.68%, and runs 10× ~ 20× faster. Furthermore, we also share our experience of deploying CMAnomaly in Huawei Cloud.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Practical Anomaly Detection over Multivariate Monitoring Metrics for Online Services

arXiv

引用

arXiv 2023年

作者： Liu, Jinyang Yang, Tianyi Chen, Zhuangbin Su, Yuxin Feng, Cong Yang, Zengyin Lyu, Michael R. The Chinese University of Hong Kong Hong Kong School of Software Engineering Sun Yat-sen University Zhuhai China Computing and Networking Innovation Lab Huawei Cloud Computing Technology Co. Ltd China

As modern software systems continue to grow in terms of complexity and volume, anomaly detection on multivariate monitoring metrics, which profile systems’ health status, becomes more and more critical and challenging. In particular, the dependency between different metrics and their historical patterns plays a critical role in pursuing prompt and accurate anomaly detection. Existing approaches fall short of industrial needs for being unable to capture such information efficiently. To fill this significant gap, in this paper, we propose CMAnomaly, an anomaly detection framework on multivariate monitoring metrics based on collaborative machine. The proposed collaborative machine is a mechanism to capture the pairwise interactions along with feature and temporal dimensions with linear time complexity. Cost-effective models can then be employed to leverage both the dependency between monitoring metrics and their historical patterns for anomaly detection. The proposed framework is extensively evaluated with both public data and industrial data collected from a large-scale online service system of Huawei Cloud. The experimental results demonstrate that compared with state-of-the-art baseline models, CMAnomaly achieves an average F1 score of 0.9494, outperforming baselines by 6.77% ∼ 10.68%, and runs 10× ∼ 20× faster. Furthermore, we also share our experience of deploying CMAnomaly in Huawei Cloud. © 2023, CC BY-NC-ND.

关键词： Software reliability

来源：评论

学校读者我要写书评

暂无评论

Enhanced Multidimensional Harmonic Retrieval in MIMO Wireless Channel Sounding

引用

IEEE Internet of Things Journal 2025年第11期12卷 16243-16255页

作者： Zhang, Yanming Xu, Wenchao Jin, A-Long Tang, Tianquan Li, Min Ma, Peifeng Jiang, Lijun Gao, Steven The Chinese University of Hong Kong Department of Electronic Engineering Hong Kong Hong Kong The Hong Kong Polytechnic University Department of Computing Hong Kong Hong Kong Xi'an Jiaotong-Liverpool University Department of Communications and Networking Suzhou China HKU Zhejiang Institute of Research and Innovation Lab for Aerodynamics and Acoustics Hangzhou China Tianjin University School of Microelectronics Tianjin China The Chinese University of Hong Kong Institute of Space and Earth Information Science Department of Geography and Resource Management Hong Kong Hong Kong Missouri University of Science and Technology Department of Electrical and Computer Engineering RollaMO United States

This paper introduces a recursive parallel dynamic mode decomposition (RPDMD) scheme tailored for multidimensional harmonic retrieval (MHR), specifically applied to MIMO wireless channel sounding. The RPDMD algorithm is devised to address the complexities inherent in multidimensional scenarios, leveraging the dynamic mode decomposition (DMD) framework within a recursive parallel structure. Initially, the observed tensorial multidimensional harmonic data is transformed into a two-dimensional matrix format along the r-th dimension. Subsequently, DMD dissects this matrix data into eigenvalues and their associated modes. The real and imaginary components of the DMD eigenvalues yield damping factors and frequencies in the r-th dimension, respectively. Furthermore, recursive DMD is employed to scrutinize each mode independently for parameter retrieval across the remaining dimensions, enabling parallel analysis. Ultimately, this high-dimensional correlated decomposition scheme delivers paired damping factors and frequencies for all tones. Notably, the proposed approach can ascertain the number of tones in undamped sinusoidal signals, making it particularly suitable for MHR even without prior knowledge of the source count. Numerical experiments demonstrate the accuracy and robustness of the RPDMD scheme, with comparative analysis indicating that RPDMD outperforms similar methods, achieving optimal results with minimal mean square error in high signal-to-noise ratio scenarios. This work presents an effective data-driven solution for the MHR problem in MIMO wireless channel sounding. © 2014 IEEE.

关键词： Eigenvalues and eigenfunctions

来源：评论

学校读者我要写书评

暂无评论

Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention

arXiv

引用

arXiv 2023年

作者： Lee, Cheryl Yang, Tianyi Chen, Zhuangbin Su, Yuxin Yang, Yongqiang Lyu, Michael R. Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong Sun Yat-sen University Guangzhou China Computing and Networking Innovation Lab Cloud BU Huawei

Prompt and accurate detection of system anomalies is essential to ensure the reliability of software systems. Unlike manual efforts that exploit all available run-time information, existing approaches usually leverage only a single type of monitoring data (often logs or metrics) or fail to make effective use of the joint information among different types of data. Consequently, many false predictions occur. To better understand the manifestations of system anomalies, we conduct a systematical study on a large amount of heterogeneous data, i.e., logs and metrics. Our study demonstrates that logs and metrics can manifest system anomalies collaboratively and complementarily, and neither of them only is sufficient. Thus, integrating heterogeneous data can help recover the complete picture of a system’s health status. In this context, we propose Hades, the first end-to-end semi-supervised approach to effectively identify system anomalies based on heterogeneous data. Our approach employs a hierarchical architecture to learn a global representation of the system status by fusing log semantics and metric patterns. It captures discriminative features and meaningful interactions from heterogeneous data via a cross-modal attention module, trained in a semi-supervised manner. We evaluate Hades extensively on large-scale simulated data and datasets from Huawei Cloud. The experimental results present the effectiveness of our model in detecting system anomalies. We also release the code and the annotated dataset for replication and future research. © 2023, CC0.

关键词： Anomaly detection

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：