检索结果-内蒙古大学图书馆

What Causes Exceptions in Machine Learning Applications? Mining Machine Learning-Related Stack Traces on Stack Overflow

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Ghadesi, Amin Lamothe, Maxime Li, Heng The Department of Computer Engineering and Software Engineering Polytechnique Montreal Canada

Machine learning (ML), including deep learning, has recently gained tremendous popularity in a wide range of applications. However, like traditional software, ML applications are not immune to the bugs that result from programming errors. Explicit programming errors usually manifest through error messages and stack traces. These stack traces describe the chain of function calls that lead to an anomalous situation, or exception. Indeed, these exceptions may cross the entire software stack (including applications and libraries). Thus, studying the patterns in stack traces can help practitioners and researchers understand the causes of exceptions in ML applications and the challenges faced by ML developers. To that end, we mine Stack Overflow (SO) and study 11,449 stack traces related to seven popular Python ML libraries. First, we observe that ML questions that contain stack traces gain more popularity than questions without stack traces;however, they are less likely to get accepted answers. Second, we observe that recurrent patterns exists in ML stack traces, even across different ML libraries, with a small portion of patterns covering many stack traces. Third, we derive five high-level categories and 25 low-level types from the stack trace patterns: most patterns are related to python basic syntax, model training, parallelization, data transformation, and subprocess invocation. Furthermore, the patterns related to subprocess invocation, external module execution, and remote API call are among the least likely to get accepted answers on SO. Our findings provide insights for researchers, ML library providers, and ML application developers to improve the quality of ML libraries and their applications. © 2023, CC BY-SA.

关键词： Libraries

EvoChain: A Framework for Tracking and Visualizing Smart Contract Evolution

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Qasse, Ilham Hamdaqa, Mohammad Jónsson, Björn Þór Department of Computer Science Reykjavik University Reykjavik Iceland Department of Computer and Software Engineering Polytechnique Montreal Montreal Canada

Tracking the evolution of smart contracts is challenging due to their immutable nature and complex upgrade mechanisms. We introduce EvoChain, a comprehensive framework and dataset designed to track and visualize smart contract evolution. Building upon data from our previous empirical study, EvoChain models contract relationships using a Neo4j graph database and provides an interactive web interface for exploration. The framework consists of a data layer, an API layer, and a user interface layer. EvoChain allows stakeholders to analyze contract histories, upgrade paths, and associated vulnerabilities by leveraging these components. Our dataset encompasses approximately 1.3 million upgradeable proxies and nearly 15,000 historical versions, enhancing transparency and trust in blockchain ecosystems by providing an accessible platform for understanding smart contract evolution. © 2025, CC BY-NC-SA.

关键词：

Advancing Formal Verification: Fine-Tuning LLMs for Translating Natural Language Requirements to CTL Specifications

学校读者我要写书评

暂无评论

Advancing Formal Verification: Fine-Tuning LLMs for Translat...

International Workshop on Rapid System Prototyping (RSP)

作者： Rim Zrelli Henrique Amaral Misson Maroua Ben Attia Felipe Gohring de Magalhaes Abdo Shabah Gabriela Nicolescu Computer Engineering and Software Engineering Department Polytechnique Montreal Montreal Canada Research & Development Humanitas Solutions Montreal Canada

ISBN: (数字)9798331519537

ISBN: (纸本)9798331519544

In the domain of formal verification, translating natural language (NL) requirements into Computation Tree Logic (CTL) specifications presents a notable challenge due to the disparity between human-readable documents and formal specifications. This paper introduces a novel approach that leverages Large Language Models (LLMs) to automate this translation process, thereby enhancing the accuracy and efficiency of formal verification practices. We fine-tune three state-of-the-art LLMs—LLAMA3, Mistral, and Qwen2—with a particular focus on optimizing the Mistral model due to its superior performance. Our methodology is supported by the Natural2CTL dataset, consisting of 2,095 NL requirements and their corresponding CTL specifications. We employ evaluation metrics such as validation loss, accuracy, semantic similarity, and Structural Operator Jaccard Similarity (SOJS) for a comprehensive assessment of model performance. Additionally, a comparative analysis with human translators, trained in CTL logic, underscores the LLMs’ potential to match or even surpass human accuracy in translating NL requirements into formal specifications. Our findings reveal that the fine-tuned Mistral model significantly outperforms the other LLMs and human participants, demonstrating superior accuracy in generating CTL specifications. This study advances the field of formal verification by proposing a scalable solution to the NL-to-CTL translation challenge, setting a new benchmark for the integration of AI tools in complex specification tasks.

关键词： Measurement Translation Accuracy Large language models Conferences Semantics Benchmark testing Logic Formal specifications Formal verification

Automated, Unsupervised, and Auto-parameterized Inference of Data Patterns and Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Qin, Qiaolin Li, Heng Merlo, Ettore Lamothe, Maxime Dept. of Computer and Software Engineering Polytechnique Montreal Montreal Canada

With the advent of data-centric and machine learning (ML) systems, data quality is playing an increasingly critical role for ensuring the overall quality of software systems. Data preparation, an essential step towards high data quality, is known to be a highly effort-intensive process. Although prior studies have dealt with one of the most impacting issues, data pattern violations, these studies usually require data-specific configurations (i.e., parameterized) or use carefully curated data as learning examples (i.e., supervised), relying on domain knowledge and deep understanding of the data, or demanding significant manual effort. In this paper, we introduce RIOLU: Regex Inferencer autO-parameterized Learning with Uncleaned data. RIOLU is fully automated, automatically parameterized, and does not need labeled samples. RIOLU can generate precise patterns from datasets in various domains, with a high F1 score of 97.2%, exceeding the state-of-the-art baseline. In addition, according to our experiment on five datasets with anomalies, RIOLU can automatically estimate a data column’s error rate, draw normal patterns, and predict anomalies from unlabeled data with higher performance (up to 800.4% improvement in terms of F1) than the state-of-the-art baseline, even outperforming ChatGPT in terms of both accuracy (12.3% higher F1) and efficiency (10% less inference time). A variant of RIOLU, with user guidance, can further boost its precision, with up to 37.4% improvement in terms of F1. Our evaluation in an industrial setting further demonstrates the practical benefits of *** Codes 68P01 © 2024, CC BY.

关键词： Self-supervised learning

Assessing the Security of GitHub Copilot's Generated Code-A Targeted Replication Study 31

学校读者我要写书评

暂无评论

Assessing the Security of GitHub Copilot's Generated Code-A ...

31st IEEE International Conference on software Analysis, Evolution and Reengineering, SANER 2024

作者： Majdinasab, Vahid Bishop, Michael Joshua Rasheed, Shawn Moradidakhel, Arghavan Tahir, Amjed Khomh, Foutse Polytechnique Department of Computer and Software Engineering Montreal Canada School of Mathematical and Computational Sciences Massey University New Zealand Information & Communication Technology Group UCOL-Te Pukenga New Zealand

ISBN: (纸本)9798350330663

AI-powered code generation models have been developing rapidly, allowing developers to expedite code generation and thus improve their productivity. These models are trained on large corpora of code (primarily sourced from public repositories), which may contain bugs and vulnerabilities. Several concerns have been raised about the security of the code generated by these models. Recent studies have investigated security issues in AI-powered code generation tools such as GitHub Copilot and Amazon Code Whisperer, revealing several security weaknesses in the code generated by these tools. As these tools evolve, it is expected that they will improve their security protocols to prevent the suggestion of insecure code to developers. This paper replicates the study of Pearce et al., which investigated security weaknesses in Copilot and uncovered several weaknesses in the code suggested by Copilot across diverse scenarios and languages (Python, C, and Verilog). Our replication examines Copilot's security weaknesses using newer versions of Copilot and CodeQL (the security analysis framework). The replication focused on the presence of security vulnerabilities in Python code. Our results indicate that, with the improvements in newer versions of Copilot, the percentage of vulnerable code suggestions has reduced from 36.54% to 27.25%. Nonetheless, it remains evident that the model still suggests insecure code. © 2024 IEEE.

关键词： Network security

On the Effectiveness of Log Representation for Log-based Anomaly Detection

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Wu, Xingfang Li, Heng Khomh, Foutse Department of Computer Engineering and Software Engineering Polytechnique Montreal MontrealQC Canada

Logs are an essential source of information for people to understand the running status of a software system. Due to the evolving modern software architecture and maintenance methods, more research efforts have been devoted to automated log analysis. In particular, machine learning (ML) has been widely used in log analysis tasks. In ML-based log analysis tasks, converting textual log data into numerical feature vectors is a critical and indispensable step. However, the impact of using different log representation techniques on the performance of the downstream models is not clear, which limits researchers and practitioners’ opportunities of choosing the optimal log representation techniques in their automated log analysis workflows. Therefore, this work investigates and compares the commonly adopted log representation techniques from previous log analysis research. Particularly, we select six log representation techniques and evaluate them with seven ML models and four public log datasets (i.e., HDFS, BGL, Spirit and Thunderbird) in the context of log-based anomaly detection. We also examine the impacts of the log parsing process and the different feature aggregation approaches when they are employed with log representation techniques. From the experiments, we provide some heuristic guidelines for future researchers and developers to follow when designing an automated log analysis workflow. We believe our comprehensive comparison of log representation techniques can help researchers and practitioners better understand the characteristics of different log representation techniques and provide them with guidance for selecting the most suitable ones for their ML-based log analysis workflow. Copyright © 2023, The Authors. All rights reserved.

关键词： Anomaly detection

Studying Logging Practice in Machine Learning-based Applications

学校读者我要写书评

暂无评论

arXiv 2023年

作者： Foalem, Patrick Loic Khomh, Foutse Li, Heng Department of Computer Engineering and Software Engineering Polytechnique Montreal MontrealQC Canada

Logging is a common practice in traditional software development. Several research works have been done to investigate the different characteristics of logging practices in traditional software systems (e.g., Android applications, JAVA applications, C/C++ applications). Nowadays, we are witnessing more and more development of Machine Learning-based applications (ML-based applications). Today, there are many popular libraries that facilitate and contribute to the development of such applications, among which we can mention: Pytorch, Tensorflow, Theano, MXNet, Scikit-Learn, Caffe, and Keras. Despite the popularity of ML, we don’t have a clear understanding of logging practices in ML applications. In this paper, we aim to fill this knowledge gap and help ML practitioners understand the characteristics of logging in ML-based applications. In particular, we conduct an empirical study on 110 open-source ML-based applications. Through a quantitative analysis, we find that logging practice in ML-based applications is less pervasive than in traditional applications including Android, JAVA, and C/C++ applications. Furthermore, the majority of logging statements in ML-based applications are in info and warn levels, compared to traditional applications where info is the majority of logging statement in C/C++ application and debug, error levels constitute the majority of logging statement in Android application. We also perform a quantitative and qualitative analysis of a random sample of logging statements to understand where ML developers put most of logging statements and examine why and how they are using logging. These analyses led to the following observations: (i) ML developers put most of the logging statements in model training, and in non-ML components. (ii) Data and model management appear to be the main reason behind the introduction of logging statements in ML-based applications. Indeed, ML developers use logging statements to keep track of the different stages of d

关键词： Machine learning

Generalization of Urban Wind Environment Using Fourier Neural Operator Across Different Wind Directions and Cities

学校读者我要写书评

暂无评论

arXiv 2025年

作者： Chen, Cheng Tian, Geng Qin, Shaoxiang Geng, Dingyang Zhan, Dongxue Yang, Jinqiu Vidal, David Wang, Liangzhu Concordia University Department of Building Civil and Environmental Engineering Department of Computer Science and Software Engineering Montreal Canada Polytechnique Montreal Department of Mechanical Engineering Montreal Canada

Simulation of urban wind environments is crucial for urban planning, pollution control, and renewable energy utilization. However, the computational requirements of high-fidelity computational fluid dynamics (CFD) methods make them impractical for real cities. To address these limitations, this study investigates the effectiveness of the Fourier Neural Operator (FNO) model in predicting flow fields under different wind directions and urban layouts. In this study, we investigate the effectiveness of the Fourier Neural Operator (FNO) model in predicting urban wind conditions under different wind directions and urban layouts. By training the model on velocity data from large eddy simulation data, we evaluate the performance of the model under different urban configurations and wind conditions. The results show that the FNO model can provide accurate predictions while significantly reducing the computational time by 99%. Our innovative approach of dividing the wind field into smaller spatial blocks for training improves the ability of the FNO model to capture wind frequency features effectively. The SDF data also provides important spatial building information, enhancing the model’s ability to recognize physical boundaries and generate more realistic predictions. The proposed FNO approach enhances the AI model’s generalizability for different wind directions and urban layouts. Copyright © 2025, The Authors. All rights reserved.

关键词： Computational fluid dynamics

Bug Characteristics in Quantum software Ecosystem

学校读者我要写书评

暂无评论

SSRN

SSRN 2023年

作者： El Aoun, Mohamed Raed Li, Heng Khomh, Foutse Tidjon, Lionel Department of Computer Engineering and Software Engineering Polytechnique Montreal MontrealQC Canada

Recent advancements in quantum computing, exemplified by IBM's quantum computers, underscore the importance of quantum software. Differing fundamentally from classical programming, quantum programming's probabilistic state and inherent error-prone nature due to quantum instabilities pose unique challenges. This study investigates bug characteristics in quantum software projects to inform effective testing and debugging strategies. We analyzed bug reports from 125 quantum software projects on GitHub and found them more error-prone and costly to rectify compared to classical software. Key areas where bugs manifested include the compiler, gate operation, and state preparation components. The predominant bug types are Program anomaly, Configuration, and Data type & structure bugs. A notable challenge in quantum software development is the absence of comprehensive quantum computation libraries and specific quantum gate operation definitions. Additionally, there's a demand for dedicated data manipulation libraries akin to Numpy for quantum computing. Our insights spotlight areas for future research, emphasizing the need for specialized debugging tools for quantum circuits. © 2023, The Authors. All rights reserved.

关键词： Quantum computers