检索结果-内蒙古大学图书馆

IEEE International Conference on software Analysis, Evolution and Reengineering (SANER)

作者： Zhi Chen Lingxiao Jiang Centre for Research on Intelligent Software Engineering School of Computing and Information Systems Singapore Management University

ISBN: (数字)9798331535100

ISBN: (纸本)9798331535117

In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with software Development Agents representing the next major leap. These agents, capable of reasoning, planning, and interacting with external environments, offer promising solutions to complex software engineering tasks. However, while much research has evaluated code generated by large language models (LLMs), comprehensive studies on agent-generated patches, particularly in real-world settings, are lacking. This study addresses that gap by evaluating 4,892 patches from 10 top-ranked agents on 500 real-world GitHub issues from SWE-Bench Verified, focusing on their impact on code quality. Our analysis shows no single agent dominated, with 170 issues unresolved, indicating room for improvement. Even for patches that passed unit tests and resolved issues, agents made different file and function modifications compared to the gold patches from repository developers, revealing limitations in the benchmark's test case coverage. Most agents maintained code reliability and security, avoiding new bugs or vulnerabilities; while some agents increased code complexity, many reduced code duplication and minimized code smells. Finally, agents performed better on simpler codebases, suggesting that breaking complex tasks into smaller sub-tasks could improve effectiveness. This study provides the first comprehensive evaluation of agent-generated patches on real-world GitHub issues, offering insights to advance AI-driven software development.

关键词： Gold Codes Large language models Computer bugs Focusing Complexity theory Security Reliability software development management software engineering

来源：评论

学校读者我要写书评

暂无评论

Evaluating software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios

arXiv

引用

arXiv 2024年

作者： Chen, Zhi Jiang, Lingxiao Centre for Research on Intelligent Software Engineering School of Computing and Information Systems Singapore Management University Singapore

In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with software Development Agents representing the next major leap. These agents, capable of reasoning, planning, and interacting with external environments, offer promising solutions to complex software engineering tasks. However, while much research has evaluated code generated by large language models (LLMs), comprehensive studies on agent-generated patches, particularly in real-world settings, are lacking. This study addresses that gap by evaluating 4,892 patches from 10 top-ranked agents on 500 real-world GitHub issues from SWE-Bench Verified, focusing on their impact on code quality. Our analysis shows no single agent dominated, with 170 issues unresolved, indicating room for improvement. Even for patches that passed unit tests and resolved issues, agents made different file and function modifications compared to the gold patches from repository developers, revealing limitations in the benchmark’s test case coverage. Most agents maintained code reliability and security, avoiding new bugs or vulnerabilities;while some agents increased code complexity, many reduced code duplication and minimized code smells. Finally, agents performed better on simpler codebases, suggesting that breaking complex tasks into smaller sub-tasks could improve effectiveness. This study provides the first comprehensive evaluation of agent-generated patches on real-world GitHub issues, offering insights to advance AI-driven software development. © 2024, CC BY.

关键词： software design

来源：评论

学校读者我要写书评

暂无评论

Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization

arXiv

引用

arXiv 2024年

作者： Chen, Zhi Jiang, Lingxiao Centre for Research on Intelligent Software Engineering School of Computing and Information Systems Singapore Management University Singapore

In the rapidly evolving field of machine learning, training models with datasets from various locations and organizations presents significant challenges due to privacy and legal concerns. The exploration of effective collaborative training settings, which are capable of leveraging valuable knowledge from distributed and isolated datasets, is increasingly *** study investigates key factors that impact the effectiveness of collaborative training methods in code next-token prediction, as well as the correctness and utility of the generated code, showing the promise of such methods. Additionally, we evaluate the memorization of different participant training data across various collaborative training settings, including centralized, federated, and incremental training, showing their potential risks in leaking data. Our findings indicate that the size and diversity of code datasets are pivotal factors influencing the success of collaborative trained code models. We demonstrate that federated learning achieves competitive performance compared to centralized training while offering better data protection, as evidenced by lower memorization ratios in the generated code. However, federated learning can still produce verbatim code snippets from hidden training data, potentially violating data privacy or copyright. Our study further explores the patterns of effectiveness and memorization in incremental learning, emphasizing the importance of the sequence in which individual participant datasets are introduced. Also, we identify the memorization phenomenon of cross-organizational clones as a prevalent challenge in both centralized and federated learning scenarios. Our findings highlight the persistent risk of data leakage during inference, even when training data remains unseen. We conclude with strategic recommendations for practitioners and researchers to optimize the use of multisource datasets, thereby propelling the cross-organizational collaboration forward. Cop

关键词： Collaborative learning

来源：评论

学校读者我要写书评

暂无评论

SYSTEMATIC LITERATURE REVIEW ON APPLICATION OF LEARNING-BASED APPROACHES IN CONTINUOUS INTEGRATION

arXiv

引用

arXiv 2024年

作者： Arani, Ali Kazemi Le, Triet Huynh Minh Zahedi, Mansooreh Babar, M. Ali CREST-The Centre for Research on Engineering Software Technologies University of Adelaide AdelaideSA5005 Australia School of Computing and Information Systems University of Melbourne MelbourneVIC3010 Australia

Context: Machine learning (ML) and deep learning (DL) analyze raw data to extract valuable insights in specific phases. The rise of continuous practices in software projects emphasizes automating Continuous Integration (CI) with these learning-based methods, while the growing adoption of such approaches underscores the need for systematizing knowledge. Objective: Our objective is to comprehensively review and analyze existing literature concerning learning-based methods within the CI domain. We endeavour to identify and analyse various techniques documented in the literature, emphasizing the fundamental attributes of training phases within learning-based solutions in the context of CI. Method: We conducted a Systematic Literature Review (SLR) involving 52 primary studies. Through statistical and thematic analyses, we explored the correlations between CI tasks and the training phases of learning-based methodologies across the selected studies, encompassing a spectrum from data engineering techniques to evaluation metrics. Results: This paper presents an analysis of the automation of CI tasks utilizing learning-based methods. We identify and analyze nine types of data sources, four steps in data preparation, four feature types, nine subsets of data features, five approaches for hyperparameter selection and tuning, and fifteen evaluation metrics. Furthermore, we discuss the latest techniques employed, existing gaps in CI task automation, and the characteristics of the utilized learning-based techniques. Conclusion: This study provides a comprehensive overview of learning-based methods in CI, offering valuable insights for researchers and practitioners developing CI task automation. It also highlights the need for further research to advance these methods in CI. © 2024, CC BY.

关键词： Automation

来源：评论

学校读者我要写书评

暂无评论

Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddings

Data-Driven Distributionally Robust Safety Verification Usin...

引用

American Control Conference (ACC)

作者： Oliver Schön Zhengang Zhong Sadegh Soudjani School of Computing Newcastle University Newcastle upon Tyne United Kingdom Sargent Centre for Process Systems Engineering Imperial College London London United Kingdom Max Planck Institute for Software Systems Kaiserslautern Germany

ISBN: (数字)9798350382655

ISBN: (纸本)9798350382662

Algorithmic verification of realistic systems to sat-isfy safety and other temporal requirements has suffered from poor scalability of the employed formal approaches. To design systems with rigorous guarantees, many approaches still rely on exact models of the underlying systems. Since this assumption can rarely be met in practice, models have to be inferred from measurement data or are bypassed completely. Whilst former usually requires the model structure to be known a-priori and immense amounts of data to be available, latter gives rise to a plethora of restrictive mathematical assumptions about the unknown dynamics. In a pursuit of developing scalable formal verification algorithms without shifting the problem to unrealistic assumptions, we employ the concept of barrier certificates, which can guarantee safety of the system, and learn the certificate directly from a compact set of system trajectories. We use conditional mean embeddings to embed data from the system into a reproducing kernel Hilbert space (RKHS) and construct an RKHS ambiguity set that can be inflated to robustify the result w.r.t. a set of plausible transition kernels. We show how to solve the resulting program efficiently using sum-of-squares optimization and a Gaussian process envelope. Our approach lifts the need for restrictive assumptions on the system dynamics and uncertainty, and suggests an improvement in the sample complexity of verifying the safety of a system on a tested case study compared to a state-of-the-art approach.

关键词： Uncertainty System dynamics Scalability Mathematical models Data models Hilbert space Safety

来源：评论

学校读者我要写书评

暂无评论

Mitigating ML Model Decay in Continuous Integration with Data Drift Detection: An Empirical Study

arXiv

引用

arXiv 2023年

作者： Arani, Ali Kazemi Babar, Muhammad Ali Le, Triet Huynh Minh Zahedi, Mansooreh CREST-The Centre for Research on Engineering Software Technologies School of Computer Science University of Adelaide Adelaide Australia School of Computing and Information Systems University of Melbourne Melbourne Australia

Background: Machine Learning (ML) methods are being increasingly used for automating different activities, e.g., Test Case Prioritization (TCP), of Continuous Integration (CI). However, ML models need frequent retraining as a result of changes in the CI environment, more commonly known as data drift. Also, continuously retraining ML models consume a lot of time and effort. Hence, there is an urgent need of identifying and evaluating suitable approaches that can help in reducing the retraining efforts and time for ML models used for TCP in CI environments. Aims: This study aims to investigate the performance of using data drift detection techniques for automatically detecting the retraining points for ML models for TCP in CI environments without requiring detailed knowledge of the software projects. Method: We employed the Hellinger distance to identify changes in both the values and distribution of input data, and leveraged these changes as retraining points for the ML model. We evaluated the efficacy of this method on multiple datasets and compared the APFDc and NAPFD evaluation metrics against models that were regularly retrained, with careful consideration of the statistical methods. Results: Our experimental evaluation of the Hellinger distance-based method demonstrated its efficacy and efficiency in detecting retraining points and reducing the associated costs. However, the performance of this method may vary depending on the dataset. Conclusions: Our findings suggest that data drift detection methods can assist in identifying retraining points for ML models in CI environments, while significantly reducing the required retraining time. These methods can be helpful for practitioners who lack specialized knowledge of software projects, enabling them to maintain ML model accuracy. © 2023, CC BY.

关键词： Transmission control protocol

来源：评论

学校读者我要写书评

暂无评论

Examining the Use of Non-fungible Tokens (NFTs) as a Trading Mechanism for the Metaverse 1

引用

29th European systems, software and Services Process Improvement, EuroSPI 2022

作者： Yilmaz, Murat Hacaloğlu, Tuna Clarke, Paul Department of Computer Engineering Engineering Faculty Gazi University Ankara Turkey Metaverse Research and Development Laboratory Gazi University Ankara Turkey Department of Information Systems Engineering Atilim University Ankara Turkey School of Computing Dublin City University Dublin Ireland Lero the Science Foundation Ireland Research Centre for Software Limerick Ireland

ISBN: (数字)9783031155598

ISBN: (纸本)9783031155581

The notion of a metaverse seems hard to define but encourages the impression that it can be considered as a new virtual metaphysical landscape that somehow goes beyond our geographical locations and understanding (i.e., independent of time and space). Based on virtual reality, augmented reality, and blockchain, it is envisioned as an independent but extended world that is planned to be a digital virtuality entrenched not only in our old habits such as gaming and entertainment but also in virtual asset trade. In particular, trading is a pillar of the virtual economy, and auction houses will be crucial for Metaverse trading. This exploratory study examines the possibility of using an auction environment to improve the trading capabilities in a virtual universe. We investigate the cases of creating a virtual auction house with the potential of social trading of virtual assets with crypto coins and bartering. To this end, we built a virtual auction house and tested it initially using a set of scenarios. Our preliminary findings suggest that creating a virtual trading environment would be beneficial as an environment for buying and selling virtual assets and exploring their consequences. © 2022, Springer Nature Switzerland AG.

关键词： Augmented reality

来源：评论

学校读者我要写书评

暂无评论

Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddings

arXiv

引用

arXiv 2024年

作者： Schön, Oliver Zhong, Zhengang Soudjani, Sadegh The School of Computing Newcastle University Newcastle upon TyneNE4 5TG United Kingdom The Sargent Centre for Process Systems Engineering Imperial College London London United Kingdom The Max Planck Institute for Software Systems Kaiserslautern Germany

Algorithmic verification of realistic systems to satisfy safety and other temporal requirements has suffered from poor scalability of the employed formal approaches. To design systems with rigorous guarantees, many approaches still rely on exact models of the underlying systems. Since this assumption can rarely be met in practice, models have to be inferred from measurement data or are bypassed completely. Whilst former usually requires the model structure to be known a-priori and immense amounts of data to be available, latter gives rise to a plethora of restrictive mathematical assumptions about the unknown dynamics. In a pursuit of developing scalable formal verification algorithms without shifting the problem to unrealistic assumptions, we employ the concept of barrier certificates, which can guarantee safety of the system, and learn the certificate directly from a compact set of system trajectories. We use conditional mean embeddings to embed data from the system into a reproducing kernel Hilbert space (RKHS) and construct an RKHS ambiguity set that can be inflated to robustify the result w.r.t. a set of plausible transition kernels. We show how to solve the resulting program efficiently using sum-of-squares optimization and a Gaussian process envelope. Our approach lifts the need for restrictive assumptions on the system dynamics and uncertainty, and suggests an improvement in the sample complexity of verifying the safety of a system on a tested case study compared to a state-of-the-art approach. © 2024, CC BY-NC-ND.

关键词： Formal verification

来源：评论

学校读者我要写书评

暂无评论

SYSTEMATIC LITERATURE REVIEW ON APPLICATION OF MACHINE LEARNING IN CONTINUOUS INTEGRATION

arXiv

引用

arXiv 2023年

作者： Arani, Ali Kazemi Le, Triet Huynh Minh Zahedi, Mansooreh Babar, Muhammad Ali CREST-The Centre for Research on Engineering Software Technologies School of Computer Science University of Adelaide AdelaideSA Australia School of Computing and Information Systems University of Melbourne MelbourneVIC Australia

Context: Machine learning (ML) is a field that involves analysing raw data and extracting useful information from it through specific phases. As continuous practices become more prevalent in software projects, there is a need to explore how ML methods can be trained to enhance the Continuous Integration (CI) pipeline. Moreover, the growing utilization of ML algorithms in CI, combined with the surge of literature on the subject, highlights the significance of establishing a comprehensive body of knowledge to support future researchers in conducting high-quality research and bridging any existing gaps. Objective: The objective of this research is to conduct a systematic review and analysis of the existing literature on ML-based methods employed in the CI domain. This study aims to identify and describe the various techniques employed in the literature and present the key characteristics of the training phases of ML-based solutions in the CI context. Method: To achieve this objective, we conducted a Systematic Literature Review (SLR) of 48 primary studies that were selected after searching relevant literature published over the past 22 years (2000–November 2022). We used statistical and thematic analysis to examine the composition phase of CI, data engineering techniques and data source types, feature engineering methods and extracted features, the employed hyper-parameter tuning methods and types of the ML models, and the evaluation methods and metrics used in the selected studies. Additionally, this article aims to present the relationship between these concepts. Results: In this paper, we have depicted the phases of CI testing, the connection between them, and the employed techniques in training the ML method phases. We presented nine types of data sources and four taken steps in the selected studies for preparing the data. Also, we identified four feature types and nine subsets of data features through thematic analysis of the selected studies. Besides, five method

关键词： Machine learning

来源：评论

学校读者我要写书评

暂无评论

Goal-Modeling Privacy-by-Design Patterns for Supporting GDPR Compliance

Goal-Modeling Privacy-by-Design Patterns for Supporting GDPR...

引用

18th International Conference on software Technologies, ICSOFT 2023

作者： Al-Obeidallah, Mohammed Ghazi Piras, Luca Iloanugo, Onyinye Mouratidis, Haralambos Alkubaisy, Duaa Dellagiacoma, Daniele Department of Software Engineering Al Ain University Abu Dhabi United Arab Emirates School of Computing Middlesex University London United Kingdom School of Computing Robert Gordon University Aberdeen United Kingdom Institute for Analytics and Data Science University of Essex Colchester United Kingdom College of Applied Studies and Community Service Imam Abdulrahman Bin Faisal University Dammam Saudi Arabia Centre for Secure Intelligent and Usable Systems University of Brighton Brighton United Kingdom

ISBN: (纸本)9789897586651

The introduction of the European General Data Protection Regulation (GDPR) has imposed obligations on organisations collecting data in the EU. This has been beneficial to citizens due to rights reinforcement achieved as data subjects. However, obligations heavily affected organisations, and their privacy requirements analysts, having issues with interpreting and implementing GDPR principles. This paper proposes visual GDPR Patterns supporting analysts through Privacy-by- Design (PbD) and GDPR compliance analysis. In order to achieve that, we extended a requirements modeling tool, SecTro, which is used to assist analysts in creating visual requirements models. Specifically, we extended SecTro with novel visual GDPR patterns representing GDPR principles. We evaluated the patterns in a healthcare case study. The evaluation results suggest that the GDPR patterns can help analysts in PbD modeling analysis, by representing GDPR principles and considering relevant ready-to-use alternatives, towards achieving GDPR compliance. © 2023 by SCITEPRESS-Science and Technology Publications, Lda.

关键词：

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：