Usually, mature Artificial Intelligence (AI) projects are developed by a team of various members, such as data engineers, data scientists, software engineers and machinelearning (ML) engineers. They often pursue high...
详细信息
ISBN:
(数字)9783031641824
ISBN:
(纸本)9783031641817;9783031641824
Usually, mature Artificial Intelligence (AI) projects are developed by a team of various members, such as data engineers, data scientists, software engineers and machinelearning (ML) engineers. They often pursue highly heterogeneous approaches, leading to new challenges in collaboration, particularly regarding software quality, data versioning and the traceability of model metrics and other resulting artifacts. These challenges are further intensified when AI projects rely on dynamic datasets, introducing an entirely new dimension that teams must deal with. Adopting principles from the machinelearning operations (MLOps) paradigm becomes essential in this context. To go beyond existing process models and develop actionable guidelines, our work introduces a Git workflow for AI projects. We present basic instructions for data and code while outlining a minimal infrastructure setup. Building upon abstract concepts, we delve into concrete, actionable steps by examining the proposed branching workflow. Through a case study, we apply the development methodology to two use cases and demonstrate that the principles and approaches positively impact project outcomes.
Testing machinelearning (ML) projects is challenging due to inherent non-determinism of various ML algorithms and the lack of reliable ways to compute reference results. Developers typically rely on their intuition w...
详细信息
ISBN:
(纸本)9781665457019
Testing machinelearning (ML) projects is challenging due to inherent non-determinism of various ML algorithms and the lack of reliable ways to compute reference results. Developers typically rely on their intuition when writing tests to check whether ML algorithms produce accurate results. However, this approach leads to conservative choices in selecting assertion bounds for comparing actual and expected results in test assertions. Because developers want to avoid false positive failures in tests, they often set the bounds to be too loose, potentially leading to missing critical bugs. We present FASER - the first systematic approach for balancing the trade-off between the fault-detection effectiveness and flakiness of non-deterministic tests by computing optimal assertion bounds. FASER frames this trade-off as an optimization problem between these competing objectives by varying the assertion bound. FASER leverages 1) statistical methods to estimate the flakiness rate, and 2) mutation testing to estimate the fault-detection effectiveness. We evaluate FASER on 87 non-deterministic tests collected from 22 popular ML projects. FASER finds that 23 out of 87 studied tests have conservative bounds and proposes tighter assertion bounds that maximizes the fault-detection effectiveness of the tests while limiting flakiness. We have sent 19 pull requests to developers, each fixing one test, out of which 14 pull requests have already been accepted.
Commit messages play an important role in communication among developers. To measure the quality of commit messages, researchers have defined what semantically constitutes a Good commit message: it should have both th...
详细信息
ISBN:
(纸本)9781665457019
Commit messages play an important role in communication among developers. To measure the quality of commit messages, researchers have defined what semantically constitutes a Good commit message: it should have both the summary of the code change (What) and the motivation/reason behind it (Why). The presence of the issue report/pull request links referenced in a commit message has been treated as a way of providing Why information. In this study, we found several quality issues that could hamper the links' ability to provide Why information. Based on this observation, we developed a machinelearning classifier for automatically identifying whether a commit message has What and Why information by considering both the commit messages and the link contents. This classifier outperforms state-of-the-art machinelearning classifiers by 12 percentage points improvement in the F1 score. With the improved classifier, we conducted a mixed method empirical analysis and found that: (1) Commit message quality has an impact on software defect proneness, and (2) the overall quality of the commit messages decreases over time, while developers believe they are writing better commit messages. All the research artifacts (i.e., tools, scripts, and data) of this study are available on the accompanying website [2].
Embedded Artificial Intelligence (AI) is becoming increasingly important in the field of healthcare where such AI enabled devices are utilized to assist physicians, clinicians, and surgeons in their diagnosis, rehabil...
详细信息
ISBN:
(纸本)9798350329964
Embedded Artificial Intelligence (AI) is becoming increasingly important in the field of healthcare where such AI enabled devices are utilized to assist physicians, clinicians, and surgeons in their diagnosis, rehabilitation and therapy planning. However, it is still a challenging task to come up with an accurate and efficient machinelearning model for resource-limited devices that work 24 x 7. It requires both intuition and experience. This dependence on human expertise and reliance on trial-and-error-based design methods create impediments to the standard processes of effort estimation, design phase planning, and generating service-level agreements for projects that involve AI-enabled MedTech devices. In this paper, we present AutoML search from an algorithmic perspective, instead of a more prevalent optimization or black-box tool view. We briefly present and point to case studies that demonstrate the efficacy of the automation approach in terms of productivity improvements. We believe that our proposed method can make AutoML more amenable to the applications of softwareengineering principles and also accelerate biomedical device engineering, where there is a high dependence on skilled human resources.
In order to analyze the variation characteristics of the measured data of the monitoring system and determine the main factors affecting the accuracy of the measured data, four models in machinelearning methods were ...
详细信息
A taxonomy of expected bugs in a typical program can be useful while performing several softwareengineering tasks such as test case design, mutation testing, and fault localization. In this context, we have analyzed ...
详细信息
The most expressive attribute for defining successful software is its quality, which can only be attained when the chances of a defect occurrence are doubtful. software defect Prediction is a cycle to develop a model ...
详细信息
Code smells are common in poorly designed software that can hinder code maintainability. Automatic detection of design flaws assists developers in identifying code smells in theirsoftware programs to avoid low-quality...
详细信息
Stack Overflow (SO), as technical Question & Answer (Q&A) platform, plays a pivotal role in facilitating knowledge exchange between software developers. Since the release of SO, numerous studies have been cond...
详细信息
Based on the size data of the human body of Chinese male pilots, this article uses the standard model library of Poser software to adjust and modify the dimensions of various parts, and establishes a three-dimensional...
详细信息
暂无评论