检索结果-内蒙古大学图书馆

Code Review Automation Via Multi-task Federated LLM - An Empirical Study

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Kumar, Jahnavi Chimalakonda, Sridhar Research in Intelligent Software & Human Analytics Lab Department of Computer Science and Engineering Indian Institute of Technology Tirupati India

Code review is a crucial process before deploying code to production, as it validates the code, provides suggestions for improvements, and identifies errors such as missed edge cases. In projects with regular production releases, the effort required for peer code-reviews remains high. Consequently, there has been significant interest from software engineering (SE) researchers in automating the code review process. Previous research on code review automation has typically approached the task as three independent sub-tasks: review necessity prediction, review comment generation, and code refinement. Our study attempts to (i) leverage the relationships between the sub-tasks of code review automation, by developing a multi-task model that addresses all tasks in an integrated manner, and (ii) increase model robustness on unseen data via collaborative large language model (LLM) modeling, while retaining the proprietary nature of code, by using federated learning (FL). The study explores five simple techniques for multi-task training, including two sequential methods, one parallel method, and two cumulative methods. The results indicate that sequentially training a federated LLM (FedLLM) for our code review multi-task use case is less efficient in terms of time, computation, and performance metrics, compared to training separate models for each task. Because sequential training demonstrates catastrophic forgetting, alternatively cumulative fine-tuning for multi-task training performs better than training models for individual tasks. This study highlights the need for research focused on effective fine-tuning of multi-task FedLLMs for SE tasks. © 2024, CC BY.

关键词： coding errors

Performance evaluation of a minimum bit error decoding algorithm of convolutional codes and its application

学校读者我要写书评

暂无评论

Electronics and Communications in Japan, Part III: Fundamental Electronic Science (English translation of Denshi Tsushin Gakkai Ronbunshi) 1993年第3期76卷 83-90页

作者： Ohashi, Masayoshi Yasuda, Yutaka R & D Lab Saitama Japan

Basic decoding characteristics of a minimum bit error decoding (MBED), originally proposed by Bahl et al., is investigated. It is shown that MBED can predict the bit error rate of each decoded bit with good precision. Then MBED is applied to the convolutional/Reed-Solomon (RS) concatenated coding scheme. The outer RS codes are decoded by an erasure decoding using the symbol erasure information provided from the MBED results of inner convolutional codes. Simulation results show that the overall bit error rate is reduced one-half to one-third more than the conventional nonerasure decoding.

关键词： coding errors

Difficulties Constructing Lattices with Exponential Kissing Number from Codes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Bennett, Huck Golovnev, Alexander Stephens-Davidowitz, Noah University of Colorado Boulder United States Georgetown University United States Cornell University United States

In this note, we present examples showing that several natural ways of constructing lattices from error-correcting codes do not in general yield a correspondence between minimum-weight non-zero codewords and shortest non-zero lattice vectors. From these examples, we conclude that the main results in two works of Vlăduţ (Moscow J. Comb. Number Th., 2019 and Discrete Comput. Geom., 2021) on constructing lattices with exponential kissing number from error-correcting codes are invalid. Exhibiting a family of lattices with exponential kissing number therefore remains an open problem. Copyright © 2024, The Authors. All rights reserved.

关键词： coding errors

REDO: EXECUTION-FREE RUNTIME ERROR DETECTION FOR coding AGENTS

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Li, Shuo Kan, Andrey Callot, Laurent Bhasker, Bhavana Rashid, Muhammad Shihab Esler, Timothy B. Department of Computer Science University of Pennsylvania United States AWS AI Labs United States

As LLM-based agents exhibit exceptional capabilities in addressing complex problems, there is a growing focus on developing coding agents to tackle increasingly sophisticated tasks. Despite their promising performance, these coding agents often produce programs or modifications that contain runtime errors, which can cause code failures and are difficult for static analysis tools to detect. Enhancing the ability of coding agents to statically identify such errors could significantly improve their overall performance. In this work, we introduce Execution-free Runtime Error Detection for coding Agents (REDO), a method that integrates LLMs with static analysis tools to detect runtime errors for coding agents, without code execution. Additionally, we propose a benchmark task, SWE-Bench-Error-Detection (SWEDE), based on SWE-Bench (lite), to evaluate error detection in repository-level problems with complex external dependencies. Finally, through both quantitative and qualitative analyses across various error detection tasks, we demonstrate that REDO outperforms current state-of-the-art methods by achieving a 11.0% higher accuracy and 9.1% higher weighted F1 score;and provide insights into the advantages of incorporating LLMs for error detection. Copyright © 2024, The Authors. All rights reserved.

关键词： coding errors

On the Linearization of Optimal Rates for Independent Zero-Error Source and Channel Problems

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Charpenay, Nicolas Treust, Maël Le Roumy, Aline Univ Rennes CNRS IRMAR UMR 6625 RennesF-35000 France Univ. Rennes CNRS Inria IRISA UMR 6074 RennesF-35000 France Centre Inria l’Université de Rennes France

Zero-error coding encompasses a variety of source and channel problems where the probability of error must be exactly zero. This condition is stricter than that of the vanishing error regime, where the error probability goes to zero as the code blocklength goes to infinity. In general, zero-error coding is an open combinatorial question. We investigate two unsolved zero-error problems: the source coding problem with side information and the channel coding problem. We focus our attention on families of independent problems for which the distribution decomposes into a product of distributions, corresponding to solved zero-error problems. A crucial step is the linearization property of the optimal rate which does not always hold in the zero-error regime, unlike in the vanishing error regime. We derive a condition under which the linearization properties of the complementary graph entropy H̅ for the AND product of graph and for the disjoint union of graphs are equivalent. Then we establish the connection with a recent result obtained by Wigderson and Zuiddam and by Schrijver, for the zero-error capacity C0. As a consequence, we provide new single-letter characterizations of H̅ and C0, for example when the graph is a product of perfect graphs, which is not perfect in general, and for the class of graphs obtained by the product of a perfect graph G with the pentagon graph C5. By building on Haemers result for C0, we also show that the linearization of H̅ does not hold for the product of the Schläfli graph with its complementary graph. © 2024, CC BY.

关键词： coding errors

Exploratory study of common coding faults in C programs

学校读者我要写书评

暂无评论

Journal of Software Maintenance 1996年第4期8卷 241-256页

作者： Duncan, Ishbel Robson, Dave Univ of Durham Durham United Kingdom

Large scale code testing can be mad viable by determining and searching for the most probable faults in the system under examination. The results of an exploratory survey carried out to determine the common error factors for code written in C indicates that the program task and the programmer experience are important considerations. Using this information, testing can be directed efficiently towards the removal of prevalent coding faults. The results of the survey can also be useful in determining features of C which are likely to become the subject of corrective maintenance.

关键词： coding errors

A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Chen, QiHong Li, Jiawei Deng, Jiecheng Yu, Jiachen Chen, Justin Tian Jin Ahmed, Iftekhar The Donald Bren School of Information and Computer Sciences University of California Irvine IrvineCA United States The University of California Riverside RiversideCA United States

Recent advancements in Large Language Models (LLMs) have led to their widespread application in automated code generation. However, these models can still generate defective code that deviates from the specification. Previous research has mainly focused on the mistakes in LLM-generated standalone functions, overlooking real-world software development situations where the successful generation of the code requires software contexts such as external dependencies. In this paper, we considered both of these code generation situations and identified a range of non-syntactic mistakes arising from LLMs’ misunderstandings of coding question specifications. Seven categories of non-syntactic mistakes were identified through extensive manual analyses, four of which were missed by previous works. To better understand these mistakes, we proposed six reasons behind these mistakes from various perspectives. Moreover, we explored the effectiveness of LLMs in detecting mistakes and their reasons. Our evaluation demonstrated that GPT-4 with the ReAct prompting technique can achieve an F1 score of up to 0.65 when identifying reasons for LLM’s mistakes, such as misleading function signatures. We believe that these findings offer valuable insights into enhancing the quality of LLM-generated code. Copyright © 2024, The Authors. All rights reserved.

关键词： coding errors

AUTODETECT: Towards a Unified Framework for Automated Weakness Detection in Large Language Models

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Cheng, Jiale Lu, Yida Gu, Xiaotao Ke, Pei Liu, Xiao Dong, Yuxiao Wang, Hongning Tang, Jie Huang, Minlie Group Tsinghua University China Zhipu AI China Tsinghua University China

Although Large Language Models (LLMs) are becoming increasingly powerful, they still exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding tasks. As these unexpected errors could lead to severe consequences in practical deployments, it is crucial to investigate the limitations within LLMs systematically. Traditional benchmarking approaches cannot thoroughly pinpoint specific model deficiencies, while manual inspections are costly and not scalable. In this paper, we introduce a unified framework, AUTODETECT, to automatically expose weaknesses in LLMs across various tasks. Inspired by the educational assessment process that measures students’ learning outcomes, AUTODETECT consists of three LLM-powered agents: Examiner, Questioner, and Assessor. The collaboration among these three agents is designed to realize comprehensive and in-depth weakness identification. Our framework demonstrates significant success in uncovering flaws, with an identification success rate exceeding 30% in prominent models such as ChatGPT and Claude. More importantly, these identified weaknesses can guide specific model improvements, proving more effective than untargeted data augmentation methods like Self-Instruct. Our approach has led to substantial enhancements in popular LLMs, including the Llama series and Mistral-7b, boosting their performance by over 10% across several benchmarks. Code and data are publicly available at https://***/thu-coai/AutoDetect. © 2024, CC BY.

关键词： coding errors

The Second Generalized Covering Radius of Binary Primitive Double-Error-Correcting BCH Codes

学校读者我要写书评

暂无评论

arXiv 2024年

作者： Yohananov, Lev Schwartz, Moshe School of Electrical and Computer Engineering Ben-Gurion University of the Negev Beer Sheva8410501 Israel Department of Electrical and Computer Engineering McMaster University HamiltonONL8S 4K1 Canada

We completely determine the second covering radius for binary primitive double-error-correcting BCH codes. As part of this process, we provide a lower bound on the second covering radius for binary primitive BCH codes... 详细信息

关键词： coding errors