This paper studies code-Text Representation (CTR) learning, aiming to learn general-purpose representations that support downstream code/text applications such as code search, finding code matching textual queries. Ho...
详细信息
This paper studies code-Text Representation (CTR) learning, aiming to learn general-purpose representations that support downstream code/text applications such as code search, finding code matching textual queries. However, state-of-the-arts do not focus on matching the gap between code/text modalities. In this paper, we complement this gap by providing an intermediate representation, and view it as "explanation." Our contribution is three fold: First, we propose four types of explanation utilization methods for CTR, and compare their effectiveness. Second, we show that using explanation as the model input is desirable. Third, we confirm that even automatically generated explanation can lead to a drastic performance gain. To the best of our knowledge, this is the first work to define and categorize code explanation, for enhancing code understanding/representation.
In recent years, large language models (LLMs) have been employed significantly in different domains of computing education. Nevertheless, these models have been focused on essential adherence to their integration as c...
详细信息
In recent years, large language models (LLMs) have been employed significantly in different domains of computing education. Nevertheless, these models have been focused on essential adherence to their integration as coding assistants in computing education. However, attention has been switched to thoroughly examining and analyzing LLM behavior, particularly in computing education for programming tasks such as code generation, code explanation, and programming error message explanation. Therefore, it becomes imperative to understand their behavior to examine potential pitfalls. This article addresses this gap systematically and details how different LLM-based coding chatbots, such as ChatGPT, codex, Copilot, and others, react to various coding inputs within computing education. To achieve this objective, we collected and analyzed articles from 2021 to 2024, and 72 studies were thoroughly examined. These objectives include investigating the existing limitations and challenges associated with utilizing these systems for coding tasks, assessing their responses to prompts containing coding syntax, examining the impact of their output on student learning, and evaluating their performance as debugging tools. The findings of this review highlight that it is premature to incorporate these systems into computing education due to their limitations that may limit their effectiveness as comprehensive coding assistants for computer science students. These limitations include issues with handling prompts containing code snippets, potential negative impacts on student learning, limited debugging capabilities, and other ineffectiveness. The finding also reports multiple research directions that can be considered in future research related to LLMs in computing education.
The teaching and assessment of introductory programming involves writing code that solves a problem described by text. Previous research found that OpenAI's codex, a natural language machine learning model trained...
详细信息
ISBN:
(纸本)9781450394314
The teaching and assessment of introductory programming involves writing code that solves a problem described by text. Previous research found that OpenAI's codex, a natural language machine learning model trained on billions of lines of code, performs well on many programming problems, often generating correct and readable Python code. GitHub's version of codex, Copilot, is freely available to students. This raises pedagogic and academic integrity concerns. Educators need to know what Copilot is capable of, in order to adapt their teaching to AI-powered programming assistants. Previous research evaluated the most performant codex model quantitatively, e.g. how many problems have at least one correct suggestion that passes all tests. Here I evaluate Copilot instead, to see if and how it differs from codex, and look qualitatively at the generated suggestions, to understand the limitations of Copilot. I also report on the experience of using Copilot for other activities asked of students in programming courses: explaining code, generating tests and fixing bugs. The paper concludes with a discussion of the implications of the observed capabilities for the teaching of programming.
A number of published studies indicate that many students who receive passing grades in CS1 may struggle in CS2. This can lead to higher attrition and failure rates in CS2, and perhaps also in subsequent courses in th...
详细信息
ISBN:
(纸本)9781450390705
A number of published studies indicate that many students who receive passing grades in CS1 may struggle in CS2. This can lead to higher attrition and failure rates in CS2, and perhaps also in subsequent courses in the curriculum. Many researchers have studied factors that lead to student success and common misconceptions that may arise in introductory courses. However, relatively little attention has been focused on the transition between CS1 and CS2. In this paper, we report on a study of a variety of types of CS1 exam questions, with the goal of finding questions that can predict a student's success (or lack of success) in CS2. Results indicate that code explanation and code completion questions can be especially good predictors, along with some code tracing questions. We discuss some of the factors that may make certain questions better predictors than others, in the process confirming some observations that have been reported by other researchers. The results from this experiment seem promising, and point to several possibilities for further research.
暂无评论