Time-on-task is one key contributor to learning. However, how timeon-task is measured often varies, and is limited by the available data. In this work, we study two different time-on-task metrics-derived from programm...
详细信息
ISBN:
(纸本)9781450390705
Time-on-task is one key contributor to learning. However, how timeon-task is measured often varies, and is limited by the available data. In this work, we study two different time-on-task metrics-derived from programming process data-for predicting performance in an introductory programming course. The first metric, coarse-grained time-on-task, is based on students' submissions to programming assignments;the second, fine-grained time-on-task, is based on the keystrokes that students take while constructing their programs. Both types of time-on-task metrics have been used in prior work, and are supposedly designed to measure the same underlying feature: time-on-task. However, previous work has found that the correlation between these two metrics is not as high as one might expect. We build on that work by analyzing how well the two metrics work for predicting students' performance in an introductory programming course. Our results suggest that the correlation between the fine-grained time-on-task metric and both weekly exercise points and exam points is higher than the correlation between the coarse-grained time-on-task metric and weekly exercise points and exam points. Furthermore, we show that the fine-grained timeon-task metric is a better predictor of students' future success in the course exam than the coarse-grained time-on-task metric. We thus propose that future work utilizing time-on-task as a predictor of performance should use as fine-grained data as possible to measure time-on-task if such data is available.
It is generally accepted that learning to program could be easier for many students. One of the most important components of this experience is the programming environment. Novices learn in a variety of environments, ...
详细信息
ISBN:
(纸本)9781450367936
It is generally accepted that learning to program could be easier for many students. One of the most important components of this experience is the programming environment. Novices learn in a variety of environments, from basic command-line interfaces to industry-strength IDEs. These environments can differ substantially in compilation behavior and error message presentation - arguably two of the most important mechanisms through which users interact with the programming language. In this study, we utilize Blackbox data to compare the programming behavior of thousands of users programming in Java, who all used BlueJ versions 3 and 4. These two versions differ drastically in terms of compilation behavior and error message presentation. BlueJ 3 is a click-to-compile editor that delivers text-based error messages from javac to the user, but only presents the first error message, even if the compiler produces several. BlueJ 4 automatically compiles in the background but retains click-to-compile ability. In addition, all error messages (not just the first) may be viewed by the user. We find that the programming experience and behavior of these users can be substantially affected by changes in these mechanisms, causing numbers of manual compilations, successful compilations, and error messages presented in each version to differ, in cases, markedly. Our results provide evidence on how changes in programming environment affect user behavior in conditions that reasonably control for variables other than the programming environment. This can inform the decisions of educators, tool designers, and HCI researchers in their work to make learning more effective for novice programmers.
I am studying the use of fine-grained programming process data for student modeling. The initial plan is to construct different types of program state representations such as Abstract Syntax Trees (ASTs) from the data...
详细信息
ISBN:
(纸本)9781450349680
I am studying the use of fine-grained programming process data for student modeling. The initial plan is to construct different types of program state representations such as Abstract Syntax Trees (ASTs) from the data. These program state representations could be used for both automatically inferring knowledge components that the students are trying to learn as well as for modeling students' knowledge on those specific components.
In this paper, we analyze keystroke log data from two introductory programming courses from two distinct contexts to investigate the proportion of events that compile, how this relates to contextual factors, the progr...
详细信息
ISBN:
(纸本)9798350322590
In this paper, we analyze keystroke log data from two introductory programming courses from two distinct contexts to investigate the proportion of events that compile, how this relates to contextual factors, the progression of programs, and academic outcomes. We find that, as students write their programs, frequency of compile and run events increases as does the proportion of events that compile. We also find a spike in the number of compile and run events as a program nears completion, that the proportion of events that compile varies by assignment, length of program, and programming context, that real-time IDE error diagnostics lead to higher proportion of events that are in compilable state, and that a student's awareness of their compilable state is correlated with exam score while the amount of time they spend in an uncompilable state is not. Among the practical implications of our work are the fact that researchers cannot rely on frequency of compilation remaining constant through an assignment and a call to researchers and practitioners to design pedagogies that enhance student awareness of their compilable state.
Student performance prediction aims to build models to help educators identify struggling students so they can be better supported. However, prior work in the space frequently evaluates features and models on data col...
详细信息
ISBN:
(纸本)9781450394314
Student performance prediction aims to build models to help educators identify struggling students so they can be better supported. However, prior work in the space frequently evaluates features and models on data collected from a single semester, of a single course, taught at a single university. Without evaluating these methods in a broader context there is an open question of whether or not performance prediction methods are capable of generalising to new data. We test three methods for evaluating student performance models on data from introductory programming courses from two universities with a total of 3, 323 students. Our results suggest that using cross-validation on one semester is insufficient for gauging model performance in the real world. Instead, we suggest that where possible future work in student performance prediction collects data from multiple semesters and uses one or more as a distinct hold-out set. Failing this, bootstrapped cross-validation should be used to improve confidence in models' performance. By recommending stronger methods for evaluating performance prediction models, we hope to bring them closer to practical use and assist teachers to understand struggling students in novice programming courses.
Learning to program is a process that relies on learning theoretical fundamentals as well as practice, and almost always involves some type of programming environment. In order to build effective environments that sup...
详细信息
ISBN:
(纸本)9781450388498
Learning to program is a process that relies on learning theoretical fundamentals as well as practice, and almost always involves some type of programming environment. In order to build effective environments that support good learning for novices, it is important to explore the interaction between novices and these environments. A variety of feedback mechanisms are employed by various environments in use in classrooms today. Some, such as text-based error messages are common to almost all. Other interaction mechanisms, such as invoking the compiler, can vary rather drastically. In this study we investigate the difference between BlueJ 3 and BlueJ 4, two versions of a pedagogical programming environment that offer different mechanisms for compilation and error message presentation. We find evidence that these differences provide users with fundamentally different programming experiences. Specifically, we find that programming process data produced by BlueJ 3 users follow a very deterministic distribution compared to BlueJ 4. Based on this, we present a formula that describes the behaviour of BlueJ 3 users in terms of compilation and error metrics. Conversely, we demonstrate that BlueJ 4 allows users to interact more freely in terms of compilation mechanism as well as how they receive error messages, and their quantity. Which is more beneficial to novices however, is an open question.
暂无评论