A Tool for Introducing Computer Science with Automatic Formative Assessment

SCHAPACHNIK, FERNANDO; MARTÍNEZ, MARÍA CECILIA; BENOTTI, LUCIANA

IEEE Transactions on Learning Technologies

IEEE Computer Society

Lugar: Los Alamos; Año: 2017 vol. PP

1939-1382

Abstract:In this paper we present a software platform called Chatbot designed to introduce high school students to Computer Science (CS) concepts in an innovative way: by programming chatbots. A chatbot is a bot that can be programmed to have a conversation with a human or robotic partner in some natural language such as English or Spanish. While programming their chatbots, students use fundamental CS constructs such as variables, conditionals and finite state automata, among others. Chatbot uses pattern matching, state of the art lemmatization techniques, and finite state automata in order to provide automatic formative assessment to the students. When an error is found, the formative feedback generated is immediate and task-level. We evaluated Chatbot in two observational studies. An online nation-wide competition where more than 10 thousand students participated. And a mandatory in-class 15-lesson pilot course in 3 high schools. We measured indicators of student engagement (task completion, participation, self reported interest, etc.) and found that girls? engagement with Chatbot was higher than boys? for most indicators. Also, in the online competition, the task completion rate for the students that decided to use Chatbot was five times higher than for the students that chose to use the renown animation and game programming tool Alice. Our results suggest that the availability of automatic formative assessment, may have an impact on task completion and other engagement indicators among high school students.