A Tool for Introducing Computer Science with Automatic Formative Assessment

LUCIANA BENOTTI; CECILIA MARTINEZ; FERNANDO SCHAPACHNIK

IEEE Transactions on Learning Technologies

Institute of Electrical and Electronics Engineers

Año: 2018 vol. 11 p. 179 - 192

1939-1382

In this paper we present a software platform called Chatbot designed to introduce high school students to ComputerScience (CS) concepts in an innovative way: by programming chatbots. A chatbot is a bot that can be programmed to have aconversation with a human or robotic partner in some natural language such as English or Spanish. While programming their chatbots,students use fundamental CS constructs such as variables, conditionals, and finite state automata, among others. Chatbot uses patternmatching, state of the art lemmatization techniques, and finite state automata in order to provide automatic formative assessment to thestudents. When an error is found, the formative feedback generated is immediate and task-level. We evaluated Chatbot in twoobservational studies. An online nation-wide competition where more than 10,000 students participated. And, a mandatory in-class15-lesson pilot course in three high schools. We measured indicators of student engagement (task completion, participation, selfreported interest, etc.) and found that girls? engagement with Chatbot was higher than boys? for most indicators. Also, in the onlinecompetition, the task completion rate for the students that decided to use Chatbot was five times higher than for the students that chose touse the renown animation and game programming tool Alice. Our results suggest that the availability of automatic formative assessmentmay have an impact on task completion and other engagement indicators among high school students.