Turn-taking events classification in continuous EEG data and speech

KAMIENKOWSKI JE; GRAVANO A; BRUSCO P

Congreso; Latin American Brain Mapping Network (LABMAN); 2017

Latin American Brain Mapping Network (LABMAN) - Organization for Human Brain Mapping

Turn-taking refers to the process in which a two-person conversation flows in alternating turns between taking the floor or leading the other speaker talk. This work describes a series of machine learning experiments for automatically classifying the type of turn-taking transition based on features extracted from speech and the EEG signal of different pair of subjects. These were conducted on a corpus of unrestricted dialogues, with simultaneous recordings of speech and EEG from both subjects. Our results indicate that it is possible to detect the listener´s perception of turn-yielding cues present in the speaker´s speech from the EEG signal of the subject. Similarly, from the current speaker´s EEG signal we can extract useful information for detecting whether the speaker will yield the turn or continue talking. These results may lead to new tools valuable for the development of brain-computer interfaces.