Using NLP models for modeling brain processes during reading

B. BIANCHI; CÓCARO, CESAR; UMFURER, ALFREDO; FERNANDEZ SLEZAK, DIEGO ; KAMIENKOWSKI, JUAN E.

Montevideo

Congreso; Khipu; 2023

The advancement of the Natural Language Processing field has allowed for the development of causal language models with a high capacity for generating text. For several years, the field of Cognitive Neuroscience has been using these models to better understand cognitive processes. In previous works, we found that models such as Ngrams and those based on LSTM networks are capable of partially modeling the predictability of words (cloze-Pred), particularly when used as a co-variable to explain the eye movements of people reading natural texts. In this work, we deepen this line of research using models based on GPT2. The results show that this architecture achieves better modeling of cloze-Pred than its predecessors. We also found and that fine-tuning the model with a domain-specific corpus generates its predictions to be less based on lexical frequency. Here we also discuss how to generate an multi-domain corpus of text in "Español Rioplantese".