Building a Question-Answering Corpus using Social Media and News Articles

PAULO R. CAVALIN; MAIRA GATTI DE BAYSER; FLÁVIO D. FIGUEREDO; LUIS G. MOYANO

Tomar

Conferencia; International Conference on the Computational Processing of Portuguese; 2016

Univerisdad de Lisboa

Is it possible to develop a reliable QA-Corpus using socialmedia data? What are the challenges faced when attempting such a task?In this paper, we discuss these questions and present our findings whendeveloping a QA-Corpus on the topic of Brazilian finance. In order topopulate our corpus, we relied on opinions from experts on Brazilianfinance that are active on the Twitter application. From these experts,we extract information from news websites that are used to populate an-swers in the corpus. Moreover, to effectively provide rankings of answersto questions, we employ novel deep-learning based similarity measuresbetween short sentences (that accounts for both questions and Tweets).We validated the employed methods on a recently released dataset ofsimilarity between short Portuguese sentences. More importantly, we alsodiscuss how we can use word vector representations to match questionsfrom real users to social media information, as well as rank answers tothe provided questions based on news websites shared on Twitter.