Subjective Evaluation of a High Quality Text-to-Speech System for Argentine Spanish

GURLEKIAN JA, COSSIO MERCADO C, TORRES HM, VACCARI ME.

Madrid

Conferencia; IberSpeech 2012; 2012

Universidad Autonoma de Madrid

Abstract. This work summarizes the perceptual evaluation of our recently developed text-to-speech system (S1), based on unit concatenation. We compare it with two commerciallyavailable systems (S2 and S3) using three different evaluation methods. One is the P.85recommendation by the International Telecommunication Union (ITU), the second method,called Syntactically Unexpected Sentences (SUS), is known to be the most strict for intelligibilityevaluation, and the third is the Mean Opinion Score (MOS) scale. Results of ITU testshowed better quality and intelligibility responses for system S2. General quality evaluatedby MOS and intelligibility evaluated by SUS presented no appreciable differences betweenS1 and S2. It is concluded that high quality performance is related to a complete intonationmodeling of different type of phrase length and styles. S1 has high intelligibility and qualityfor general information sentences but presented lower scores for specific tasks as proposed inITU tests, where short phrases coverage should be introduced in the intonational modelingof our system.