VISUALIZATION OF INTERNAL ARTICULATOR DYNAMICS AND ITS INTELLIGIBILITY IN SYNTHETIC AUDIOVISUAL SPEECH

Katja Grauwinkel, Britta Dewitt & Sascha Fagel
Institute for Speech and Communication, Berlin University of Technology, Germany

ID 1023
[full paper]

This paper presents the result of a study investigating the influence of visualization of internal articulator movements on the intelligibility of synthesized audiovisual speech. A talking head was supplemented by internal passive and active articulators. A comparative perception test before and after two different training lessons was carried out, where one type of display included all internal articulator movements and the other displayed dynamics without tongue dorsum height, velum opening/closing and tongue forward/backward movements. Results show that recognition scores were significantly higher in audiovisual compared to auditory alone presentation with non-significantly different recognition scores for both kinds of display. But only in case of additional motion information of internal articulators the training lesson was able to significantly increase the visual and audiovisual intelligibility.