Multilingual TTS Accent Impressions for Accented ASR

 

G. Karakasidis, N. Robinson, Y. Getman, A. Ogayo, R. Al-Ghezi, A. Ayasi, S. Watanabe, D. R. Mortensen, M. Kurimo

In Proceedings Text, Speech, and Dialogue (TSD) 2023

pdf

Abstract

Automatic Speech Recognition (ASR) for high-resource languages like English is often considered a solved problem. However, most high-resource ASR systems favor socioeconomically advantaged dialects. In the case of English, this leaves behind many L2 speakers and speakers of low-resource accents (a majority of English speakers). One way to mitigate this is to fine-tune a pre-trained English ASR model for a desired low-resource accent. However, collecting transcribed accented audio is costly and time-consuming. In this work, we present a method to produce synthetic L2-English speech via pre-trained text-to-speech (TTS) in an L1 language (target accent). This can be produced at a much larger scale and lower cost than authentic speech collection. We present initial experiments applying this augmentation method. Our results suggest that success of TTS augmentation relies on access to more than one hour of authentic training data and a diversity of target-domain prompts for speech synthesis.

Bibtex

@InProceedings{10.1007/978-3-031-40498-6_28,
author="Karakasidis, Georgios
and Robinson, Nathaniel
and Getman, Yaroslav
and Ogayo, Atieno
and Al-Ghezi, Ragheb
and Ayasi, Ananya
and Watanabe, Shinji
and Mortensen, David R.
and Kurimo, Mikko",
editor="Ek{\v{s}}tein, Kamil
and P{\'a}rtl, Franti{\v{s}}ek
and Konop{\'i}k, Miloslav",
title="Multilingual TTS Accent Impressions for Accented ASR",
booktitle="Text, Speech, and Dialogue",
year="2023",
publisher="Springer Nature Switzerland",
address="Cham",
pages="317--327",
abstract="Automatic Speech Recognition (ASR) for high-resource languages like English is often considered a solved problem. However, most high-resource ASR systems favor socioeconomically advantaged dialects. In the case of English, this leaves behind many L2 speakers and speakers of low-resource accents (a majority of English speakers). One way to mitigate this is to fine-tune a pre-trained English ASR model for a desired low-resource accent. However, collecting transcribed accented audio is costly and time-consuming. In this work, we present a method to produce synthetic L2-English speech via pre-trained text-to-speech (TTS) in an L1 language (target accent). This can be produced at a much larger scale and lower cost than authentic speech collection. We present initial experiments applying this augmentation method. Our results suggest that success of TTS augmentation relies on access to more than one hour of authentic training data and a diversity of target-domain prompts for speech synthesis.",
isbn="978-3-031-40498-6"
}