TY - GEN
T1 - Joint Speech-Text Embeddings with Disentangled Speaker Features
AU - Gonzales, Michael Gian
AU - Corcoran, Peter
AU - Harte, Naomi
AU - Schukat, Michael
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - This paper presents a novel model architecture for speech processing that takes advantage of a joint speech-text embedding space and disentangled speaker features. Here unsupervised representation learning extracts latent features from the input without labels which results in task-agnostic, but information-entangled embeddings. On the other hand, a unified embedding space of speech and text aims to leverage acoustics and semantic knowledge from the two modalities, respectively. The model was trained on 4 speakers from the CMU Arctic dataset and evaluated on three downstream tasks: speaker recognition, automatic speech recognition (ASR), and text-to-speech (TTS). Results show 96.87% speaker classification accuracy, 11.57% Word Error Rate, and 8.91 Mel Cepstral Distortion mean on the evaluation set.
AB - This paper presents a novel model architecture for speech processing that takes advantage of a joint speech-text embedding space and disentangled speaker features. Here unsupervised representation learning extracts latent features from the input without labels which results in task-agnostic, but information-entangled embeddings. On the other hand, a unified embedding space of speech and text aims to leverage acoustics and semantic knowledge from the two modalities, respectively. The model was trained on 4 speakers from the CMU Arctic dataset and evaluated on three downstream tasks: speaker recognition, automatic speech recognition (ASR), and text-to-speech (TTS). Results show 96.87% speaker classification accuracy, 11.57% Word Error Rate, and 8.91 Mel Cepstral Distortion mean on the evaluation set.
KW - speech recognition
KW - speech synthesis
KW - speech-text
UR - https://www.scopus.com/pages/publications/85165949937
U2 - 10.1109/ISSC59246.2023.10162046
DO - 10.1109/ISSC59246.2023.10162046
M3 - Conference Publication
T3 - 2023 34th Irish Signals and Systems Conference, ISSC 2023
BT - 2023 34th Irish Signals and Systems Conference, ISSC 2023
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 34th Irish Signals and Systems Conference, ISSC 2023
Y2 - 13 June 2023 through 14 June 2023
ER -