Who's speaking? Predicting speaker profession from speech

Yaru Wu (1, 2, 3) , Lihu Chen (4, 5, 6) , Benjamin Elie (7) , Fabian M. Suchanek (4, 5, 6) , Ioana Vasilescu (7, 2) , Lori Lamel (8)

1 CRISCO - Centre de recherche inter-langues sur la signification en contexte
2 LISN - Laboratoire Interdisciplinaire des Sciences du Numérique
3 LPP - LPP - Laboratoire de Phonétique et Phonologie - UMR 7018
4 DIG - Data, Intelligence and Graphs
5 INFRES - Département Informatique et Réseaux
6 LTCI - Laboratoire Traitement et Communication de l'Information
7 Université Paris-Saclay
8 LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur

Variations in speech can reveal the gender, birth place, age, and socioeconomic level of the speaker. In this paper, we show that even the profession of the speaker can be recovered from a recording. For this purpose, we design a method that combines features from both the speech signal and the transcription. For the features from the transcription, we used pretrained language models. This allows us to train a model that predicts the speaker profession from both signals. Our empirical results show that our model can narrow down the profession of the speakers considerably.

Mots clés: Knowledge base; Large corpora; Multimodal representation; Pre-trained language models

Domaines: Informatique [cs]

Yaru Wu, Lihu Chen, Benjamin Elie, Fabian M. Suchanek, Ioana Vasilescu, et al.. Who's speaking? Predicting speaker profession from speech. International Congress of Phonetic Sciences 2023, Aug 2023, Prague, Czech Republic. pp.3086-3090. ⟨hal-04190126⟩