Luca Ducceschi

AUGUSTA (Artificial UnderstandinG of Utterances Spoken in Tyrolean Accents) is a family of Speech-to-Text language models fine-tuned for South Tyrolean dialect translation into Standard German.

This study presents the first fine-tuned Whisper model for the automatic translation of South Tyrolean dialectal speech into Standard German text. To address an unmet need for subtitling and translation, we introduce a small corpus of manually annotated and synthetic speech data compiled for this task. Through fine-tuning and hyperparameter optimisation, our model achieves a BLEU score of 86.18 significantly outperforming baseline error rates. Our findings highlight Whisper's effectiveness in handling dialectal speech, contributing to low-resource language research. The model is already being used in a heritage collaboration for large-scale translation of audiovisual archival material and is also being considered for application in news broadcasting and tourism promotion. Future directions include expanding the training data and extending hyperparameter optimisation to improve the model's performance and generalisation across South Tyrolean dialectal variations.