Quality Analysis of Multilingual Neural Machine Translation Systems and Reference Test Translations for the English-Romanian language pair in the Medical Domain

Veröffentlichungen: Beitrag zu KonferenzPaperPeer Reviewed

Abstract

Multilingual Neural Machine Translation (MNMT) models allow translation across
multiple languages based on a single system. We study the quality of a domain adapted MNMT model in the medical domain for English-Romanian with automatic metrics and a human error typology annotation based on the Multidimensional Quality Metrics (MQM) framework. We further expand the MQM typology to include terminology-specific error categories. We compare the out-of-domain MNMT with the in-domain adapted MNMT on a standard test dataset of abstracts from medical publications. The in-domain MNMT model outperforms the out-of-domain MNMT in all measured automatic metrics, and produces fewer errors. We also manually annotate the reference test dataset to study the quality of the reference translations, and we identify a high number of omissions, additions, and mistranslations. We therefore question the assumed accuracy of existing datasets. Finally, we compare the correlation between the COMET, BERTScore, and chrF automatic metrics with the MQM annotated translations; COMET shows a better correlation with the MQM scores.
OriginalspracheEnglisch
Seiten355–364
Seitenumfang10
PublikationsstatusVeröffentlicht - 12 Juni 2023
VeranstaltungThe 24th Annual Conference of the European Association for Machine Translation - Tampere, Tampere, Finnland
Dauer: 12 Juni 202315 Juni 2023
https://events.tuni.fi/eamt23/

Ausstellungen

AusstellungenThe 24th Annual Conference of the European Association for Machine Translation
KurztitelEAMT 2023
Land/GebietFinnland
OrtTampere
Zeitraum12/06/2315/06/23
Internetadresse

ÖFOS 2012

  • 102001 Artificial Intelligence
  • 102019 Machine Learning
  • 602051 Translationswissenschaft

Fingerprint

Untersuchen Sie die Forschungsthemen von „Quality Analysis of Multilingual Neural Machine Translation Systems and Reference Test Translations for the English-Romanian language pair in the Medical Domain“. Zusammen bilden sie einen einzigartigen Fingerprint.

Zitationsweisen