What is the impact of raw MT on Japanese users of Word: preliminary results of a usability study using eye-tracking

This paper presents preliminary results of a study of Japanese native speakers
working with the Microsoft Word application in two modalities: the released Japanese version and a machine translated (MT) version (the raw MT strings
incorporated into the MS Word interface). To explore the effect of translation
modality on task completion, time and satisfaction, an experiment using an eye-tracker was set up with a group of 42 users: 22 native Japanese and 20 native
English speakers. The results suggest that Japanese-native speakers have higher
completion scores and are more efficient when working with the released versions of the product than with the MT version, but these differences are not significant. Their self-reported satisfaction, however, is significantly higher when working with the released product as opposed to the raw MT version.