WMT17 Quality Estimation Shared Test Data

Name: WMT17 Quality Estimation Shared Test Data
License: https://lindat.mff.cuni.cz/repository/xmlui/page/licence-TAUS_QT21

Specia, Lucia; Logacheva, Varvara

WMT17 Quality Estimation Shared Test Data

LRT + Open Submissions

Authors: Specia, Lucia and Logacheva, Varvara

Item identifier: http://hdl.handle.net/11372/LRT-2135

Project URL: http://www.statmt.org/wmt17/quality-estimation-task.html

Date issued: 2017-04-13

Type: corpus, text

Language(s): English , German

Description: Test data for the WMT17 QE task. Train data can be downloaded from http://hdl.handle.net/11372/LRT-1974 This shared task will build on its previous five editions to further examine automatic methods for estimating the quality of machine translation output at run-time, without relying on reference translations. We include word-level, phrase-level and sentence-level estimation. All tasks will make use of a large dataset produced from post-editions by professional translators. The data will be domain-specific (IT and Pharmaceutical domains) and substantially larger than in previous years. In addition to advancing the state of the art at all prediction levels, our goals include: - To test the effectiveness of larger (domain-specific and professionally annotated) datasets. We will do so by increasing the size of one of last year's training sets. - To study the effect of language direction and domain. We will do so by providing two datasets created in similar ways, but for different domains and language directions. - To investigate the utility of detailed information logged during post-editing. We will do so by providing post-editing time, keystrokes, and actual edits. This year's shared task provides new training and test datasets for all tasks, and allows participants to explore any additional data and resources deemed relevant. A in-house MT system was used to produce translations for all tasks. MT system-dependent information can be made available under request. The data is publicly available but since it has been provided by our industry partners it is subject to specific terms and conditions. However, these have no practical implications on the use of this data for research purposes.