Czech Grammar Agreement Dataset for Evaluation of Language Models

  Authors

Baisa, Vít

  Item identifier

http://hdl.handle.net/11234/1-1933

 Project URL

https://www.muni.cz/vyzkum/publikace/1362555

 Demo URL

https://nlp.fi.muni.cz/~xbaisa/agree/

 Date issued

2016-12-02

 Type

corpus, text

 Size

10000000 sentences

 Language(s)

Czech

 Description

AGREE is a dataset and task for evaluation of language models based on grammar agreement in Czech. The dataset consists of sentences with marked suffixes of past tense verbs. The task is to choose the right verb suffix which depends on gender, number and animacy of subject. It is challenging for language models because 1) Czech is morphologically rich, 2) it has relatively free word order, 3) high out-of-vocabulary (OOV) ratio, 4) predicate and subject can be far from each other, 5) subjects can be unexpressed and 6) various semantic rules may apply. The task provides a straightforward and easily reproducible way of evaluating language models on a morphologically rich language.

 Publisher

Masaryk University, NLP Centre

 Subject(s)

agreement past tense verb suffix language model training data

 Collection(s)

LINDAT / CLARIAH-CZ Data & Tools

Show full item record