Package of word embeddings of Czech from a large corpus

  Authors

Kyjánek, Lukáš and Bonami, Olivier

  Item identifier

http://hdl.handle.net/11234/1-4920

 Date issued

2022-11-07

 Type

lexicalConceptualResource, text

 Size

8 files

 Language(s)

Czech

 Description

This package comprises eight models of Czech word embeddings trained by applying word2vec (Mikolov et al. 2013) to the currently most extensive corpus of Czech, namely SYN v9 (Křen et al. 2022). The minimum frequency threshold for including a word in the model was 10 occurrences in the corpus. The original lemmatisation and tagging included in the corpus were used for disambiguation. In the case of word embeddings of word forms, units comprise word forms and their tag from a positional tagset (cf. https://wiki.korpus.cz/doku.php/en:pojmy:tag) separated by '>', e.g., kočka>NNFS1-----A----. The published package provides models trained on both tokens and lemmas. In addition, the models combine training algorithms (CBOW and Skipgram) and dimensions of the resulting vectors (100 or 500), while the training window and negative sampling remained the same during the training. The package also includes files with frequencies of word forms (vocab-frequencies.forms) and lemmas (vocab-frequencies.lemmas).

 Publisher

Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics (UFAL)

 Acknowledgement

Charles University

Project code: START/HUM/010

Project name: A data-based approach to competition in word-formation: selected semantic categories across seven languages

 

Ministerstvo školství, mládeže a tělovýchovy České republiky

Project code: LM2018101

Project name: LINDAT/CLARIAH-CZ: Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy

 Subject(s)

word embeddings word vectors large corpus word2vec skipgram cbow

 Collection(s)

LINDAT / CLARIAH-CZ Data & Tools

Show full item record