Package of word embeddings of Czech from a large corpus

  Autoři

Kyjánek, Lukáš and Bonami, Olivier

  Identifikátor

http://hdl.handle.net/11234/1-4920

 Datum vydání

2022-11-07

 Typ

lexicalConceptualResource, text

 Velikost

8 files

 Jazyky

Czech

 Popis

This package comprises eight models of Czech word embeddings trained by applying word2vec (Mikolov et al. 2013) to the currently most extensive corpus of Czech, namely SYN v9 (Křen et al. 2022). The minimum frequency threshold for including a word in the model was 10 occurrences in the corpus. The original lemmatisation and tagging included in the corpus were used for disambiguation. In the case of word embeddings of word forms, units comprise word forms and their tag from a positional tagset (cf. https://wiki.korpus.cz/doku.php/en:pojmy:tag) separated by '>', e.g., kočka>NNFS1-----A----. The published package provides models trained on both tokens and lemmas. In addition, the models combine training algorithms (CBOW and Skipgram) and dimensions of the resulting vectors (100 or 500), while the training window and negative sampling remained the same during the training. The package also includes files with frequencies of word forms (vocab-frequencies.forms) and lemmas (vocab-frequencies.lemmas).

 Nakladatel

Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics (UFAL)

 Sponzoři

Charles University

Kód projektu: START/HUM/010

Jméno projektu: A data-based approach to competition in word-formation: selected semantic categories across seven languages

 

Ministerstvo školství, mládeže a tělovýchovy České republiky

Kód projektu: LM2018101

Jméno projektu: LINDAT/CLARIAH-CZ: Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy

 Klíčová slova

word embeddings word vectors large corpus word2vec skipgram cbow

 Kolekce

LINDAT / CLARIAH-CZ Data & Tools

Zobrazit celý záznam