Czech Text Document Corpus v 2.0

  Authors

Král, Pavel and Lenc, Ladislav

  Item identifier

http://hdl.handle.net/11234/1-2884

 Project URL

http://ctdc.kiv.zcu.cz/

 Referenced by

http://www.lrec-conf.org/proceedings/lrec2018/summaries/671.html

 Date issued

2018-05-07

 Type

corpus, text

 Size

694 mb

 Language(s)

Czech

 Description

BASIC INFORMATION -------------------- Czech Text Document Corpus v 2.0 is a collection of text documents for automatic document classification in Czech language. It is composed of the text documents provided by the Czech News Agency and is freely available for research purposes. This corpus was created in order to facilitate a straightforward comparison of the document classification approaches on Czech data. It is particularly dedicated to evaluation of multi-label document classification approaches, because one document is usually labelled with more than one label. Besides the information about the document classes, the corpus is also annotated at the morphological layer. The main part (for training and testing) is composed of 11,955 real newspaper articles. We provide also a development set which is intended to be used for tuning of the hyper-parameters of the created models. This set contains 2735 additional articles. The total category number is 60 out of which 37 most frequent ones are used for classification. The reason of this reduction is to keep only the classes with the sufficient number of occurrences to train the models. Technical Details ------------------------ Text documents are stored in the individual text files using UTF-8 encoding. Each filename is composed of the serial number and the list of the categories abbreviations separated by the underscore symbol and the .txt suffix. Serial numbers are composed of five digits and the numerical series starts from the value one. For instance the file 00046_kul_nab_mag.txt represents the document file number 46 annotated by the categories kul (culture), nab (religion) and mag (magazine selection). The content of the document, i.e. the word tokens, is stored in one line. The tokens are separated by the space symbols. Every text document was further automatically mophologically analyzed. This analysis includes lemmatization, POS tagging and syntactic parsing. The fully annotated files are stored in .conll files. We also provide the lemmatized form, file with suffix .lemma, and appropriate POS-tags, see .pos files. The tokenized version of the documents is also available in .tok files. This corpus is available only for research purposes for free. Commercial use in any form is strictly excluded.

 Publisher

European Language Resources Association (ELRA)

 Acknowledgement

Czech Ministry of Education, Youth and Sports

Project code: LO1506

Project name: Sustainability support of the centre NTIS - New Technologies for the Information Society

 Subject(s)

corpus Czech document classification multi-label text

 Collection(s)

LINDAT / CLARIAH-CZ Data & Tools

Show full item record