RobeCzech Base
Please use the following text to cite this item or export to a predefined format:
Straka, Milan; Náplava, Jakub; Straková, Jana and Samuel, David, 2021,
RobeCzech Base, LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL),
http://hdl.handle.net/11234/1-3691.
Authors
Item identifier
Date issued
2021-05-25
Type
Size
1035 mb
Language(s)
Description
RobeCzech is a monolingual RoBERTa language representation model trained on Czech data. RoBERTa is a robustly optimized Transformer-based pretraining approach. We show that RobeCzech considerably outperforms equally-sized multilingual and Czech-trained contextualized language representation models, surpasses current state of the art in all five evaluated NLP tasks and reaches state-of-theart results in four of them. The RobeCzech model is released publicly at https://hdl.handle.net/11234/1-3691 and https://huggingface.co/ufal/robeczech-base, both for PyTorch and TensorFlow.
Acknowledgement
Grantová agentura České republiky
Project code:GX20-16819X
Project name:LUSyD – Language Understanding: from Syntax to Discourse
Grantová agentura Univerzity Karlovy v Praze
Project code:GAUK 578218
Project name:Automatická korekce jazyka pomocí neuronových sítí
Mellon Foundation, Brandeis University
Project code:G-1901-06505
Project name:Transatlantic Collaboration between LAPPS Grid and CLARIN: Implementation of NLP-enabled Tools using Text Archives as a Use Case
Collections
This item isPublicly Available
and licensed under:
Files in this item
- Name
- robeczech-base-tf.zip
- Size
- 589.1 MB
- Format
- application/zip
- Description
- Tensorflow Huggingface checkpoint of RobeCzech base model.
- MD5
- 621eafb6c5d8784089b2e5d091874d30
- Name
- robeczech-base-pytorch.zip
- Size
- 446.5 MB
- Format
- application/zip
- Description
- PyTorch Huggingface checkpoint of RobeCzech base model.
- MD5
- 2b3cfe1dd91f31f0963220e784b4a304