RobeCzech Base

  Authors

Straka, Milan ; Náplava, Jakub ; Straková, Jana and Samuel, David

  Item identifier

http://hdl.handle.net/11234/1-3691

 Date issued

2021-05-25

 Type

languageDescription, text

 Size

1035 mb

 Language(s)

Czech

 Description

RobeCzech is a monolingual RoBERTa language representation model trained on Czech data. RoBERTa is a robustly optimized Transformer-based pretraining approach. We show that RobeCzech considerably outperforms equally-sized multilingual and Czech-trained contextualized language representation models, surpasses current state of the art in all five evaluated NLP tasks and reaches state-of-theart results in four of them. The RobeCzech model is released publicly at https://hdl.handle.net/11234/1-3691 and https://huggingface.co/ufal/robeczech-base, both for PyTorch and TensorFlow.

 Publisher

Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics (UFAL)

 Acknowledgement

Grantová agentura České republiky

Project code: GX20-16819X

Project name: LUSyD – Language Understanding: from Syntax to Discourse

 

Grantová agentura Univerzity Karlovy v Praze

Project code: GAUK 578218

Project name: Automatická korekce jazyka pomocí neuronových sítí

 

Mellon Foundation, Brandeis University

Project code: G-1901-06505

Project name: Transatlantic Collaboration between LAPPS Grid and CLARIN: Implementation of NLP-enabled Tools using Text Archives as a Use Case

 Subject(s)

Czech BERT RoBERTa

 Collection(s)

LINDAT / CLARIAH-CZ Data & Tools

Show full item record