C4Corpus (publicdomain part)

C4Corpus (publicdomain part)

LRT + Open Submissions

Authors: Gurevych, Iryna ; Habernal, Ivan and Zayed, Omnia

Item identifier: http://hdl.handle.net/11372/LRT-2209

Project URL: https://dkpro.github.io/dkpro-c4corpus/

Referenced by: http://www.lrec-conf.org/proceedings/lrec2016/pdf/388_Paper.pdf

Date issued: 2016-04-14

Type: corpus, text

Size: 10000000000 tokens

Language(s): Afrikaans , Arabic , Bulgarian , Croatian , Czech , Danish , Dutch , English , Estonian , Finnish , French , German , Hungarian , Indonesian , Italian , Japanese , Korean , Latvian , Lithuanian , Modern Greek (1453-) , Norwegian , Persian , Polish , Portuguese , Russian , Slovenian , Somali , Spanish , Swahili(macrolanguage) , Swedish , Tagalog , Thai , Turkish , Ukrainian , Undetermined , Vietnamese

Description: A large web corpus (over 10 billion tokens) licensed under CreativeCommons license family in 50+ languages that has been extracted from CommonCrawl, the largest publicly available general Web crawl to date with about 2 billion crawled URLs.

Publisher: Technische Universität Darmstadt

Subject(s): CommonCrawl Creative Commons Web corpus Amazon Web Services

Collection(s): LRT + Open Submissions Data & Tools

Show full item record

Files in this item

Download all files in item (84.91 MB)

This item is

Publicly Available

and licensed under:
Public Domain Mark (PD)

Name: Lic_publicdomain_Lang_af_NoBoilerplate_true_MinHtml_true-r-00009.seg-00000.warc.gz
Size: 851 bytes
Format: application/x-gzip
MD5: 5fa2ab45c8d2d3a3b45809ffaa9c09eb

Name: Lic_publicdomain_Lang_ar_NoBoilerplate_true_MinHtml_true-r-00021.seg-00000.warc.gz
Size: 62.62 KB
Format: application/x-gzip
MD5: cafa5a853b4fffed9fb7c81ffaae80bf

Name: Lic_publicdomain_Lang_bg_NoBoilerplate_true_MinHtml_true-r-00010.seg-00000.warc.gz
Size: 37.94 KB
Format: application/x-gzip
MD5: d291cd5deeff806db6dac27e7f31a816

Name: Lic_publicdomain_Lang_cs_NoBoilerplate_true_MinHtml_true-r-00022.seg-00000.warc.gz
Size: 2.21 KB
Format: application/x-gzip
MD5: fed503373e97e581ed2dacb999185c26

Name: Lic_publicdomain_Lang_da_NoBoilerplate_true_MinHtml_true-r-00004.seg-00000.warc.gz
Size: 2.56 KB
Format: application/x-gzip
MD5: 7a8f8b03417a91e543f821516056fc37

Name: Lic_publicdomain_Lang_de_NoBoilerplate_true_MinHtml_true-r-00008.seg-00000.warc.gz
Size: 208.56 KB
Format: application/x-gzip
MD5: 84a5f4d980c3a7784fe5c44cb095977c

Name: Lic_publicdomain_Lang_el_NoBoilerplate_true_MinHtml_true-r-00015.seg-00000.warc.gz
Size: 10.11 KB
Format: application/x-gzip
MD5: 46a1035ba227e9817309954432c921a6

Name: Lic_publicdomain_Lang_en_NoBoilerplate_true_MinHtml_true-r-00017.seg-00000.warc.gz
Size: 73.52 MB
Format: application/x-gzip
MD5: a4ca095d945ec3338ad157ac794192b2

Name: Lic_publicdomain_Lang_es_NoBoilerplate_true_MinHtml_true-r-00022.seg-00000.warc.gz
Size: 4.91 MB
Format: application/x-gzip
MD5: 900a30f1f159595c78c006bc9d75912d

Name: Lic_publicdomain_Lang_et_NoBoilerplate_true_MinHtml_true-r-00023.seg-00000.warc.gz
Size: 759 bytes
Format: application/x-gzip
MD5: 0d5a6aaefdae7cb922b87ba74efff946

Name: Lic_publicdomain_Lang_fa_NoBoilerplate_true_MinHtml_true-r-00004.seg-00000.warc.gz
Size: 29.18 KB
Format: application/x-gzip
MD5: 3df1b8cae322307c85890d56e3b4feff

Name: Lic_publicdomain_Lang_fi_NoBoilerplate_true_MinHtml_true-r-00012.seg-00000.warc.gz
Size: 190 KB
Format: application/x-gzip
MD5: c5a4e5f17ea95cb668a80677868b8e72

Name: Lic_publicdomain_Lang_fr_NoBoilerplate_true_MinHtml_true-r-00021.seg-00000.warc.gz
Size: 2.84 MB
Format: application/x-gzip
MD5: d81c7ef00f15dab384215c3e0a06ebd0

Name: Lic_publicdomain_Lang_hr_NoBoilerplate_true_MinHtml_true-r-00021.seg-00000.warc.gz
Size: 10.76 KB
Format: application/x-gzip
MD5: 68509cea87e36e9241aead807363a163

Name: Lic_publicdomain_Lang_hu_NoBoilerplate_true_MinHtml_true-r-00024.seg-00000.warc.gz
Size: 398.67 KB
Format: application/x-gzip
MD5: 1a7305b25d4013a6245febe67b36b8ac

Name: Lic_publicdomain_Lang_id_NoBoilerplate_true_MinHtml_true-r-00007.seg-00000.warc.gz
Size: 239.58 KB
Format: application/x-gzip
MD5: 1487be32b9a1dab8d3d903ade1cf37b2

Name: Lic_publicdomain_Lang_it_NoBoilerplate_true_MinHtml_true-r-00023.seg-00000.warc.gz
Size: 831.26 KB
Format: application/x-gzip
MD5: e23e233f2f82ef2c25874a11f7ecaa4f

Name: Lic_publicdomain_Lang_ja_NoBoilerplate_true_MinHtml_true-r-00004.seg-00000.warc.gz
Size: 11.47 KB
Format: application/x-gzip
MD5: e3ad404aa65691c3bac4798fab27b50f

Name: Lic_publicdomain_Lang_ko_NoBoilerplate_true_MinHtml_true-r-00018.seg-00000.warc.gz
Size: 11.32 KB
Format: application/x-gzip
MD5: 565d03dae2a0c4f1825a4483cf15c8a1

Name: Lic_publicdomain_Lang_lt_NoBoilerplate_true_MinHtml_true-r-00023.seg-00000.warc.gz
Size: 1.58 KB
Format: application/x-gzip
MD5: 74730fe8a76b422e5bb2c6b28834fe2f

Name: Lic_publicdomain_Lang_lv_NoBoilerplate_true_MinHtml_true-r-00025.seg-00000.warc.gz
Size: 1.63 KB
Format: application/x-gzip
MD5: 117ffcbc34f519f66a2d94a2f14031a5

Name: Lic_publicdomain_Lang_nl_NoBoilerplate_true_MinHtml_true-r-00015.seg-00000.warc.gz
Size: 13.96 KB
Format: application/x-gzip
MD5: 27cc4b3391289c54a3daddbdb72915b1

Name: Lic_publicdomain_Lang_no_NoBoilerplate_true_MinHtml_true-r-00018.seg-00000.warc.gz
Size: 84.05 KB
Format: application/x-gzip
MD5: 6a292adfa78baee6d768739691128996

Name: Lic_publicdomain_Lang_pl_NoBoilerplate_true_MinHtml_true-r-00015.seg-00000.warc.gz
Size: 102.69 KB
Format: application/x-gzip
MD5: 0b093028be51e1628de18afcb45f1d4b

Name: Lic_publicdomain_Lang_pt_NoBoilerplate_true_MinHtml_true-r-00023.seg-00000.warc.gz
Size: 196.11 KB
Format: application/x-gzip
MD5: 1e2bf067709585f6e08de56ec5c1f66a

Name: Lic_publicdomain_Lang_ru_NoBoilerplate_true_MinHtml_true-r-00024.seg-00000.warc.gz
Size: 85 KB
Format: application/x-gzip
MD5: 366f92911df0414a36f29c99b6eaa28b

Name: Lic_publicdomain_Lang_sl_NoBoilerplate_true_MinHtml_true-r-00015.seg-00000.warc.gz
Size: 1.56 KB
Format: application/x-gzip
MD5: 2a20bf71fe6fd1b380f09d577cb07643

Name: Lic_publicdomain_Lang_so_NoBoilerplate_true_MinHtml_true-r-00018.seg-00000.warc.gz
Size: 2.3 KB
Format: application/x-gzip
MD5: 65db2258fb6cefc459f1553e340b4a15

Name: Lic_publicdomain_Lang_sv_NoBoilerplate_true_MinHtml_true-r-00025.seg-00000.warc.gz
Size: 20.91 KB
Format: application/x-gzip
MD5: 02b457c3b0c4e1fba9b7bd513083d205

Name: Lic_publicdomain_Lang_sw_NoBoilerplate_true_MinHtml_true-r-00026.seg-00000.warc.gz
Size: 5.85 KB
Format: application/x-gzip
MD5: 8f2d3712cc1657b1b98f2da757c8ec8b

Name: Lic_publicdomain_Lang_th_NoBoilerplate_true_MinHtml_true-r-00011.seg-00000.warc.gz
Size: 102.62 KB
Format: application/x-gzip
MD5: 00ce2bc416d8084fdff1e97c6ad24dab

Name: Lic_publicdomain_Lang_tl_NoBoilerplate_true_MinHtml_true-r-00015.seg-00000.warc.gz
Size: 45.09 KB
Format: application/x-gzip
MD5: 277267fc1ebec66a88e2b94c7a3a36a5

Name: Lic_publicdomain_Lang_tr_NoBoilerplate_true_MinHtml_true-r-00021.seg-00000.warc.gz
Size: 29.59 KB
Format: application/x-gzip
MD5: c9cbaf7f1f9c5b2782ceda2fdaae954b

Name: Lic_publicdomain_Lang_uk_NoBoilerplate_true_MinHtml_true-r-00014.seg-00000.warc.gz
Size: 1.95 KB
Format: application/x-gzip
MD5: 4a75e3f4c722ca9b0c67639455392a30

Name: Lic_publicdomain_Lang_unknown_NoBoilerplate_true_MinHtml_true-r-00017.seg-00000.warc.gz
Size: 987.9 KB
Format: application/x-gzip
MD5: f59469c5775f3eeca670d37936d32d8e

Name: Lic_publicdomain_Lang_vi_NoBoilerplate_true_MinHtml_true-r-00012.seg-00000.warc.gz
Size: 3.63 KB
Format: application/x-gzip
MD5: 8a771141caecd61602369182342c9fc7