Zobrazit minimální záznam

 
dc.contributor.author Galuščáková, Petra
dc.contributor.author Devaud, Romain
dc.contributor.author Gonzalez-Saez, Gabriela
dc.contributor.author Mulhem, Philippe
dc.contributor.author Goeuriot, Lorraine
dc.contributor.author Piroi, Florina
dc.contributor.author Popel, Martin
dc.date.accessioned 2023-05-22T16:24:01Z
dc.date.available 2023-05-22T16:24:01Z
dc.date.issued 2023-05-22
dc.identifier.uri http://hdl.handle.net/11234/1-5151
dc.description The collection comprises the relevance judgments used in the 2023 LongEval Information Retrieval Lab (https://clef-longeval.github.io/), organized at CLEF. It consists of three sets of relevance judgments: 1) Relevance judgments for the heldout queries from the LongEval Train Collection (http://hdl.handle.net/11234/1-5010). 2) Relevance judgments for the short-term persistence (sub-task A) queries from the LongEval Test Collection (http://hdl.handle.net/11234/1-5139). 3) Relevance judgments for the long-term persistence (sub-task B) queries from the LongEval Test Collection (http://hdl.handle.net/11234/1-5139). These judgments were provided by the Qwant search engine (https://www.qwant.com) and were generated using a click model. The click model output was based on the clicks of Qwant's users, but it mitigates noise from raw user clicks caused by positional bias and also better safeguards users' privacy. Consequently, it can serve as a reliable soft relevance estimate for evaluating and training models. The collection includes a total of 1,420 judgments for the heldout queries, with 74 considered highly relevant and 326 deemed relevant. For the short-term sub-task queries, there are 12,217 judgments, including 762 highly relevant and 2,608 relevant ones. As for the long-term sub-task queries, there are 13,467 judgments, with 936 being highly relevant and 2,899 relevant.
dc.language.iso fra
dc.language.iso eng
dc.publisher Université Grenoble Alpes
dc.relation.isreferencedby https://arxiv.org/abs/2303.03229
dc.rights Qwant LongEval Attribution-NonCommercial-ShareAlike License
dc.rights.uri https://lindat.mff.cuni.cz/repository/xmlui/page/Qwant_LongEval_BY-NC-SA_License
dc.source.uri https://clef-longeval.github.io/
dc.subject information retrieval
dc.subject automatic evaluation
dc.subject evaluation
dc.title LongEval Click-Model Relevance Judgements (Qrels)
dc.type corpus
metashare.ResourceInfo#ContentInfo.mediaType text
dc.rights.label PUB
has.files yes
branding LINDAT / CLARIAH-CZ
contact.person Petra Galuščáková galuscakova@gmail.com Université Grenoble Alpes
sponsor French Agence Nationale de la Recherche ANR-19-CE23-0029 Kodicare Other
sponsor Austrian Science Fund I4471-N Kodicare Other
sponsor Ministerstvo školství, mládeže a tělovýchovy České republiky LM2018101 LINDAT/CLARIAH-CZ: Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy nationalFunds
sponsor Ministerstvo školství, mládeže a tělovýchovy České republiky LM2023062 LINDAT/CLARIAH-CZ: Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy nationalFunds
size.info 27104 units
files.size 208284
files.count 1


 Soubory tohoto záznamu

Licenční kategorie:
Publicly Available

Licence: Qwant LongEval Attribution-NonCommercial-ShareAlike License
Attribution Required Noncommercial Share Alike
Icon
Název
longeval-relevance-judgements.zip
Velikost
203.4 KB
Formát
application/zip
Popis
Longeval Relevance Judgements
MD5
2d95047a282c311e86927e99c05704b6
 Stáhnout soubor

Zobrazit minimální záznam