======================= EVALD 4.0 for Beginners ======================= EVALD 4.0 for Beginners is a software that serves for automatic evaluation of Czech texts written by non-native speakers of Czech – language beginners. The evaluation part (the server) is implemented in Treex (http://ufal.cz/treex), a highly modular NLP framework written in the Perl programming language, and uses the Weka toolkit (http://www.cs.waikato.ac.nz/ml/weka/) for the final prediction of a coherence mark. It can be used directly from a command line or as a backend server for a client. The client part is implemented as a web server, accessible with a web browser. =============================================================== There are three possible ways of using EVALD 4.0 for Beginners: =============================================================== 1. interactively as a web demo and RESTful web service hosted at the LINDAT/CLARIN server, 2. interactively but locally, with both the server and the client running on the same machine (or two machines in the same network), 3. in a batch mode run on the local machine. ================================================= Ad 1) No installation is needed in this case; in a web browser (such as Firefox or Chrome), go to https://lindat.mff.cuni.cz/services/evald-begin/. ================================================= Ad 2) Both the backend server and the client components are distributed via the Docker software (https://www.docker.com/), which needs to be installed first. Docker greatly simplifies the installation process of the two components and allows them to be run on Linux-based operating systems, Windows 10, as well as Mac OS X. To download both the server and the client, run the following commands: docker pull ufal/evald.treex-server:4.0 docker pull ufal/evald.php-server:4.0 See the EVALD 4.0 for Beginners project web pages (http://ufal.mff.cuni.cz/evald/documentation) for further instructions. ================================================= Ad 3) The dockerized backend component can be run locally also in a batch mode. In a command line, the user must specify paths to input and output directories. All files in the input directory are then processed and the result is stored in the output directory. In order (for more experienced users) to gain better control over the processing pipeline, the backend application can be installed also without being wrapped by a Docker container. Treex (http://ufal.cz/treex) needs to be installed on the local machine, along with all (mostly CPAN) dependencies for the Czech text analysis (please note that in this case, some parts of the Czech analysis can only be run on Linux-based systems). Treex must be in the revision tagged as EVALD_4.0 (https://github.com/ufal/treex/releases/tag/EVALD_4.0). In addition, Vowpal Wabbit 8.1.1 (https://github.com/JohnLangford/vowpal_wabbit/releases/tag/8.1.1) must be installed to the location installed_tools/ml/vowpal_wabbit-v8.1-3cf3f692/ relative to the Treex Share directory. The Treex scenario to be run is a part of this Lindat EVALD 4.0 for Beginners distribution (file Evald-4.0-Begin.scen). See the EVALD 4.0 for Beginners project web pages (http://ufal.mff.cuni.cz/evald/documentation) for further instructions. ============ Further Info ============ EVALD 4.0 for Beginners project web pages: https://ufal.mff.cuni.cz/evald/evald-beginners If you have questions or need technical support, please contact evald@ufal.mff.cuni.cz. =================================== How to Cite EVALD 4.0 for Beginners =================================== Michal Novák, Jiří Mírovský, Kateřina Rysová, Magdaléna Rysová, Eva Hajičová: EVALD 4.0 for Beginners – Evaluator of Discourse. Data/software, LINDAT/CLARIN digital library, Prague, Czech Republic, http://hdl.handle.net/11234/1-3067, Oct 2019. There are also papers describing the related research and experiments: Kateřina Rysová, Magdaléna Rysová, Michal Novák, Jiří Mírovský, Eva Hajičová: EVALD – a Pioneer Application for Automated Essay Scoring in Czech. In: The Prague Bulletin of Mathematical Linguistics, Vol. 113, Univerzita Karlova, Prague, Czech Republic, ISSN 0032-6585, pp. 9–30, Oct 2019. WWW: https://ufal.mff.cuni.cz/pbml/113/art-rysova-et-al.pdf Michal Novák, Jiří Mírovský, Kateřina Rysová, Magdaléna Rysová: Exploiting Large Unlabeled Data in Automatic Evaluation of Coherence in Czech. In: Lecture Notes in Computer Science, Vol. 11697, Proceedings of the 22nd International Conference on Text, Speech and Dialogue – TSD 2019, Springer International Publishing, Cham / Heidelberg / New York / Dordrecht / London, ISBN 978-3-030-27946-2, ISSN 0302-9743, pp. 197–210, 2019. WWW: https://link.springer.com/chapter/10.1007%2F978-3-030-27947-9_17 Michal Novák, Jiří Mírovský, Kateřina Rysová, Magdaléna Rysová: Topic–Focus Articulation: A Third Pillar of Automatic Evaluation of Text Coherence. In: Advances in Computational Intelligence (LNAI 11289): 17th Mexican International Conference on Artificial Intelligence, MICAI 2018, Proceedings, Part II, Springer, Switzerland, ISBN 978-3-030-04497-8, pp. 92–105, 2018. WWW: https://link.springer.com/chapter/10.1007/978-3-030-04497-8_8 Michal Novák, Kateřina Rysová, Magdaléna Rysová, Jiří Mírovský: Incorporating Coreference to Automatic Evaluation of Coherence in Essays. In: Statistical Language and Speech Processing, Springer International Publishing, Cham, Switzerland, ISBN 978-3-319-68455-0, ISSN 1611-3349, pp. 58–69, 2017. WWW: https://link.springer.com/content/pdf/10.1007%2F978-3-319-68456-7_5.pdf Kateřina Rysová, Magdaléna Rysová, Jiří Mírovský: Automatic Evaluation of Surface Coherence in L2 Texts in Czech. In: Proceedings of the 28th Conference on Computational Linguistics and Speech Processing ROCLING XXVIII (2016), The Association for Computational Linguistics and Chinese Language Processing (ACLCLP), Taipei, Taiwan, ISBN 978-957-30792-9-3, pp. 214–228, 2016. WWW: http://aclweb.org/anthology/O/O16/O16-1021.pdf =============== Acknowledgement =============== EVALD 4.0 for Beginners was developed at the Institute of Formal and Applied Linguistics (ÚFAL, http://ufal.mff.cuni.cz/), Faculty of Mathematics and Physics, Charles University, with the financial support of the Ministry of Culture of the Czech Republic, project Automatic Evaluation of Text Coherence in Czech (DG16P02B016, http://ufal.mff.cuni.cz/grants/evald-evaluator-discourse).