The aim of the article is to quantify how often in leading Czech social-science journals (Československá psychologie / Czechoslovak Psychology, Pedagogika/Pedagogy, and Sociologický časopis / Czech Sociological Review) authors choose the wrong procedures to analyse quantitative data. In particular, attention is focused on the incorrect choice of statistical tests, their misinterpretation and mechanical application, and the use of effect sizes, that are so highly recommended nowadays. The basic research period was ten years, from 2005 to 2014, and for the Czech Sociological Review the period was extended back to 1995. The results of the content analysis of published articles (N=363) show that statistical tests are applied quite often to data that are not suitable for statistical tests: this is found in about one-fifth of cases in Czech Sociological Review, one-half in Pedagogy, and more than three-quarters in Czechoslovak Psychology. In addition, authors often make mechanical use of statistical methods or make incorrect interpretations (in over 40% of articles in the Czech Sociological Review over the last 10 years) and there are rarely any substantive interpretations of results (especially in Czechoslovak Psychology). Effect sizes are applied relatively often, but there are also gaps in their usage. It is clear from the results that changes are necessary both in the teaching of quantitative methodology and publishing practices in this subject area.
In this article it is argued that one of the main problems in data analysis is an over-emphasis on statistical rather than substantive significance. Statistical significance reports the improbability of specific outcomes from sample data using a null hypothesis. In contrast, substantive significance is concerned with the real-world meaning of data modelling results for a population, regardless of p value, where an effect size estimator is used for evaluation. The argument presented in this article begins with a consideration of how substantive significance may be defined. Thereafter, there is a summary of the literature on substantive significance and its measurement using a variety of effect size estimators, many of which are little known to researchers. This article also examines the topics of economic and clinical significance. In the conclusion, this study discusses attempts to synthesise different concepts of substantive significance and recommends some practical usage of these concepts.
In this article it is argued that one of the main problems in data analysis is an over-emphasis on statistical rather than substantive significance. Statistical significance reports the improbability of specific outcomes from sample data using a null hypothesis. In contrast, substantive significance is concerned with the real-world meaning of data modelling results for a population, regardless of p value, where an effect size estimator is used for evaluation. The argument presented in this article begins with a consideration of how substantive significance may be defined. Thereafter, there is a summary of the literature on substantive significance and its measurement using a variety of effect size estimators, many of which are little known to researchers. This article also examines the topics of economic and clinical significance. In the conclusion, this study discusses attempts to synthesise different concepts of substantive significance and recommends some practical usage of these concepts., Petr Soukup., and Obsahuje bibliografii