Abstract

MATOS, Daniel Abud Seabra. Reliability and agreement among evaluators: applications in the educational area. Est. Aval. Educ. [online]. 2014, vol.25, n.59, pp.298-324. ISSN 0103-6831.  https://doi.org/10.1822/255920142750.

The aims of this study were to: (1) investigate the strategies for verifying reliability and agreement among evaluators, focusing on the applications in the educational area; (2) conduct a review of the national literature on the techniques of reliability and agreement among judges and their areas of application; and (3) illustrate the application of the techniques of reliability and agreement among evaluators by analyzing the corrections of the Vestibular (college entrance exam) essays from one public university in Minas Gerais. We used the intraclass correlation coefficient to analyze the reliability and agreement among evaluators in the correction of the essays from 2005 to 2010. We identified little use, in the educational research, of agreement techniques among evaluators. As for the analysis of the correction of essays, some results were satisfactory (example: mean reliability of the evaluators for total scores of the essays) and others were unsatisfactory (example: low agreement in some criteria of correction).

Keywords : Rate of Reliability; Vestibular Examination; Evaluators; Essay..

        · abstract in Portuguese | Spanish     · text in Portuguese     · Portuguese ( pdf )