Novice teachers in the face of assessment: analysis of learning situations in the MIR for teachers in the Community of Madrid

Subject of study

In line with the above, this research focuses on studying how formative assessment, which must be integrated into the SA programmed in the final activity of the educational MIR, is reflected. Specifically, the research objectives are as follows:

To analyse what types of assessment new teachers consider in the programmed SA, checking whether the formative assessment required by the SA is used.

Discover what types of assessment tools teachers use in their SAs.

Check the consistency in the use of assessment instruments in the SA.

Participants

This research focuses on the study of 527 teaching plans programmed and delivered in Spanish by novice teachers who were studying for the MIR educational qualification in the Community of Madrid. Specifically, the novice teachers delivered the presentation associated with the programming of a teaching plan on a free topic that had been assigned to them 15 days in advance.

From these 527 documents, a sample of 390 was selected at random for analysis in this study. Therefore, based on the size of the population and the sample, the data analysed with a 95% confidence level has only a 5% margin of error. With regard to the teachers who programmed the 390 SA designed, it should be noted that 278 are women, 56 are men, and in 56 cases no data has been identified to show the gender of the participants.

Quantitative analysis procedure

The quantitative analysis data is derived from a prior inductive analysis of the SA. To this end, the researchers created an Excel template where variables (educational area, educational level, types and instruments of assessment) were counted and recorded. After filling in the data in Excel, the document was imported into the SPSS IBM Statistics 29 programme, from which two types of analysis were performed: 1) descriptive, which allows basic frequencies to be obtained for easy reading of the results (Hidalgo Troya, 2019); and 2) inferential, based on comparison using cross tables and Pearson's X2 test.

Qualitative analysis procedure

The quantitative study of the data has been enriched in this research with a qualitative analysis of the same. This qualitative analysis helps to deepen the research results given that ‘[...] it studies the association or relationship between the variables that have been quantified, which further aids in the interpretation of the results’ (Sarduy Domínguez, 2007, p. 5). After a preliminary general reading of the information, a deductive coding of the data was carried out to perform a qualitative analysis for thematic robustness; along these lines, the categories and subcategories of analysis shown in Table 1 were established.

Quantitative results

The results derived from the descriptive analysis are presented below. First, the results obtained for the courses in which teachers who have completed the educational MIR have decided to schedule their SA are shown. As we can see in Table 2, the course in which most teachers decide to schedule is the 5th year, and the course in which the fewest schedules have been recorded is the 2nd year. If we analyse the results by cycle, we see that the cycle in which the fewest SAs have been programmed is the first cycle, and the cycle in which the most have been programmed is the third (frequency 62, 146 and 169 respectively). Furthermore, we can see that most teachers prefer to programme in the first years of the cycle (1^st, 3^rd and 5^th), with the last years having the lowest number of SAs.

By studying the relationship between the course in which it is programmed and the rest of the variables using the χ² test, we can see that there are significant differences in terms of teacher-student self-assessment (p value = .032). Upon in-depth analysis of this relationship, we can observe (Table 3) that in the first cycle courses, the relationship between teachers who use student self-assessment and those who do not is more balanced; although more teachers use it than do not, the frequency is similar. In the second cycle, there is a discrepancy between the courses, as there is greater use of student self-assessment in the SA programmed for the third year than for the fourth year. Furthermore, the fourth year is the only year for which more SA have been designed that do not contain self-assessment compared to those that do. In the third cycle, we find more SAs with self-assessment than those that do not use this assessment, with it being more prevalent in the fifth year than in the sixth year.

Secondly, we move on to study the SA programmed by main area. According to Table 4, the main area in which teachers programmed most frequently is Spanish Language and Literature, followed by Mathematics. In third place is Natural Sciences (CCNN) and in fourth place is Social Sciences (CCSS), with an anecdotal appearance for Social and Civic Values and Art Education.

When contrasting this variable using the χ² test, several significant differences are detected. The first is recorded in the area of Spanish Language and Literature, where there are significant differences regarding the inclusion of self-assessment (p = .003). Specifically, when studying the case, clear differences are observed in the number of SAs that include self-assessment in this area (100 SAs with self-assessment) in contrast to those that do not include it (47 SAs that do not contemplate student self-assessment). This same relationship, but in reverse, is found in the area of mathematics (p = .03), where there are more teaching plans that do not include self-assessment (276) than those that do (114).

The second significant relationship is observed between the planned teaching and learning activities in the area of Natural Sciences and teacher self-assessment. On this occasion, the same relationship is identified as in the area of language, with teaching and learning activities involving self-assessment being more frequent in this area. Specifically, there were 52 teaching and learning activities involving student self-assessment compared to 28 without it (p = 0.043).

Thirdly, the types of assessment selected by teachers in the SA developed during the MIR are studied. These types are teacher-student hetero-assessment (HM-A), co-assessment or peer assessment (CE), student self-assessment (AA), teacher self-assessment and/or Learning Situation self-assessment (AMSA), and student-teacher hetero-assessment (HA-D). Table 5 shows that the most common type of assessment is hetero-assessment, followed by teacher and/or learning situation self-assessment and student self-assessment. It is noteworthy that only 44.4% of the SA include co-evaluation processes and that only 10.5% of these include student-teacher hetero-evaluation processes.

Regarding the relationship between the types of assessment using Pearson's chi-square test, the following relationships are noteworthy. Firstly, the analysis of the relationships between student self-assessment and other types of assessment shows significance in the relationships with peer assessment (HM-A) (p < 0.001), co-assessment (p < 0.001) and teacher self-assessment (AMSA) (p = 0.005). In the first case, although the relationship is significant, it does not provide much information, since, except for the missing cases, all teachers perform HM-A, so the figures stand out, although their interpretation establishes them as something common.

In the second case, we find the results presented in Table 6, which shows that most teachers who carry out student self-assessment (SSA) also take into account other types of assessment such as peer assessment (PA) and teacher self-assessment (TSA). However, we see that this relationship is more pronounced in the relationship between teacher self-assessment (TSA) and student self-assessment (SSA), since only 30 teachers who choose to conduct student self-assessment have not chosen to self-assess themselves or the SA designed.

Finally, the analysis of co-evaluation in relation to the other types of evaluation is shown (Table 7). In this case, only one significant result is obtained when relating co-evaluation (CE) to student-teacher hetero-evaluation (HA-M) (p = 0.009). When studying this case, we must remember that only 41 teachers have included this type of assessment in their AS, and within such a low N, the data show that among the teachers who do include HA-M, the majority also include CE.

Dimensions

This section describes the dimensions that make up our object of study: the assessment proposals made by teachers enrolled in the educational MIR programme. To this end, we will delve into the type of assessment, what they use to assess (instrument), how they do it (methodology) and grading.

Type of assessment

With regard to the type of assessment, it has been observed that most teachers tend to opt for: student self-assessment, peer assessment, teacher-student assessment, teacher self-assessment, student-teacher assessment, or to propose another form of assessment by selecting the ‘other’ option. It should be clarified that it is possible to choose one or more types of assessment. In this regard, some teachers choose to select only one type of assessment (D7), but the vast majority select more than one (SA153, SA174). In this selection, all teachers, without exception, choose teacher-to-student assessment (SA12, SA14, SA78, SA106), while very few select student-to-teacher assessment (SA11, SA391).

On the other hand, although some teachers do not select any type of self-assessment (SA54, SA158), the most common choice is teacher self-assessment (SA5, SA17), followed by student self-assessment (SA42, SA191). With regard to the possibility of proposing another form of assessment, the most common option is to resort to continuous assessment (SA54, SA156).

Finally, peer assessment is rarely selected among teachers (SA19, SA391) and, as will be detailed in the ‘inconsistencies’ category, some of those who choose this option do so incorrectly.

Techniques and instruments

Some SAs have a very comprehensive section on techniques and instruments (SA15, SA23, SA331, SA384) used for assessment. They present a wide variety of instruments, such as rubrics, templates, SWOT analyses, daily anecdotes, checklists, student notebooks, targets, scales, written and oral tests, among others. It should be noted that the most commonly used techniques and instruments are rubrics (SA16, SA369), estimation scales (SA46, SA75) or Likert scales (SA52, SA73), co-evaluation (SA64, SA370) and written tests (SA103, SA145).

Oral tests, also known as oral presentations, are frequently mentioned, especially by teachers who present an SA in the area of Spanish Language (SA6, SA175), although they are also used in SAs in other areas, especially in mathematics, which relate the SA to the area of Spanish Language (SA151, SA389).

Methodology

One of the most widely used methods is peer assessment, which is usually associated with evaluating attitude, behaviour and/or teamwork (SA3, SA42). Teamwork (SA230, SA348), thinking routines (SA1, SA73) and cooperative work (SA333, SA381) are also commonly mentioned and often appear in content assessment. With regard to the latter, expert groups (SA333, SA381), rotating folios (SA377, SA384), followed by expert minds (SA381) or pencils in the centre (SA377), are the most common activities.

Rating

Two groups can be distinguished in terms of grading. On the one hand, there are those who prefer not to specify how the grading process will be carried out and develop the assessment process extensively (SA37, SA280). On the other hand, there are those who opt for a grading system based on continuous, formative and summative assessment. For this reason, in the grading section, they detail the percentages that will be taken into account for the final grade (SA29, SA384). Among the most common are percentages for: final product, individual tasks, group tasks, oral presentation, participation and attitude.

Inconsistencies

Terminological confusion

Peer assessment is one of the most frequently cited assessment methods (SA249, SA65), but none of these citations mention external reviewers such as families, teachers of other subjects, etc. Nor is there any mention of student-teacher assessment in most cases. These data indicate that peer assessment is understood as the most traditional form of assessment: teacher-student assessment.

On the other hand, there is terminological confusion regarding the instruments used for the assessment process, SA321 being an example of this. This Learning Situation describes co-assessment using rubrics, but the instrument used is a graded, non-descriptive scale.

As we pointed out in the previous section, the SA analysed do not always explain the grade. This fact shows that teachers have begun to separate these concepts and there are SA with complete assessment processes that include different processes and instruments but do not reflect the jump to the grade (SA176, SA210). However, others maintain this confusion between concepts and link assessment with summative grades (SA45, SA211), which leads them to a final grade for the Learning Situation.

Type of assessment inconsistent with objects and/or techniques

SA11 does not mention student self-assessment as a method; however, in the assessment objectives, it explains that students assess the project they have carried out, evaluating themselves and their peers. This shows that there is an inconsistency between the assessment objectives and the chosen method.

Similarly, SA1 mentions that a student-teacher assessment is carried out, but this process is not reflected in the assessment objectives (initial registration form with a thinking routine, cooperative groups to complete the initial form, individual assessment of an activity) or in the techniques and tools (initial thinking routine, radio programme and self-assessment).

For its part, SA7 selects as its assessment tool or technique: the rubric for assessing SA and self-assessment for students and teachers. The only subject of study it mentions is the assessment of SA, and for grading it opts for hetero-assessment using a rubric. As can be seen, although different methods and techniques are mentioned, grading is summarised in a hetero-assessment that is not explained how it is carried out. Thus, the assessment process proposed with the grading is inconsistent, as the self-assessment by students or teachers is not reflected in the grading in any way.