Learning

Since the 1990’s, the National Institute for Educational Studies and Research “Anísio Teixeira” (Inep) has implemented the Saeb in order to analyze student learning by administering proficiency tests to students in a sample of public and private schools. From 2005 onwards, the Saeb expanded its scope to include a sample-based part, with public and private schools - i.e., the National Basic Education Assessment (Aneb) - and a nearly census-like part with only public primary and secondary education schools, i.e., the National School Performance Assessment (Anresc), known as Prova Brasil.³

In Saeb assessments, learning is defined as the skills students demonstrate to have mastered regarding Portuguese (emphasis on reading) and mathematics (problem solving) by the end of each stage of primary and lower secondary education. Obviously, school learning is not restricted to these areas, and many skills developed at school cannot be assessed by external evaluation (^{VIANNA, 2003}). However, for a large-scale analysis, the tests’ results - called proficiency - are taken as evidence of this learning in schools, since the focus is not on individual student assessment (^{FONTANIVE, 2013}).

There is a vast literature explaining the conception and technical aspects of Saeb (^{FONTANIVE; KLEIN, 2000}; KLEIN, 2003). To date, there is no official interpretation for the Saeb scale, which, according to the PNE, is overdue. One of the plan’s strategies provides that by the middle of the decade at least 70% of students will have achieved a “sufficient” learning level, and at least 50% a “desirable” level for each stage of basic education (BRAZIL, 2014).

With no official precept in place, we will use in this paper the interpretation proposed by ^{Soares (2009}), which divides the Saeb scale into four levels: below basic, basic, adequate and advanced. The author used a proficiency distribution correspondence technique to create a benchmark distribution for the Saeb scale.⁴ According to this proposal, it would be ideally acceptable for only 5% of students to be at the first level (below basic), and at least 25% of students should be at the highest level (advanced). The other two levels were defined considering that percentile 70 of the benchmark distribution separates the students who have met learning expectations from those who have not. Based on these assumptions, the cut off points were defined and named according to Chart 1.

Student Grade Promotion

A student is promoted to the next grade level if her/his academic achievement meets the system’s standard requirements; otherwise, she/he has to repeat that grade level . There are also cases where the student ceases to attend school, which characterizes a dropout (^{RIGOTTI; CERQUEIRA, 2004}). In an effective education system, grade repetition is expected to be a rare event, dropout never happens, and children in the same cohort will stay together and complete basic education within the regular time period.

In Brazil, since the 1930’s, official statistics have shown high grade repetition rates (^{GIL, 2018}). Although widely used as a pedagogical resource, grade repetition has proved ineffective and has been associated with poor education quality and with exclusion of the poor (^{RIBEIRO, 1991}). The repeat student does not overcome obstacles by doing it all over again; on the contrary, he/she is more likely to fail again (^{CRAHAY, 2006}; ^{FERRÃO; COSTA; MATOS, 2017}), to undergo early transfer to Youth and Adult Education (EJA) (^{PEREIRA; OLIVEIRA, 2018}) and to drop out without having completed basic education (^{SIMÕES, 2016}). All empirical studies based on educational assessment data suggest that students who have not been promoted are found to perform worse than those who have never failed a grade (FERRÃO; BELTRÃO; SANTOS, 2002a; ^{KARINO; LAROS, 2017}; ^{KLEIN, 2006}).

Grade repetition, grade progression, dropout and student flow rates are all determined based on the Census of Basic Education; however, one cannot combine data from the Census and data from Prova Brasil, except within the Inep. Therefore, we use information from the Prova Brasil student questionnaire, which provides student age (5^th graders) or year of birth (9^th graders) and birthday month, which allows calculating the age-grade correspondence. There are also items about failure and dropout experiences. In educational research, the age-grade lag variable is most commonly found as a control variable (^{ANDRADE; SOARES, 2008}; ^{BARBOSA; FERNANDES, 2001}; SOARES; ^{ALVES; XAVIER, 2016}; ^{KASMIRSKI; GUSMÃO; RIBEIRO, 2017}; ^{FERRÃO et al. 2001}; FERRÃO; BELTRÃO; SANTOS, 2002a, 2002b). However, when the focus is on the grade repetition phenomenon, previous failure experiences are more often used (ALVES; ORTIGÃO; ^{FRANCO, 2007}; ORTIGÃO; AGUIAR, 2013; FERRÃO; COSTA; MATOS, 2017).

In the Prova Brasil databases, we found students above the adequate age for a particular grade, but who never failed or left school. Therefore, in this paper, we chose to analyze the student’s self-declaration about grade repetition and take this answer as a proxy about their educational trajectory. The dropout phenomenon is not so common in primary and lower secondary education levels and when it does occur, it usually entails repetition of that grade. We therefore assume the variable on failure to captures both phenomena.

Educational inequality

The relationship between school results and social background is one of the most studied phenomena in educational research, a subject that intermingles with the origin of sociology of education (^{FORQUIN, 1995}). Research on school effectiveness and improvement has shown very promising results in mitigating inequalities (^{KASMIRSKI; GUSMÃO; RIBEIRO, 2017}; ^{MORTIMORE; WHITTY, 1997}), specifically in identifying the factors of school and/or teacher effectiveness that contribute to mitigate the inequalities that prevent the child or youth from achieving full development. Thus, the large number of studies is justified by the problem’s persistence, despite the use of various policies to mitigate the effects of social disadvantage and promote greater equity in education.

An education system or school is considered equitable if its policies and practices can reduce the negative impact of social background (^{CASASSUS, 2007}). In Brazil, research shows that when performance improves, inequalities tend to persist or even increase in relation to socioeconomic status (^{FRANCO et al., 2007}), race/color (^{ALVES; ORTIGÃO; FRANCO, 2007}; ^{SOARES; ALVES, 2003}), gender (ALVES; SOARES; XAVIER, 2016; XAVIER; ALVES, 2015) and place of residence (^{ÉRNICA; BATISTA, 2012}; ^{KOSLINSKI; ALVES; LANGE, 2013}). Since basic education is a constitutional right, such persistent inequality patterns challenge educational policies.

The problem of quality distribution between groups can be analyzed through the notion of school justice, whereby equity is understood as the situation in which all students have educational outcomes above a certain level, regardless of their social background (^{RIBEIRO, 2013}). Obviously, one does not expect absolute equality, but rather the equivalence of results between social groups. This idea of j ustice is grounded, for example, in ^{Crahay (2000}, 2013), for whom education systems must guarantee basic knowledge for all students, assuming that differences will exist beyond that basis. Likewise, ^{Dubet (2004}) argues that the school should promote social justice and guarantee a minimum of resources to the disadvantaged, thus setting limits to the meritocratic system that excludes the weak.

Analyzing external evaluation proficiency by means of levels is a way of operationalizing this notion of school justice. Students who fail to reach a certain learning level are the ones served by an education system that fails to guarantee basic knowledge equity for all social groups (^{ALVES et al., 2017}; ^{KASMIRSKI; GUSMAO; RIBEIRO, 2017}; ^{SOARES, 2009}; SOARES; DELGADO, 2016; ^{XAVIER; ALVES, 2015}).

With regard to grade repetition, the idea of s chool justice is almost like an antonym of it, since failing the student is associated with increasing inequality from the beginning of his/her school trajectory (^{CORREA; BONAMINO; SOARES, 2014}; ^{LOUZANO, 2013}). A school justice framework applied to grade repetition can be found in the PNE. The target for primary education provides that at least 95% of students will complete this stage at the recommended age, which would give an “acceptable” margin of 5% of retained students during the nine years of primary and lower secondary education. This target is still far away because in 2017, 76% of students completed primary education at the recommended age, but with poor students this percentage was only 63% (BRASIL, 2018b).

What do we know about the evolution of quality in education?

The quality of the education provided for the Brazilian population has been a recurring subject in the literature since the beginning of this century (^{FERRÃO; BELTRÃO; SANTOS, 2002a}; ^{SOARES, 2004}), when large-scale education assessment data became available to researchers (^{GATTI, 2004}). Given the high percentage of students with an age-grade lag at the turn of the century, ^{Ferrão, Beltrão and Santos (2002a}, p. 52-53) affirm that,

Whatever measure is implemented to correct the age-grade lag, it must preserve the quality of the education provided for the population so as to ensure, in particular, that the formal promotion of the student corresponds to an actual promotion and, therefore, that a student who holds a certificate not only have the adequate age, but also master the knowledge and skills necessary for fully exercising citizenship. Otherwise, the public education system itself will impose educational exclusion. As Muñoz-Repizo (1999)⁵ says, being excluded from education today is equivalent to exclusion from work and to not count as a citizen or as a person.

Several authors have researched subjects related to either education quality or the correction of school flow. To our knowledge, there are no studies addressing both dimensions of the system’s performance since the Ideb was implemented in 2007. For example, ^{Carnoy et al. (2015}) analyzed the ranking of Brazilian students in the Programme for International Student Assessment (Pisa) from 2000 to 2012, and in the Saeb from 1995 to 2013 in order to assess the “effectiveness of Brazilian primary and lower secondary education (grades 1-8/9)” (^{CARNOY et al., 2015}, p. 450). Such factors were also mentioned by ^{Soares and Delgado (2016}, p. 773), who considered the student population in the 5^th and 9^th grades in their chronological analysis from 2005 to 2013, in which they found that “there have been improvements in both reading and mathematics”, although the improvement rate varied between groups defined by socioeconomic status, race/color and gender. This analysis was complemented by ^{Alves, Soares and Xavier (2016}, p. 49), who quantified the learning inequalities between the groups above to conclude that “where there has been quality improvement, there has been no decrease in inequalities”.

In analyzing the performance of eight-year-old students in Saeb from 1995 to 2003, ^{Alves (2007}, p. 537) noted “a slight decline in the quality of performance” and considered it

[...] very likely that the efforts of authorities to meet the demands imposed by the targets will have effects at different times. Probably, in the coming years, the first results of the policies implemented to regulate the school flow will begin to appear. Only after a while, with the school flow almost regularized (or stable at low levels), will the results on school performance be noticed. Thus, in the near future, we will possibly see once more a sharp drop in non-promotion rates, with no change in school performance improvement. (^{ALVES, 2007}, p. 539)

However, according to the National Household Sample Survey (INSTITUTO BRASILEIRO DE GEOGRAFIA E ESTATÍSTICA - IBGE, 2008), in 2008, the schooling rate for the school age population (6 to 14 years old) was 97.5%, and for the age group from 15 to 17 years old, it was 84%. School flow statistics also showed considerable improvement. Comparing the scenarios before and after the passing and implementation of the Education Guidelines and Framework Law (LDB/96, Law No. 9394/96) (BRASIL, 1996) allows observing some changes. Specifically, the LDB/96 encouraged the development and implementation of school flow correction proposals, such as learning acceleration classes, the cycle system, automatic promotion, among others. The effect of measures arising from LDB/96 is partially visible through the statistics reported by ^{Klein (2006}), which allowed a comparison between 1992 and 2003. In 1992, the average grade repetition rate was 35% for the 1^st to 4^th grades of primary education and 30% for the 5^th to 8^th grades of primary education; the highest completion rate achieved by an age cohort was 77% for the 4^th grade and 42% for the 8^th grade, while in 2003 the average repetition rate was 19.8% for the 1^st to 4^th grades and 18.5% for the 5^th to 8^th grades; the highest completion rate observed for a cohort was 89% for the 4^th grade and 65% for the 8^th grade.

From our knowledge to date, studies on Brazilian education quality evolution focus either on students’ performance in standardized tests or on aspects related to school flow correction. Therefore, in the context of progressive decrease in grade repetition rates, this article aims to find evidence that unmistakably demonstrates that, in the last few years, grade repetition has been decreasing without worsening school education quality, and to achieve this goal by answering the two research questions previously stated.

Learning Tendencies

Table 2 shows the distribution of students at state and municipal schools who took the Prova Brasil from 2007 and 2017 over four learning levels. Considering the 2007 baseline year, the change in the percentage of students who reached at least the “adequate” learning level reflects a consistent increase (a minimum growth of 13% in mathematics for the 9^th grade from 2007 to 2009; a maximum increase of 104% in reading for the 5^th grade from 2007 to 2015), which was greater in reading than in mathematics.

It is worth noting that since 2015, the percentage of 5^th graders who were below basic dropped to less than half what it was in 2007. At the other extreme, the percentage of students at the advanced level in 2017 increased fivefold compared to ten years earlier. In 2017, the combined percentages of adequate and advanced levels (57%) brought the 5^th grade reading result closer to the target of 70% of students at levels adequate and above (see Table 1). In mathematics, 5^th graders’ performance was also found to increase, though far from what would be adequate as 55% of students still had not reached the learning expectation by the end of the period.

With regard to 9th grade reading proficiency, students at the adequate and advanced levels increased threefold during the 2007-2017 period, but 65.5% did not reach the adequate level in 2017. In mathematics, the percentage of students who reached at least the adequate level grew by little more than 50% during the analyzed period, but in 2017 there were still 84% of students who did not reach adequate.

Table 3 shows the results by location and country region. For the sake of simplicity, results will henceforth be described as the percentage of students who have reached at least the adequate level. With regard to the whole of Brazil, there is clear improvement for all locations and regions, though important differences were found that confirm the known patterns of regional inequalities (^{CERQUEIRA; SAWYER; 2007}; ^{SOARES et al., 2012}; ^{FERRÃO et al., 2001}). In rural schools in the North and Northeast, there are systematically fewer students who achieved adequate proficiency than in urban schools in the South and Southeast. Results for the Central-West are in an intermediate range.

For the 5^th grade, the best reading results are in the South and Southeast regions, where respectively 68% and 67.4% of students reached the adequate level in 2017. For the 9^th grade, despite some improvement, all percentages are low, especially for students in rural schools in the Northeast and North. In mathematics, few students have reached the adequate level. In rural areas and in the North and Northeast, about 90% of students did not reach the learning target for the 9^th grade.

Tables displaying percentages of students by state are in tables A3 (reading) and A4 (mathematics) in the Appendix. There are some noteworthy exceptions in the North and Northeast, where, as seen earlier, fewer students reached the adequate level. In the North, the state of Acre has the best results since 2009 for the 5^th grade, and in 2011 for the 9^th grade, closely followed by the state of Rondônia. In the Northeast, results for the state of Ceará stand out both in regional and in national terms.

We then analyze the percentage of students who reached the adequate learning level according to social group, i.e., by gender, color/race and SES. Table 4 shows that female students perform better in reading than male students, and the opposite occurs in mathematics, a well-known pattern in the educational literature (EURYDICE, 2011; ^{ALVES; SOARES; XAVIER, 2016}; XAVIER; ALVES, 2015; ^{MARTÍNEZ; SERNA, 2018}). During the analyzed period, the percentage of girls who reached the adequate level in the 5^th grade increased by 34%. Boys advanced slightly less (31%). We also found growth, albeit more modest, in mathematics, i.e., around 24% for both groups; thus, the initial difference practically remained in 2017. That year, the difference between boys and girls at the adequate level in mathematics was only 1.3%; in reading, there were 11% more girls at this level.

For the 9^th grade, inequality patterns remained as in the first stage of primary education. However, boys’ advantage in mathematics increased slightly over the period (7% higher for boys at the adequate level in 2017). The tendency towards equity in the previous stage did not remain; on the contrary, girls’ disadvantage in mathematics increased.

With regard to the race/color variable, the learning patterns for the 5^th and 9^th grades in reading and mathematics are similar. The percentage of students who reached the adequate level increased for all groups, but the initial differences remained. Growth was higher for white students, except for reading for the 5^th grade, where brown students had the same increment pattern. As to black students, although the percentage who reached the adequate level also increased, the increase was smaller and their difference from whites increased in 2017. Students who declared themselves yellow showed a better performance than brown ones, while indigenous students had similar results to those for blacks.

In order to analyze the tendencies by family Socioeconomic Status (SES), the original continuous scale (0 to 10 points) was divided into quintiles for each year so as to maintain comparability. We found that the association between SES and the percentage of students who reached the adequate level is unmistakable. The highest percentage of students with adequate learning in all results presented so far appears in the highest SES quintile for reading for the 5^th grade (69%, in 2017). At the other extreme, the lowest percentages in all years are those of students in the lowest SES quintile, for the 9^th grade, for mathematics.

Tendencies about Student Promotion

Considering students who have never experienced grade repetition, i.e., the status of education success, Table 5 shows that the percentage of students in this situation increased from 2007 to 2017. However, in 2017 there are still 22% of students in the 5^th grade who declared that they had been previously retained, although this is a stage in which grade repetition is not recommended. This rate is 30% for 9^th graders.

The Southeast had the best result and closest to the target: in 2017, 84.4% of 5^th grade students had never been retained. The Central-West was the region that advanced the most during the analyzed period and surpassed the South in education success statistics for the 9^th grade. It is worth noting that the differences have decreased between urban and rural schools and between Southeast (greatest promotion figures) and Northeast (smallest promotion figures).

Among the states (Table A5, Appendix), Acre stands out in the North for its learning level. Rondônia, which also stood out for its learning, did not succeed as much in promoting students; on the contrary, there were almost 30% of students in the 5^th grade and 40% in the 9^th grade who said they had been retained once or more. In the Northeast, Ceará has the largest percentage of students who have always been promoted, far above the other states in the region, thus confirming the success of its educational policies to ensure student learning and decrease grade repetition. In the Southeast, São Paulo and Minas Gerais have the greatest number of students who have always been promoted. In the South, Santa Catarina is the positive case and Rio Grande do Sul, the negative one. In Mato Grosso, the increase in the percentage of students with education success from 2007 to 2015 is almost twice that of other states in the region.

The percentage of students who declared no repetition by gender, color/race and SES are shown in Table 6. We highlight three results: improvement for all groups during the period; the reduction of inequalities between self-declared white and black, white and brown, yellow and indigenous students, as well as between those of higher and lower SES; and the pattern of inequality that is practically unchanged between male and female students, the latter being much more successful in their school trajectory.

Multiplied Inequalities

We have so far described inequalities both in learning and in education success (non-repetition) according to group characteristics, one at a time. But inequalities are complex, and when groups are defined by multiple criteria, differences become even sharper, as shown by ^{Alves, Soares and Xavier (2016}) and Soares and Delgado (2016) based on data from Prova Brasil until 2013.

In line with previous research, we will show the learning gap between groups defined by multiple social criteria (gender, color/race and SES), while adding student history (‘never retained’ or ‘has been retained’) and considering the percentage of those who reached the adequate learning level. For this analysis, SES was divided by the median, and 24 groups⁸ were created. For the sake of simplicity, we present only the descriptive analyzes for 5^th graders in reading, who have the best performance. Table 7 shows these results, in which the percentages are in decreasing order considering the 2017 result.

The group of white girls with higher SES and no repetition has the best results for all years. In 2017, 82.8% of students in this group were at least at the adequate level. Since 2013, they have reached the 70% target, and by 2009 most had already shown adequate learning. In addition to this group, brown girls and white boys with higher SES and no repetition have also reached the over 70% mark since 2015. In 2017, the group of brown boys with higher SES and no repetition also reached this target.

In 2017, all groups of “no repetition” students exceeded the 50% mark at the adequate level, except one, i.e., black boys with lower SES (39%). The lowest values i n the table are found for groups with some grade repetition experience, the lowest percentage being for black students with lower SES, i.e., just above 20%.