1 Introduction

Education has a transforming role in the development of citizens due to a series of factors, such as: the economic development of nations ( ^{CARVALHO; SOUSA, 2014} ), determination and distribution of income ( ^{NEAMTU, 2015} ) and the improvement of social markers ranging from, among others, crime rates, population health conditions, social cohesion, life expectancy and infant mortality rates ( ^{CARVALHO; SOUSA, 2014} ).

Such aforementioned factors, hand in hand with high values of current expenditure by governments and families, have raised the interest of economic and administration science in the Education area, generating new studies (GOMES et al ., 2016; ^{DUMCIUVIENE, 2015} ; ^{MELHUISH, 2013} ; ^{NEAMTU, 2015} ; SALGADO JUNIOR; NOVI, 2015; SALGADO JUNIOR et al ., 2016), which have focused on the debate related to “efficiency in Education” ( ^{WITTE; LÓPEZ-TORRES, 2015} ). The demand for such studies emerges from the necessity of governments to balance their expenditure in the area with other essential public services, such as health and transportation (JOHNES et al ., 2017).

When evaluating Education, it is necessary to understand that quality is the most important variable to explain economic development, outweighing quantity (SALGADO JUNIOR; NOVI, 2014). The difference in economic performance among countries is related to differences in their Education levels ( ^{MERCAN; SEZER, 2014} ). Similarly, due to internal quality differences, this phenomenon may be observed within the same country or Educational system (WU et al ., 2017).

The concept of quality in the educational area addresses structures, processes and educational outcomes ( ^{DAVOK, 2007} ). In this sense, two distinct dimensions are approached: (i) from the institutional point of view, referred to educational quality; (ii) and from the point of view on generated results at the central main agent, the student. In the first sense, efficiency, efficacy, effectiveness and relevance of educational systems and institutions are contemplated; and in the second sense, the development of learning skills on predicted contents, knowledge of scientific-literary culture, and technical knowledge for the labor market and social transformation are considered.

^{Dourado and Oliveira (2009)} acknowledge the recent advance in Brazil to access and coverage, especially in elementary school; however, it is necessary to improve learning process in an efficient way that improves the quality of teaching. Education is transversal to pedagogical, economic, social, cultural and political dynamics of a given society, according to ^{Darling-Hammond and Ascher (1991)} and Dourado et al. (2007). These authors claim that influencing factors are: from the social point of view, income concentration, social inequality, Education as a right, and institutional issues such as school organization and management that interfere with working conditions, school management processes, curricular dynamics, teacher training and teacher professionalization.

“There is a certain consensus that the school quality assessment theme needs to be addressed and that quality measurement must necessarily involve the use of different tools”¹ ( ^{GOUVEIA; SOUZA, 2013} , p. 841). The evaluation of the quality of Education remains a controversial issue in the literature and has been carried out through large-scale evaluations, which generally evaluate the performance of students in specific subjects, such as mathematics and reading ( ^{FERNANDES; GREMAUD, 2009} ).

Among the criticisms of current large-scale evaluation models, it is possible to mention the limited focus, which restricts the evaluation of a multi-context and facets system to some subjects to the detriment of other expected school objectives, such as ethics and the development of some non-cognitive skills as well as the non-identification of other social development factors related to Education ( ^{FERNANDES; GREMAUD, 2009} ).

Brazil has not presented satisfactory results in those evaluations. In the Program for International Students Assessment (Pisa), Brazil presented, in the 2015 edition, an average of 407 points in reading in comparison with the average of 493 points from other evaluated countries (OECD, 2015).

Inside national scope, the Development of Basic Education Index (Ideb) is the main quality indicator available. It is composed by two components: approval index and performance on the Prova Brasil (Brazil Exam). The introduction of Ideb by the federal government represented a significant change in the way Education has been monitored in Brazil. Passing through Ideb, the evolution of educational services offered inside the country raised, this being possible thanks to tracking the results by state, municipality and school, providing a comparative value in relation to goals, and also comparing to other schools and municipalities ( ^{ALVES; SOARES, 2013} ).

In the Brazil Exam, the largest instrument for evaluating Brazilian basic Education, the average score in the subject of Portuguese language, in the 2015 edition, was 252 points, which was below the score obtained by students in 1995, when 256 average points were reached in the final years of elementary school ( ^{INEP, 2016} ). In spite of the growing current expenditure in recent years, this diagnosis demonstrates the need for improvements in the country’s educational process.

Despite the strong connection identified among extra-school factors, such as the socioeconomic level of the student and family, and their educational results (ALVES et al., 2013; MATOS et al., 2017), several studies pinpoint the importance of the “school effect”, which is a part of the result of students that can be attributed to the pedagogical and management practices adopted in schools ( ^{ALVES; SOARES, 2007} ; BLOOM et al., 2015; GONÇALVES; FRANCE, 2013; ^{GOUVEIA; SOUZA, 2013} ; SALGADO JUNIOR; NOVI, 2015; SALGADO JUNIOR et al., 2016).

Several studies have indicated the relevance of intra-school factors in student performance, such as administrative management (BLOOM et al., 2015), low remuneration of professionals, lack of training and inadequate work environment (AMÂNCIO-VIEIRA et al., 2015), principal profile in pedagogical results ( ^{DHUEY; SMITH, 2014} ), innovative didactic project ( ^{DOLZ, 2016} ), offer of classes during school holidays to students at disproof risk (VANDECANDELAERE et al., 2016) and emotional factors ( ^{PARISOTTO; RINALDI, 2016} ).

The identification of these practices is fundamental, as they can contribute to the improvement of the performance of the educational service and the results of students, allowing a systemic view of Education that encompasses pedagogical aspects as well as those related to management activities (SALGADO JUNIOR; NOVI, 2015).

In addition, since Education is a process of aggregation of knowledge, the studies that focus on improving elementary school, which is one of the most important level of Education, are essential as the experience of a child at this stage brings relevant consequences to the rest of his/her academic life ( ^{MELHUISH, 2013} ). This Education level is also relevant due to its representativeness in the Brazilian educational system: data from the 2013 School Census indicated that 29.7 million out of the 50.5 million students enrolled in basic Education in 2012, 68.2%, studied in the municipal educational systems (REGALO et al., 2016). Among the competencies acquired in the school, reading and writing skills are of paramount importance, as students who have better reading skills, perform better in other disciplines (OLIVEIRA et al ., 2008). Apart from the benefits in the educational field, proper literacy provides the individual with economic gains, as people with higher literacy earn more than those with lower literacy levels (BRAY et al., 2014).

In light of the above, the present work aims to identify the indicators and goals for the increase in school performance in the subject of Portuguese language in municipal lower secondary Education. By using such parameters, some evidences show the improvement of students’ performance in reading and writing in the final years of elementary school.

2 Methodological aspects

In order to improve school performance and identify its indicators and goals, the methodology used was divided into two parts. The first consisted of the separation of schools that were studied according to the level of efficiency in relation to knowledge aggregation as well as the knowledge of the students of lower secondary Education, which is also called school effect. Once the level of efficiency was defined, the next move dealt with identifying the factors related to school performance and its goals.

^{Barra and Zotti (2016)} indicate the use of non-parametric methods for the educational sector as more appropriate because, different from the parametric techniques, they do not require any knowledge of the theoretical boundary of production. Another advantage is that “non-parametric methods can handle multiple inputs and outputs in a simple manner, while most stochastic approaches require the choice of a single explanatory variable” ( Figure 1 ) ( ^{WITTE; LÓPEZ-TORRES, 2015} , p.15).

Source: Elaborated by the authors (2021)

Figure 1 DEA efficiency model 

Thus, the authors chose to use the Data Envelopment Analysis to measure efficiency. The technique allows a relatively efficient comparison between the groups of schools that were studied, which proved to be a necessary procedure for the development of this research (SALGADO JUNIOR; NOVI, 2015).

The aim of the DEA is to allow the comparison of Decision Making Units (DMU), which perform similar tasks obtaining different results in the relationship between the number of input used and the amount of output produced (MEZA et al., 2005). The efficiency frontier resulting from the application of the DEA technique ( Figure 2 ) indicates the efficient DMU that can be studied and compared to find the best practices. In addition, the technique also shows how distant other DMU are from those so-called efficient (FLACH et al ., 2017).

Source: Elaborated by the authors (2021)

Figure 2 DEA efficiency frontier model 

In light of this, it is essential to define the efficiency model in order to find the correct relationship with their respective explanatory factors. The proposed efficiency model aims to identify schools that have managed to aggregate more performance in the results of the subject of Portuguese-language in the Prova Brasil in relation to the same group of students.

For this, this research takes as input the average per school of the performance of students in Portuguese language in the Prova Brasil , taken at the end of the fifth year. The use of this input corrects one of the main problems of research on the subject, since most studies ignore the previous results obtained by students, resulting in the impossibility of estimating efficiency impartially ( ^{WITTE; LÓPEZ-TORRES, 2015} ). As output, the average performance was used in this evaluation at the end of the ninth year as well as the performance indicator of each school in this period, which is an index that presents the approval rate of students among the evaluations as shown in Chart 1 and Figure 3 .

Source: Elaborated by the authors (2021)

Figure 3 Evaluation cycle of Prova Brasil – efficiency model 

Thus, the proposed model seeks to evaluate the same cycle of students of each school, according to Figure 3 , to verify those who aggregated the highest performance, considering that, for the most part, students remained in the same institution throughout all the years of elementary school.

Two models considered classic are CCR and BCC. The CCR model (CHARNES et al ., 1978), “considers constant returns to scale, that is, any variation in input produces proportional variation in output” (MEZA et al., 2005, p. 2525). The BCC model considers variable returns to scale, being more appropriate in relation to CCR to produce more efficient DMU (BANKER et al. , 1984).

According to Flach et al. (2017), output guidance is the most appropriate because the objective of the public service is not to reduce input, but rather to have the best application to obtain the best results to the society. In this study, since the goal is to improve the student’s performance in the output test of the model ( Prova Brasil of the 8th grade), and not reduce his/her entry grade ( Prova Brasil of the 4th grade), the orientation to output will be used. Furthermore, in the hypotheses in which the establishment of proportionality between input and output is not possible, as in this study, it is recommended the adoption of the BCC model (SALGADO JUNIOR; NOVI, 2015). The model formulation can be represented as follows:

Subject to restrictions:

ur, vi ≥ 0

j = 1, ... , s

r = 1, ... , m

i = 1, ... , n

Where Y _ij and X _ij are the products and input of j-th DMU, u _r and v _i represent the weights (weighting coefficients or relative importance of each variable) to be determined by the solution of the problem. The u _r and v _i weights obtained are corresponding to the current DMU. This process is repeated for each of the DMU, obtaining different values for u _r and v _i .

However, since the DEA is a non-parametric technique, it is not susceptible to statistical inference, presenting great dependence on the quality of the sample and the specification of the efficiency model adopted ( ^{BARRA; ZOTTI, 2016} ; ^{MARQUES; SILVA, 2006} ). An alternative to this scenario is the implementation of the bootstrap methodology ( ^{EFRON, 1979} ) to DEA estimators. It allows statistical inference ( ^{MARQUES; SILVA, 2006} ) and the achievement of more reliable estimates of efficiency indexes and confidence intervals ( ^{DELGADO; MACHADO, 2007} , p. 435).

Because of this, for this study, we opted for the application of the BCC DEA with bootstrap, in which “estimates of corrected frontier by adjusted polarization are obtained, indicating the sensitivity of efficiency scores in relation to the sampling variations of the estimated frontier” ( ^{BARRA; ZOTTI, 2016} , p. 13). Finally, the PIM DEA software, version 3.2, was used to calculate the efficiency scores.

For the application of the DEA technique in the proposed model, data from the municipal elementary schools of Prova Brasil were extracted from the Inep website; Inep’s test is applied every 2 years. The results are only available for consulting at school level; therefore, the data of the average grades in the subject of Portuguese language of the students by school are used. From the 2011 base of Prova Brasil , the 5th year and 2015 base grades were used as well as the Performance Indicator of the Ideb of 2015. The use of data from the base years 2011 and 2015 was adopted to measure the evolution of the performance of the same cycle of students, since there is a high probability that those who took the test in the 9th year in 2015 are the same who performed it in 2011, during the 5th year. Data extraction was performed by developing a SQL program using Postgre SQL software.

The initial database had the performance data of 40,957 municipal elementary schools from all over the country. Initially a filter was performed, remaining only those schools that offered both the early years and the final years of elementary school, admitting that, in general, students tend to continue their studies at the same institution. Following the procedures, there was a new restriction in order to maintain only schools that had all the necessary data for the execution of the DEA. Then, schools that had obtained a performance indicator of 2015 lower than 0.50 were eliminated from the base, keeping 9,120 schools.

The DEA technique presents as a restriction the inability to distinguish the inefficiency attributed to the poor management practices from those resulting from socioeconomic differences or other factors that are not under the managers control ( ^{CARVALHO; SOUSA, 2014} ). Thus, the homogeneity of DMU is presented as pre-supposed for its use, that is, the units must perform equal tasks, with similar objectives, acting under the same market conditions and having autonomy for decision-making (MEZA et al., 2005). Ensuring their homogeneity is an essential item for the efficiency scale obtained to reflect only the differences between the practices adopted by schools and not the interference of external factors.

Thus, initially schools need to be separated according to socioeconomic level - SEL, to allow a fairer comparison among them ( ^{ANDRADE; SOARES, 2008} ), since this factor has been identified in educational research as one of the main influencers of student performance ( ^{ALVES; SOARES, 2007} ; ^{HAELERMANS; RUGGIERO, 2013} ).

To ensure homogeneity among DMU, schools were classified into seven bands, according to the SEL proposed by Alves, Soares and Xavier (2014), with the sample having 3,330 schools of SEL 5 and 6, which were used to generate the efficiency ranking using the DEA technique.

The following were the data of the contextual questionnaires of Prova Brasil (four context questionnaires: about the school, the teacher, the principal and the students) and the data of the School Census for each school. Data on the school and the principal were extracted from the bases of the School Census and the Prova Brasil of 2013, to identify the structure available during the period in which the student was studying as well as the characteristics of the principal and the management actions implemented by him/her, whose effect can be observed over a longer period. For students and teachers, we opted for the use of the 2015 database, since such data more accurately portray the characteristics of these actors at the time of the last edition of Prova Brasil , understanding that these characteristics have a direct influence on the result of the evaluation; 1,392 variables were obtained in these databases.

Due to the large volume of variables, a two-step analysis was performed: initially a quintile analysis was performed to identify those with significant differences between the groups and then a second stage of DEA, which is the Multiple Linear Regression, was used.

For each of the variables, normality test was performed to guide the next stages of the analysis. Then, the averages and/or medians of these variables were obtained, and, through the Mann-Whitney test, those with significant differences between the lower and upper quintiles were identified.

Subsequently, those that appeared in duplicity between the bases and those inversely significant dichotomous were removed since they offered the same response, opting for the maintenance of only one of the variables. Some variables were also maintained, which, despite not having a difference between high and low-performance schools, were understood as relevant and were also the object of other studies identified during the survey of the theoretical framework. These were classified as “neutral” and are the ones whose average or median showed no statistically significant difference between the upper and lower quintiles. After this last analysis, 162 variables remained.

From these variables, the second stage of DEA was performed, through Multiple Linear Regression. This technique has been widely applied to research in Education ( ^{ALVES; SOARES, 2013} ; AMÂNCIO-VIEIRA et al., 2015; ^{ANDRADE; SOARES, 2008} ), and allows to identify the existence of a dependent relationship between the dependent variable, that is, the one that one wishes to explain – in the case of this study, the efficiency score of each school generated by the DEA technique – and the independent ones, that is, those that may be related to this performance; in this work, independent ones are related to those available in the bases of the School Census and the Prova Brasil (CORRAR et al., 2009, p. 132).

Multiple Linear Regression Equation:

Where:

Y = Dependent Variable (DEA BCC score);

β ₀ = Intercept (Value of Y when X = 0);

X ₁, X ₂, ... X _n = independent variables (variables identified in the quintile analysis);

β ₁, β ₂, ... β _n = regression parameters;

ε = residual or regression error.

The method used was sequential search, with stepwise estimation. According to Corrar et al. (2009, p. 160), “this method performs an initial estimate with a set of independent variables and adds or eliminates variables until they achieve the best measure within the criterion used”. For this, the IBM SPSS Statistics version 22 software was used. The Figure 4 summarizes the steps of the method.

Source: Elaborated by the authors (2021)

Figure 4 Research steps 

Based on this two-stage DEA analysis, it was possible to both measure the level of efficiency of schools (first step) and identify factors related to their performance. From the exploration of the method, it was possible to determine the goals of each identified indicator. The constant value of the upper quintile and its respective trend was set as a goal. Thus, it is possible for each school to evaluate its own individual performance in relation to each of the indicators related to reading and writing. The next section details the search results.

3 Results

The selection of the statistical techniques is linked to the type of distribution of the data. Initially we present the analysis of the BCC DEA scores to verify whether they presented normal distribution or not.

The visual analysis of Graph 1 seems to indicate that the DEA BCC score does not present normal distribution, which can be confirmed by the Kolmogorov-Smirnov statistical test, according to Table 1 , since its Sig was 0.000 (Sig < 0.05).

Source: Elaborated by the authors (2021)

Graph 1 Histogram DEA BCC - schools of SEL 5 and 6 - Brazil 

The following is the descriptive statistics for the quintiles with DEA score, performance in Portuguese language in Prova Brasil , in their editions of 2011 and 2015 and Performance Indicator of 2015 ( Table 2 ).

A quintile analysis was performed to identify those ones that presented a statistically significant average or median difference between the upper quintile (schools considered of high performance in the DEA ranking) and the lower quintile (schools considered of low performance in the DEA ranking).

From the 161 variables identified in the quintile analysis, the analysis was performed using Multiple Linear Regression. The stepwise model pointed to a result composed of 39 selected independent variables. This model, due to the complexity involved in the educational service and the influence of numerous external factors on the students’ results, obtained a R² of 0.248, presenting limited explanatory power, according to Table 3 . However, if the technical assumptions are met, we can use its results to reinforce the analysis on the items highlighted by it.

According to Corrar et al. (2009), the main assumptions for regression analysis are normality and homocedasticity of the residuals. The analysis of the histogram of the residuals ( Graph 2 ) seems to indicate normality, which is confirmed by the Kolmogorov-Smirnov test ( Table 4 ).

Source: Elaborated by the authors (2021)

Graph 2 Histogram of regression residuals 

The verification of linear relationships between the variables can be performed through statistical tests such as the Variance Inflation Factor - VIF and Tolerance (CORRAR et al., 2009). For these indexes, it can be observed in Table 5 that there is no multicollinearity among the independent variables that remained in the model, since both, VIF and tolerance, are within the acceptable ranges.

Thus, the model complies with the statistical assumptions of the analysis, assisting in the identification of variables with greater explanatory potential among those initially selected, which are identified in Table 5 .

Once regression was performed, it was possible to identify, at a level of 5% confidence, which variables were significant to explain the students’ school performance in the Ideb evaluations. The variables were classified according to their effect in schools – negative, positive or neutral. Variables with negative effect were considered those that, presenting statistically significant difference, were more present in low-performance schools; and positive, those who, also presenting statistically significant difference, had higher frequency in schools considered efficient by the model. For each variable, the constant value of the upper quintile and its respective trend was set as a goal.

In addition to the identification of indicators, it is also essential to define the goals for each of these, allowing each school to evaluate its individual performance compared to the others in each aspect addressed in this study.

School A is located in a municipality with approximately 15,000 inhabitants with GDP per capita of R$ 16,058.64 ( ^{IBGE, 2016} ). It is located in a country town of São Paulo state, close to important centers such as Piracicaba and Limeira cities. The municipality also does not have private elementary schools.

The municipality referring to school B has approximately 137,000 inhabitants with GDP per capita of R$ 35,223.10 ( ^{IBGE, 2016} ), and is located in a country town of São Paulo state, exactly 60 kilometers from Campinas and Limeira cities. The municipality has a strong performance in citrus, besides housing large industrial enterprises in the metal mechanics sector and the paper industry. The municipality has a large network of private primary schools.

The school C is located in a municipality with approximately 29,000 inhabitants with GDP per capita of R$ 31,168.75 ( ^{IBGE, 2016} ), and is located in a country town of São Paulo state, with 16 kilometers from the city of Piracicaba. The city has municipal, state and private schools to offer basic education.

Finally, the municipality of school D has about 51,000 inhabitants with GDP per capita of R$ 30,453.74 ( ^{IBGE, 2016} ), and is located in a country town of São Paulo state, 130 km distant from Ribeirão Preto city and 160 km from Campinas city. The government is the largest employer of the municipality, being also responsible for most of the local economy, followed by industry and agriculture, respectively. The municipality has a network of private primary schools.

Thus, from the results found in linear regression, the results of high-performance schools were evaluated, using the quintile analysis; and the constant value of the upper quintile and its respective trend were determined as the goal of each variable. The Chart 2 presents the compiled results, as well as the values observed in each of the variables in the selected schools.

4 Conclusion

In this research, indicators and pedagogical and management goals used in high-performance municipal elementary schools in reading and writing evaluations were identified. Aware of the importance of socioeconomic factors in the performance of students, the authors chose to analyze the practices performed in Brazilian municipal elementary schools of similar socioeconomic level. Indicators related to the various actors of the educational process were found, namely: students, teachers and principals. Of these actors, some were related to the Principal’s management skills, reinforcing his/her role in promoting the quality of teaching.

This article approaches the multiple dimensions of quality pointed out in the literature through social, economic, organizational and institutional performance variables.

The results contribute to show that factors are related to quality of Education, represented here by the concept of school performance; however, other dimensions such as social inequality, teacher Education, management capacity of the principal, pedagogical activities and intra and extra school learning strategies are understood as explanatory factors.

For policymakers, this paper also contributes by presenting a benchmarking of efficient schools. This measure points to a direction for the planning of school principals and education departments.

Results corroborate the importance of the school and the adoption of good pedagogical and management practices in the educational process. In addition, resorting to this study, it is possible for each school to evaluate its own individual performance in relation to each of the indicators related to school performance in reading and writing.

This research is limited to the use of quantitative data for analysis and completion. In addition, the database used to identify explanatory factors comes from declaratory and opinion research, which could have subjectivity in perception. Still, we chose it use it because it is the only database available. Future research should include qualitative data analysis and on-site research on how these practices are performed, thus providing subsidies to the various educational actors for the performance of their functions.