Servicios Personalizados
Revista
Articulo
texto en 
Inglés (pdf)
Articulo en XML
Traducción automática
Enviar articulo por emailCompartir
Permalink
Educação e Pesquisa
versión impresa ISSN 1517-9702versión On-line ISSN 1678-4634
Educ. Pesqui. vol.45 São Paulo 2019 Epub 18-Sep-2019
https://doi.org/10.1590/s1678-4634201945199962
SECTION: ARTICLES
Resolute factors for the performance of Brazilian Federal Institutes in the National Examination of High School Education *
1
http://orcid.org/0000-0002-1352-1310
2
http://orcid.org/0000-0001-6752-5870
1- Instituto Federal de Educação, Ciência e Tecnologia do Ceará, Tauá, Ceará, Brasil. Contact: rogerio.sdutra@yahoo.com.br.
2- Escola Estadual de Educação Profissional Professor Antônio Valmir da Silva, Caucaia, Ceará, Brasil. Contact: gibmdutra@gmail.com.
3- Instituto Federal de Educação, Ciência e Tecnologia do Ceará, Acaraú, Ceará, Brasil. Contact: paulo.parente@ifce.edu.br.
4- Universidade Estadual do Ceará, Fortaleza, Ceará, Brasil. Contact: lya.loss@gmail.com.
This study aims to investigate to what extent the context of the Brazilian Federal Institutes determines their performance in ENEM (High School National Examination). For the development of this work, we chose to use ENEM as a source of educational evaluation, since it features summing and proactive functions, with prompt frequency, and guidance to decision-making, it is a large scale formal assessment. Data collection and survey of the learning performance and context of the Brazilian Federal Institutes disclosed by INEP (National Institute for Educational Studies and Research) included a sample of 865 reports concerning the years 2011, 2012, 2013, 2014, and 2015. The methodological outline comprised the application of univariate methods of measures of central tendency and dispersion, as well as of sample mean comparison tests; Student’s T-Test to compare two samples; and Anova for more than two samples. The main findings of this present research show that its hypothesis has been confirmed to the extent that regional contexts, geographical location, and the socio-economic profile of students have strongly affected student performance in the Brazilian Federal Institutes. As for the context of school facility size, even not presenting significant general performance difference in test results, the fields of Humanities and Writing showed evidence of higher performance in smaller facilities; and for the fields of Mathematics and Technologies, the tests showed significant differences, with better performance in larger facilities. This research has also found a greater number of upper-middle class students enrolled in Federal Institutes that participate in ENEM, strongly perceived in the Brazilian Northeast and Southeast urban units.
Key words: Educational evaluation; School context; Educational performance; Brazilian Federal Institutes
O objetivo do estudo é investigar em que medida os contextos determinam o desempenho dos Institutos Federais do Brasil no ENEM. Para a consecução do trabalho, optou-se por utilizar o ENEM como fonte de avaliação educacional, por apresentar características como funções somativa e proativa, de periodicidade pontual, com orientação a tomada de decisões, do tipo formal e de larga escala. O levantamento e a coleta de dados relativos aos contextos e aos desempenhos educacionais dos Institutos Federais do Brasil divulgados pelo INEP constituíram o conjunto amostral de 865 observações relativas aos anos de 2011, 2012, 2013, 2014 e 2015. O delineamento metodológico dispôs da aplicação de métodos univariados de medidas de tendência central e de dispersão, assim como de testes de comparação de médias amostrais; Teste t de Student para comparação de duas amostras; e Anova para comparação acima de duas amostras. Como principais achados da pesquisa, tem-se que a hipótese de que os contextos determinam diferenças significantes entre os desempenhos dos Institutos Federais do Brasil foi confirmada para os contextos regionais, para o espaço geográfico e para o perfil socioeconômico discente. Quanto ao contexto relativo ao porte escolar, mesmo não apresentando diferenças significativas de desempenho evidenciadas pelos testes, as áreas do conhecimento em Ciências Humanas e Redação apresentaram indícios de superioridade nos Institutos Federias de menor porte; já em Matemática e suas Tecnologias, o teste apresentou diferenças significativas, com desempenho superior nos Institutos de maior porte escolar. Outro achado da pesquisa foi a identificação do perfil dos Institutos Federais participantes do ENEM, o que foi fortemente percebido na zona urbana das regiões Nordeste e Sudeste, que apresentaram elevado quantitativo de alunos de classe média matriculados.
Palavras-Chave: Avaliação educacional; Contexto escolar; Desempenho educacional; Institutos Federais do Brasil
Introduction
The primary aspect of the educational evaluation process is: “conocer el papero que tiepeña, la función que cumple, saber quién se benefits from ella y, en definitiva, al servicio de quién se pone” ( RUIZ, 1996 , p. 17). According to Andriola (1999) determining the elements that improve the quality of education is the most relevant objective of educational evaluation.
Endogenous and exogenous factors capable of clarify the quality of education can be found in school environment ( GADOTTI, 2010 ). These factors highlight the context of the educational institutions and determine the setting of goals for a successful educational output and, consequently, educational performance.
Educational research has shown that diverging student performance is associated with differences in the contexts in which student groups are inserted ( BRYK; LEE, 1989 ; FRANCO et al., 2007 ). To check for such differences, this study has selected the Brazilian Federal Institutes in their contexts, by means of information provided by the National Institute for Educational Studies and Research Anísio Teixeira (INEP) with respect to (i) regional location; (ii) geographical area; (iii) socioeconomic profile of students; and (iv) school facility size.
The existing diversity among school locations is a parameter that can show differences in educational opportunity that can affect educational performance ( BARROS et al., 2001 ). Similarly, school results can be determined by the socio-economic and cultural origin of students enrolled in schools ( COLEMAN et al., 1966 ; BOURDIEU; PASSERON, 2008 ). Also, the size of the facilities is a context parameter that entails students of different levels and courses, that need larger areas, with furniture and equipment adapted to students of different age groups and specific needs, as well as more members in the staff and other supporting jobs ( ALVES; SOARES, 2013 ). This context parameter can highlight the educational performance of whether they can or not meet school needs.
The development of knowledge, skills and competences should be stimulated for the construction of critical and analytical thinking ( LIMBERGER, 2013 ), enabling learning to be constructive, contextual, collaborative, and self-directed ( JONG et al., 2014 ). The educational performance (proficiency) evaluations carried out inside and outside the school facilities are used to ascertain the results of the teaching-learning process ( HANUSHEK, 1979 ; PRIMI; HUTZ; SILVA, 2011), but not necessarily student learning level with accuracy.
External large-scale evaluations are used to assess the factors (resolute contexts) related to higher (or lower) performance, aimed at finding information that can help to promote actions that will improve educational results ( CAMARGO et al., 2016 ). Thus, the focus of this research is to assess student proficiency using their results by field of study and their mean score at the National Examination of High School Education (ENEM) as parameters of the educational performance of the institutions.
ENEM was established in 1998 and is considered a large-scale evaluation tool devised by the Ministry of Education and Culture (MEC) with the aim of assessing high school student learning and skills in the following areas of knowledge: Languages and Codes; Mathematics and Technologies; Humanities; Natural Sciences; and Writing ( COTTA, 2001 ).
Thus, considering that the context of institutions is resolute for educational performance, the following research question comes up: in what measures do the contexts determine the performance of the Brazilian Federal Institutes in the National Examination of High School Education (ENEM)? To answer this question, based in the objective of this study, we intend to investigate to what extent school context might determine the performance of the Brazilian Federal Institutes in ENEM.
Taking into account that Brazil is a very diverse country, due to its great extension, we have expected to find significant differences in educational performance among the Brazilian Federal Institutes, according to each reality determined by their context. Therefore, our research has proposed the hypothesis that school contexts determine significant differences in the student performance of Brazilian Federal Institutes in ENEM.
This present study on the context of the Brazilian Federal Institutes is considered relevant because it: (i) presents statistical test results that are able to provide important supporting information on the learning proficiency levels of students in ENEM, which can be used in the decision-making of public policies referred to education; (ii) uses indicators from the surveys of INEP to represent school environments; (iii) uses student results from ENEM, which is a learning evaluation featuring summing and proactive functions, with prompt frequency, and guidance to decision-making, formal typology and in large scale; and (iv) monitors the evolution of student performance in education over the research period of time.
This work has been organized into four sections, including this introduction. Section 2 brings the theoretical reference, some possible contexts of school institutions that are resolute for educational performance, introduces the National Examination of High School Education (ENEM), and previous studies. Section 3 describes the methodology used. Section 4 shows the results found; and Section 5 presents the main conclusion of our research.
Theoretical reference
Context of school institutions
Teaching proposals should depend on school context. Knowing the regional, geographical, socio-economic contexts of students and the demand for educational services is paramount to formulate educational practices ( SOUZA et al., 2013 ).
The characteristics of the location where educational projects are to be employed is a strategic context parameter in which we should observe, among other factors, the local economy, the shortage of qualified employees in the labor market, life expectancy, schooling, the cultural levels of the community for the proper employment of courses, whether the area is metropolitan or not, urban or rural ( BARROS et al., 2001 ).
Another context parameter to be observed, according to Alves and Soares (2009) , is student’s socio-economic level, measured through the schooling and job position of their parents, along with family income and goods, their potential to the consumption of goods and services, and their different sources of income.
The size of the school facility, measured by the number of students enrolled, is a context parameter that determines the number of rooms, the number of employees, the offer of teaching course modalities and the number of groups. The different levels and teaching modalities – regular education, from nursery to High School, and special education for the same grades – need larger facilities, more furniture and equipment adapted to students of different age groups and with specific needs, as well as a larger staff and varied support assistants ( ALVES; SOARES, 2013 ).
Thus, the reality of each institution defines its context, which can be considered a resolute factor for the success of students and institution, which is often measured by educational performance by means of internal evaluation or large-scale evaluations, which highlight the quality of the education offered by such schools according to region, state, education network, administrative facilities, socio-economic profile, urban or rural locations, complexity of school management, etc.
Educational assessment
Educational assessment is a tool for verifying the scenario of the teaching environment, with the aim of guiding decision-making and improving educational processes ( CORRÊA, 2013 ). Therefore, evaluative functions vary according to conceptions, data collection methods, and interpretative techniques (WORTHEN; SANDERS; FITZPATRICK, 2004).
For Luckesi (2014) , the evaluation process has the function of diagnosing. Luckesi believes that it checks the initial level of learning and the path to be taken in order to reach a particular goal. For Nevo (1997) , evaluation has both the formative and the summing functions. The formative function, of a procedural nature, makes it possible to improve methods and techniques that help the development or formation of the project, curriculum, organization and student ( CORRÊA, 2013 ). On the other hand, the summing function emphasizes the resulting product of the process, measuring the achievement of the objectives and previously established goals to a program, to an institution, to a curriculum or to learning ( SCRIVEN, 2000 ).
One can also consider the concepts mentioned, in which the frequency of evaluation can be defined as procedural (daily basis), and be categorized as formative, or defined as prompt (conclusive), and is characterized as summative, making it possible in the formative function, to build a daily process of learning, and in the summative function, to check for the difficulties that have been pointed out ( LUCKESI, 2014 ).
For educational assessment, Stufflebean and Shinkfield (1987) suggest the proactive function, used for decision-making based on the context of students; and the retroactive function, which is carried out through teacher interventions, identifying differences between previous and current learning processes, identifying learning deficiencies. There is also, in the literature, models of educational evaluation that refer to: (i) objectives ( TYLER, 1986 ); (ii) decision-making ( CRONBACH, 1963 ; SCRIVEN, 2000 ; STUFFLEBEAM, 2000); (iii) negotiating ( STAKE, 1978 ; PARLETT; HAMILTON, 1976 ; GUBA; LINCOLN, 1988 ); and (iv) empowerment, ( SAUL, 1988 ; FETTERMAN, 2001 ).
In turn, there is the model of Tyler (1986) , which pursuits the achievement of the outlined objectives, and it is the result of educational processes measured by the level of achievement of goals outlined in the program. And the result of the evaluation is used as support for the decision-making process to the educational structure offered.
The model developed by Cronbach (1963) does not prioritize unique criteria for the diagnosis of a program, instead it characterizes them as multidimensional results oriented to the decision-making process. Similarly, Scriven (2000) considers that the results of the evaluations are used for various decision-making purposes. According to such orientation, Stufflebeam (2000) defines that evaluation is intended to improve processes and not to prove them, generating data that will assist the decision-making process.
The negotiation-oriented model developed by Stake (1978) provides information for groups with different interests and needs to be reported to data managers. Parlett and Hamilton (1976) understand that the diverse interactions among the actors of the teaching process help to guide on the decisions made to the development of education. Guba and Lincoln (1988) argue that evaluation is a negotiating process, within the scope of providing information required by their target audience.
Saul’s (1988) empowerment-oriented self-evaluation is acknowledged as a collaborative activity of assessing subjects, entities, or communities, in order to improve consolidated programs. For Fetterman (2001 , p. 3), this assessment model aims to “assist people on helping themselves and improving their programs, using self-evaluation and reflection”.
In general, Ristoff (2002 , p. 10) states that assessing is
[...] first of all a process of self-acknowledgment, of institutional learning, it is a continuous, permanent, global, democratic process, politically legitimized, technically consolidated, and able to point the strengths, weaknesses, and potentialities of the institution in time and space.
In short, assessment enables managers to streamline decision-making processes with a focus on improving educational programs.
On a different classification, Fernandes (2013) points out that educational assessment can be both formal and informal. For him, in both cases there is the generation of data on the learning of each student. Schmidt (1993) acknowledges that these types of evaluation are complementary in the acquisition of knowledge. Formal evaluation deals with the verification of content related to scientific logic, while informal evaluation focuses on the logic of daily experience reasoning ( HAMMERSLEY, 2003 ; POLKINGHOME, 2000).
Educational assessment can also be employed for the (I) learning; (ii) institution; and (iii) large-scale. The first, related to learning, refers to the evaluation of each student in the processes or situations, with the aim of acquiring new knowledge or skill ( BELLONI, 1999 ). This author stated that this level of evaluation is also intended for the analysis of the syllabus or course outline, of teaching modalities and professional qualification activities. As for the evaluation of institutions, it focuses on policies, projects, and educational institutions, and is considered a tool for improving the administrative and educational management of schools and teaching systems ( BELLONI, 1999 ). The author also confirms that factors involving the objectives or mission of the institution, the social, economic, political, and cultural contexts should be taken into account in the evaluation of processes and results. As for the large-scale evaluation, its elaboration aims at a wide scope, both of contents and of technical and methodological applicability, characterized by the standardization of data collection and data processing ( COTTA, 2001 ).
Table 1 shows a summary of the classification raised for the construction of this section, without, however, exhausting the various modalities and classifications present in the literature on the subject of educational assessment.
Table 1 – Summary of the classification and typology of educational assessment
| Classification by: | Typology | Author(s) addressed |
|---|---|---|
| Function | (i) diagnosis | Luckesi (2014) . |
| (ii) summing | Nevo (1997) ; Scriven (2000) . | |
| (iii) formative | Nevo (1997) ; Corrêa (2013) . | |
| (iv) proactive | Stufflebean and Shinkfield (1987). | |
| (v) retroactive | Stufflebean and Shinkfield (1987). | |
| Frequency | (i) prompt | Luckesi (2014) . |
| (ii) procedure | ||
| Aimed at | (i) the objectives | Tyler (1986) . |
| (ii) decision making | Cronbach (1963) ; Scriven (2000) ; Stufflebeam (2000). | |
| (iii) negotiation | Stake (1978) ; Parlett and Hamilton (1976) ; Guba and Lincoln (1988) . | |
| (iv) to empowerment | Saul (1988) ; Fetterman (2001) . | |
| Type | (I) formal | Fernandes (2013) ; Schmidt (1993) ; Hammersley (2003) ; Polkinghome (2000). |
| (ii) informal | ||
| Level | (i) learning | Belloni (1999) . |
| (ii) institutional | ||
| (iii) large-scale | Cotta (2001) . |
Source: Drawn up by the authors based on the literature.
National Examination of High School Education (ENEM)
Evaluation is acknowledged as a complex process. Evaluations carried out in or outside the school seek somehow to assess the mastery and aptitude of students (levels of learning) in formal education. Thus, one way to objectively and formally measure educational performance is by assessing the student competence in the large-scale external educational evaluations at a national level ( OLIVEIRA; ARAÚJO, 2005 ).
The large-scale evaluation can present formative and summing objectives, as shown in the previous section. In the formative aspect, this type of examination reports to students both the knowledge they acquired and their need to improve in further examinations. From the summing perspective, for instance, the result of the test determines the score required to access the courses available at Colleges or Universities (IES).
In addition, this model of evaluation can have a proactive decision-making function by relating the results found to the context parameters of students and institutions (STUFFLEBEAN; SHINKFIELD, 1987).
The Brazilian government proposes through extensive learning tests a way to evaluate and measure the educational performance (competence) of students and educational institutions. The National Examination of High School Education (ENEM) makes part of the Brazilian system of educational assessment, along with the System of Evaluation of Basic Education (SAEB), the Brazilian Student Performance Examination (ENADE) and the Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES). These examinations / systems can assess the levels of high school, elementary school, higher and postgraduate education, respectively, presenting parameters of the Brazilian educational system ( TRAVITZKI, 2013 ).
Founded in 1998, ENEM aims to evaluate the level of competence of the student enrolled in basic education, through cognitive axes that seek to identify acquired competences and skills – through interdisciplinary questions on everyday experiences in exams of different areas of study – as well as the ability to comprehend and express their thoughts through writing (ADEODATO; SANTOS FILHO; RODRIGUES, 2014).
Cotta (2001) states that ENEM, through standardized measurement tools, survey, and procedures categorized on a large scale, provides feedback on the development of knowledge, competences, and skills of student enrolled on basic education in pursuit of higher education or a first job.
ENEM uses an array of required skills and abilities, with cognitive axis that define the following fields of study: (i) Languages, Codes, and its Technologies, which assesses a range of 9 competences and 30 skills; (ii) Mathematics, and its Technologies, which assesses 7 competences and 30 skills; and (iii) Natural Sciences and its Technologies, which assesses 8 competencies and 30 skills; and (iv) Humanities and its Technologies, which assesses 6 competencies and 30 skills ( PRIMI et al., 2001 ).
Test results are measured by the evaluation system of Item Response Theory (IRT), which assesses the amount of right answers, assigning a score according to the different levels of difficulty ( QUARESMA, 2014 ).
In addition to the student’s learning performance, ENEM has been a school parameter that assesses the performance of High Schools in general ( ANDRIOLA, 2011 ). Thus, educational institutions need to synchronize their syllabus with the principles of interdisciplinary study and contextualization, aimed at the development of knowledge, competence, and skills to integrate relevant areas of study and social inclusion, to provide for student entry in university and job market ( MACENO et al., 2011 ).
In summary, educational performance, measured by student effectiveness in ENEM, has been the purpose of evaluating basic education egress, meanwhile it guides educational institutions in learning and adapting to the needs of providing the students with the ability to reason, have abilities, and competences needed for the planning of their academic and professional development.
Previous studies that have referred to educational context and performance
Table 2 shows the main results of previous research that have referred to educational context and performance, with the aim to display the literature on the topic, and the introduction of our research to show how it differs, particularly because it focuses on students enrolled in the Brazilian Federal Institutes, and it is based on student effectiveness on the National Examination of High School education (ENEM).
Table 2 – Previous research
| Authors / Year | Objective / Methodology | Result |
|---|---|---|
| Barros et al. (2001) | He has investigated resolute matters of the Brazilian educational performance. The analysis was focused on a group of individuals aged between 11 and 25 years old, specific from urban areas located in the Northeast and Southeast of Brazil, 1996. | Its results show that better student educational performance is associated with better family socio-economic condition. Another result found is that Southeast students have shown to be more effective than Northeast students. |
| Soares and Sátyro (2008) | They have investigated the effects of infrastructure on the school performance of students in elementary schools in the period from 1998 to 2005. | School supplies underlined by the size of the facility, proved to be relevant to the achievement of the best educational output. |
| Lee (2008) | He investigated the school context consisting of the characteristics of students enrolled in each school –, and compared students from Catholic, private and public high schools. | Results showed that students enrolled in large schools tend to have lower performance output compared to students of small or medium-sized schools. |
| Padilha et al. (2012) | He examined the relations between the quality of public education in middle school, based on the Brazilian education quality index (IDEB) 2009, and the characteristics of the cities. | The main result found pointed to performance differences, in which the Southeast and the South regions indicated better rates than the other regions. |
| Alves (2013) | He examined how public policies could be associated with educational performance in schools in the Brazilian capital cities based on income data reported by the System of Evaluation of Basic Education (SAEB) in 1999, 2001, and 2003. | Controlled by factors related to the socio-economic context of the students and the social composition of the education networks, a strong association was found between the average socio-economic level variable and increased student performance. |
| Alves and Soares (2013) | They analyzed the relationship between IDEB and the school context, considering, for this purpose, the profile of students and the characteristics of the institution. | Results show that schools that have lower socio-economic students have worse results. Another result found relates to the impact of more complex school environments, which indicates that the results of schools with larger facilities and that have more classes with different grades and modalities find greater difficulties in reaching a higher IDEB. |
| Adeodate, Santos Filho and Rodrigues (2014) | They researched the quality of private secondary education in Brazil, based on ENEM and the 2011 School Census. | Results indicate that the main factors that affect the good quality of the school performance are the economic and financial aspects and, indirectly, the location of the school. |
Source: Drawn up by the authors based on the literature.
In general, the results show that, in fact, educational performance, when objectively measured, can differ depending on the school contexts the students experience. Thus, the results found on this research can be confirmed, or not, by results found in other research works that refer to existing income differences, given the school context of students and schools. Unlike these studies, we have chosen a research object that has not yet been explored (Brazilian Federal Institutes of Education, Science and Technology), besides analyzing data from a quantitative methodological perspective, seeking to provide statistical reliability to the results.
Methodology
This is a descriptive research, considering its objective; as for its procedures, it is documentary. According to Sampieri, Collado and Lucio (2013, p. 102), “descriptive research seeks to specify important properties and characteristics of any phenomenon that is analyzed”. The documentary procedure, according to Martins and Theóphilo (2009 , p. 85), “resembles bibliographic research, but does not raise edited material – books, periodicals etc. – it looks for material that has not been edited yet.” Thus, this present work uses data reported by INEP to describe how school contexts may affect the educational performance of Brazilian Federal Institutes.
The approach to the problem has a quantitative nature, as it uses the data collected to test hypotheses by means of statistical tests in order to establish behavior patterns (SAMPIERI; COLLADO; LUCIO, 2013). Through the statistical Student’s T-Test and variance analysis (Anova), specified at the end of this section, we intended to check for differences in performance affected by the contexts selected for this study.
The outlined sample for this present research can be seen on Table 1 , and it was defined according to the availability of data required to apply the statistical tests, on INEP’s portal. The choice of investigating Brazilian Federal Institutes (IFs) was based on the relevance of performance evaluation (proficiency) of students in a large-scale education network initiated in 2008, as well as by the diverse target audience of IFs students, justifying the comparative analysis of performance from different contexts (see Table 3 ).
Table 3 – Research sample
| Year | 2011 | 2012 | 2013 | 2014 | 2015 | Total |
| Observations | 124 | 84 | 215 | 227 | 215 | 865 |
Source: Research data.
The following Table 4 shows the operationalization of the contexts selected for this research to categorize the Brazilian Federal Institutes. The contexts expressed in Table 2 are presented by INEP and are usually used as a comparative performance parameter: socioeconomic, region and complexity of the school (ADEODATO; SANTOS FILHO; RODRIGUES, 2014; ALVES; SOARES, 2013 ; BARROS et al., 2001 ).
Table 4 – Categorization of contexts
| Regional | Demographic space | School size | Student socio-economic background |
|---|---|---|---|
| CO: Midwest; NE: North-East; N: North; SE: Southea; S: South. | Rural; Urban. | A: from 1 to 30 students; B: from 31 to 60 students; C: from 61 to 90 students; D: over 90 students. | I: very high; II: high; III: medium high; IV: medium; V: medium low; VI: low. |
Source: Prepared by the authors based on data provided by INEP.
The educational performance collected from the INEP database is calculated according to Equation (1) below,
where DESEMP refers to the means assigned at each Brazilian Federal Institute ( i ) for each year ( j ) in: Languages and Codes (LC); Mathematics and Technologies (MT); Humanities (CH); Natural Sciences (CN); Writing (RED); and National Examination of High School Education (ENEM). Thus, the educational performance was calculated by the ratio between the sum of the grades of all ENEM’s formative itineraries and the total of students of each Brazilian Federal Institute. This procedure was carried out for the five periods analyzed (2011, 2012, 2013, 2014 and 2015).
In accordance with the purpose of our research, we have adopted statistical procedures, which are univariate with measures of central tendency and dispersion to describe the sample, as well as tests of mean comparison (Student T-Test and Analysis of Variance) to check for differences in performance due to the context of the Brazilian Federal Institutes.
While the Student T-Test verifies the existence of differences in independent sample means, the Analysis of Variance (Anova) tests the equality of three or more population means, based on the analysis of sample variances. Thus, Student T-Test is applied to compare whether the mean educational performance in ENEM differs according to geographic location. In turn, Anova is applied to compare whether the mean educational performance in ENEM differs according to school size, socio-economic, and regional context.
Data found were grouped in spreadsheets of Office Excel ® and treatment was carried out in Stata® ( Data Analysis and Statistical Software ), version 13.
Analysis and discussion of results
Preliminary, the Kolmogorov-Smirnov normality test was carried out to determine the normality of the data set (865 observations), in order to verify if the sample set analyzed represents the population to which it is related (Brazilian Federal Institutes). Results are presented on Table 5 below.
Table 5 – Kolmogorov-Smirnov Normality Tests
| LC | MT | CH | CN | RED | ENEM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Est. | Sig. | Est. | Sig. | Est. | Sig. | Est. | Sig. | Est. | Sig. | Est. | Sig. |
| 0.035 | 0.015 | 0.031* | 0.055 | 0.049 | 0.000 | 0.021* | 0.200 | 0.032 | 0.038 | 0,018* | 0.200 |
Notes: if a test is significant (Sig. > 0.05), the null hypothesis is accepted (H0:data with normal distribution); otherwise, the null hypothesis is rejected and the alternative hypothesis is accepted (H1: data with no normal distribution). LC – Languages and Codes; MT – Mathematics and Technologies; CH – Humanities; CN-Natural Sciences; RED – Writing; and ENEM – National Examination of High School Education. *Statistical Significance at 1% level.
Source: Research data.
It appears that the data presented normal distribution for the areas of Mathematics and Technologies, and Natural Sciences. Although the other fields of study did not show normal distribution, the normality test for the mean of ENEM indicated low amount or absence of extreme values to mean.
In addition to the normality test, Table 6 shows the changing of the means among the years.
Table 6 – Educational performance: years researched
| Year | Obs. | LC | MT | CH | CN | RED | ENEM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | ||
| 2011 | 124 | 559.1 | 33.1 | 603.5 | 65.7 | 527.4 | 35.2 | 522.0 | 38.8 | 592.4 | 46.7 | 553.0 | 42.0 |
| 2012 | 84 | 517.0 | 31.4 | 568.5 | 59.0 | 557.2 | 35.2 | 512.8 | 36.1 | 567.1 | 56.7 | 544.5 | 41.0 |
| 2013 | 215 | 529.0 | 33.0 | 575.7 | 57.0 | 564.7 | 35.1 | 512.0 | 35.0 | 583.3 | 44.7 | 553.0 | 38.8 |
| 2014 | 227 | 550.3 | 28.0 | 553.9 | 62.5 | 595.7 | 30.0 | 538.2 | 35.3 | 591.8 | 46.3 | 566.0 | 41.2 |
| 2015 | 215 | 542.0 | 28.6 | 533.2 | 58.0 | 599.0 | 24.7 | 520.1 | 32.3 | 607.4 | 50.4 | 560.3 | 35.0 |
| Test F | 37.38* | - 31.01* | 139.02* | 18.01* | 10.81* | 6.36* | |||||||
Notes: LC – Languages and Codes; MT – Mathematics and Technologies; CH – Humanities; CN-Natural Sciences; RED – Writing; and ENEM – National Examination of High School Education. *Statistical Significance at 1% level. SD represents the standard deviation, as it shows the distance of the grades from the general mean.
Source: Research data.
Increasing differences considered significant for the competence levels assessed by the examination are found mainly in the data by means of comparison tests. The field of Humanities is highlighted, and that fact points out that, over the years, students enrolled in Brazilian Federal Institutes have greatly expanded (high coefficient: Test F = 139.02) the ability to solve questions related to the content required by the test. On the other hand, the field of Mathematics and Technologies has decreasing means; however, it is possible to noticed that this variation decreases (low coefficient: Test F = 31.01) among the years researched, indicating reduced performance throughout the years. The variations between the years in the researched fields of knowledge were enough to cause ENEM grade point averages to show significant and increasing differences. Results can also be checked in Chart 1 .
The following analysis on Table 7 refers to the educational performance of the Brazilian Federal Institutes in relation to their regional location.
Table 7 – Educational performance: regional context
| Region | Obs. | LC | MT | CH | CN | RED | ENEM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | ||
| CO | 91 | 536.5 | 28.2 | 551.6 | 42.9 | 572.6 | 35.5 | 521.7 | 25.4 | 579.6 | 46.7 | 551.5 | 29.1 |
| NE | 336 | 532.2 | 30.7 | 537.6 | 55.6 | 566.7 | 38.1 | 507.8 | 30.0 | 587.5 | 56.7 | 544.7 | 36.0 |
| N | 112 | 520.3 | 31.7 | 521.8 | 56.1 | 555.7 | 41.2 | 498.0 | 32.8 | 564.4 | 44.7 | 530.8 | 35.2 |
| SE | 199 | 561.7 | 29.1 | 618.7 | 55.3 | 595.3 | 38.6 | 553.3 | 33.6 | 618.0 | 46.3 | 588.7 | 34.8 |
| S | 127 | 553.1 | 27.5 | 585.5 | 49.5 | 585.9 | 35.4 | 534.4 | 28.1 | 591.4 | 50.4 | 569.5 | 31.5 |
| Test F | 51.11* | 95.93* | 28.13* | 92.91* | 23.10* | 74.30* | |||||||
Notes: LC – Languages and Codes; MT – Mathematics and Technologies; CH – Humanities; CN-Natural Sciences; RED – Writing; and ENEM – National Examination of High School Education; CO – Midwest; NE – Northeast; N – North; SE – Southeast; and S – South. *Statistical Significance at 1% level. SD represents the standard deviation, as it shows the distance of the grades from the general mean.
Source: Research data.
The Northeast (336 observations) and Southeast (199 observations) were the regions with the largest number of participants in ENEM. Results found confirm this research hypothesis that the performances show significant differences considering the regional context, agreeing with the results found by Adeodato, Santos Filho, and Rodrigues (2014). Superior performances of the Federal Institutes were predominantly found for the Southeast and South regions, corroborating the findings of Padilha et al. (2012) , and that, in general, the means showed low dispersion in each region. Another important point refers to the greatest performances in the fields of Humanities (CH) and Writing (RED), which can be better viewed in Chart 2 .
Although the performances of the Brazilian Federal Institutes differ statistically in different regions of the country, this difference is irrelevant, when observed on Chart 2 . Such result may indicate the consequence of an educational policy aimed at guaranteeing education to all Brazilian regions.
In Table 8 , the performance is presented according to the location of the unit, represented by the geographical location context (rural and urban).
Table 8 – Educational performance: geographical location context
| Zone | Obs. | LC | MT | CH | CN | RED | ENEM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | ||
| Rural | 223 | 524.6 | 28.3 | 544.9 | 53.3 | 557.6 | 38.5 | 510.7 | 31.3 | 561.1 | 51.7 | 538.8 | 34.3 |
| Urban | 642 | 546.7 | 32.7 | 568.9 | 66.6 | 581.4 | 39.1 | 526.4 | 37.3 | 601.7 | 50.2 | 563.8 | 39.4 |
| Test t | -9.0* | -4.9* | -7.9* | -5.7* | -10.3* | -8.4* | |||||||
Notes: LC – Languages and Codes; MT – Mathematics and Technologies; CH – Humanities; CN-Natural Sciences; RED – Writing; and ENEM – National Examination of High School Education. *Statistical Significance at 1% level. SD represents the standard deviation, as it shows the distance of the grades from the general mean.
Source: Research data.
The observations (642) show that the participation of the Brazilian school network in ENEM includes a high concentration of students enrolled in the urban area. In this context, there is the low dispersion of the best-performing means for the Brazilian Federal Institutes located in the urban area. ENEM grade averages range in 25 points, the largest difference of which is on the Writing exam, with a difference of 40.6 points. It is also noted that the differences found are considered significant, as evidenced in Chart 3 , which confirms the hypothesis of our research for the geographic location context parameter and thus supporting the results found by Barros et al. (2001) .
Next, we present the educational performance under the context of school facility size for each Brazilian Federal Institute ( Table 9 ). In this analysis, there was only a difference in the formative itinerary of Mathematics and Technologies, according to the size of the school facility.
Table 9 – Educational performance: school facility size
| Facility size | Obs. | LC | MT | CH | CN | RED | ENEM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | ||
| A | 36 | 538.8 | 31.2 | 559.4 | 60.0 | 583.7 | 37.0 | 523.2 | 40.4 | 602.0 | 69.1 | 561.4 | 42.5 |
| B | 179 | 536.4 | 32.3 | 550.5 | 59.7 | 576.6 | 40.2 | 519.8 | 36.4 | 587.3 | 60.4 | 553.7 | 39.6 |
| C | 271 | 541.0 | 32.9 | 556.5 | 64.7 | 575.0 | 40.0 | 520.0 | 36.8 | 594.0 | 48.0 | 555.8 | 39.2 |
| D | 433 | 543.1 | 33.5 | 571.1 | 65.1 | 574.2 | 40.8 | 524.5 | 36.0 | 590.6 | 51.8 | 559.4 | 39.7 |
| Test F | 1.8 | 5.4* | 0.7 | 1.1 | 1.0 | 1.2 | |||||||
Notes: LC – Languages and Codes; MT – Mathematics and Technologies; CH – Humanities; CN – Natural Sciences; RED – Writing; ENEM – National Examination of High School Education; A – from 1 to 30 students; B – from 31 to 60 students; C – from 61 to 90 students; and D – over 90 students. * Statistical significance at 1%. SD represents the standard deviation, as it shows the distance of the grades from the general mean.
Source: Research data.
The number of findings (Facility size C and D), collected from the data, show that the Brazilian Federal Institutes with a great number of students provided stronger participation in ENEM. Unlike the findings of Soares and Sátyro (2008) , Lee (2008) , and Alves and Soares (2013) , student performances, when analyzed under the context of school facility size, predominantly did not show significant differences – which can be better viewed in Chart 4 –, therefore not confirming the hypothesis of our research. Although results do not converge with the literature, it can be noted that the performances in Humanities and Writing presented indications in favor of smaller facilities. For Mathematics and Technologies, data pointed out that largest facility size favored the learning process of the students, possibly due to the need for greater investments, the specificities, and the focus given in the courses offered by the units, approximating theory and practice.
Finally, educational performance is analyzed, in Table 10 , under the context of the student socioeconomic profile. In general, the analyses were significant, that is, there is a difference between the grades of the formative itineraries based on the socio-economic profile of the Brazilian Federal Institutes.
Table 10 – Educational performance: student socio-economic context
| Profile | Obs. | LC | MT | CH | CN | RED | ENEM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | ||
| I | 5 | 611.8 | 13.6 | 713.8 | 59.4 | 655.1 | 17.1 | 624.4 | 37.1 | 659.4 | 20.3 | 652.9 | 27.6 |
| II | 121 | 577.7 | 20.2 | 627.3 | 53.6 | 615.3 | 27.9 | 563.1 | 31.4 | 628.6 | 48.1 | 602.3 | 29.1 |
| III | 239 | 555.5 | 22.3 | 588.4 | 52.0 | 590.6 | 27.8 | 538.3 | 24.3 | 611.3 | 42.2 | 575.8 | 26.1 |
| IV | 303 | 531.4 | 26.5 | 546.5 | 50.4 | 565.2 | 35.7 | 512.5 | 26.5 | 582.4 | 50.2 | 546.2 | 30.1 |
| V | 184 | 515.4 | 27.4 | 514.9 | 49.1 | 547.6 | 35.0 | 492.0 | 25.8 | 557.6 | 48.5 | 524.0 | 29.2 |
| VI | 13 | 491.0 | 20.1 | 486.7 | 23.8 | 516.7 | 39.7 | 471.0 | 15.9 | 531.7 | 40.3 | 497.2 | 19.5 |
| Test F | 138.1* | 104.8* | 94.5* | 156.2* | 50.9* | 161.9* | |||||||
Notes: LC – Languages and Codes; MT – Mathematics and Technologies; CH – Humanities; CN-Natural Sciences; RED – Writing; and ENEM – National Examination of High School Education; I – Very high; II – High; III – Medium high; IV – Medium; V – Medium low; and VI – Low. *Statistical significance at 1%. SD represents the standard deviation, as it shows the distance of the grades from the general mean.
Source: Research data.
Results show on the many findings in the data that students enrolled in the Brazilian Federal Institutes participating in ENEM are predominantly middle class (socioeconomic profile – medium: 303; medium high: 239; and medium low: 184). It is inferred from the analysis that the performances were superior for the students categorized in the highest socio-economic profile (see also Chart 5 ), according to the works of Alves (2013) and Alves and Soares (2013) . This can be justified by cultural differences and the level of care of basic needs, provided by the family nucleus. Thus, our research hypothesis is confirmed by the results of such differences.
In general, the educational performances of the Brazilian Federal Institutes proved to be different and significant, determined by factors (contexts) such as regional and geographical location (urban and rural area) and the socioeconomic profile of the students, confirming the hypothesis of our research for these particular cases. As for the context of school facility size, we were not able to determine differences in educational performance. The research also provided a profile of the Brazilian Federal Institutes participating in ENEM with regard to the following contexts: theis participation in ENEM was strongly perceived in the institutions of urban areas in the Northeast and Southeast regions, as well as those with high numbers of middle class students enrolled.
Final remarks
This study aims to investigate to what extent the context of the Brazilian Federal Institutes determines their performance in ENEM. For this purpose, the ENEM exam was chosen as a parameter of educational evaluation, with its summing and proactive characteristics, prompt frequency, decision-making orientation, formal and large scale, aiming to provide managers and the community with data support on the results of educational public policies. It should be noted that the performances mentioned in our work refer to student competence measured in ENEM.
The path of statistical inferences to answer the question raised and test the research hypothesis began with the verification of the distribution of sample normality and with the application of the tests of comparison for the means. The different results did not affect the normality test of the average grade point of ENEM.
As for the regional context, the Brazilian Federal Institutes had better performance rates predominantly in the Southeast and South regions. This can be associated with differences in their structural scenario and student access to school in these regions. It was also clear that the assimilation of knowledge and development of skills and competencies were further enhanced in the field of Humanities, as well as the ability to express thought and comprehension in the Writing. In addition, the context of geographical location shows that these better performances were achieved by Federal Institutes located in the urban areas.
As for the assessment on Brazilian Federal Institute facility size, it did not show relevant rates of performance differentiation. However, student performance on the fields of Humanities and Writing tended to be higher for smaller institutes. On the other hand, the field of Mathematics and Technologies showed better performance for institutes of larger facilities. This fact can be associated with larger investments that favor the cognitive axis of the courses offered, providing, in many cases, approximation between theory and practice due to the specificity of the courses.
Finally, significant and superior differences were found in the performances of Federal Institutes with educators who presented better conditions of meeting basic needs, according to the classification of the socio-economic profile identified. Such results confirm that the social and cultural background of the family is a resolute parameter for the necessary means to improve educational performance.
We point out that, notably, our results show that educational performance is lower for students from rural areas, and low socio-economic class. Thus, it is paramount that new policies are created to tailor educational practices so that performance can be enhanced ( SOUZA et al., 2013 ). Besides, welfare policies can also be taken into account, so that these clear performance differences can be assessed by managers and public politicians.
This research has also found a greater number of upper-middle class students enrolled in Federal Institutes that participate in ENEM, strongly perceived in the Brazilian Northeast and Southeast urban units.
In general, this present research contributes to the topic of educational performance and school context, and provides management information to public managers and society on the output of the existing public policies. In addition, it highlights key context parameters of educational performance, measured by the achievements of students in ENEM, which may indicate an urgent need for action on the part of the Brazilian government.
Further study should be carried out to extend this research, namely on: (i) comparative studies between federal, state, municipal and private institutions; (ii) the use of other performance metrics, such as SAEB and IDEB; and (iii) the use of contexts related to financial factors and services offered.
Referências
ADEODATO, Paulo Jorge Leitão; SANTOS FILHO, Maílson Melo dos; RODRIGUES, Rodrigo Lins. Predição de desempenho de escolas privadas usando o ENEM como indicador de qualidade escolar. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 25., 2014, Dourados. Anais... Porto Alegre: SBC, 2014. p. 891-895. [ Links ]
ALVES, Fátima. Políticas educacionais e desempenho escolar nas capitais brasileiras. Cadernos de Pesquisa , São Paulo, v. 38, n. 134, p. 413-440, 2013. [ Links ]
ALVES, Maria Teresa Gonzaga; SOARES, José Francisco. Contexto escolar e indicadores educacionais: condições desiguais para a efetivação de uma política de avaliação educacional. Educação e Pesquisa , São Paulo, v. 39, n. 1, p. 177-194, jan./mar. 2013. [ Links ]
ALVES, Maria Teresa Gonzaga; SOARES, José Francisco. Medidas de nível socioeconômico em pesquisas sociais: uma aplicação aos dados de uma pesquisa educacional. Opinião Pública , Campinas, v. 15, n. 1, p. 1-30, 2009. [ Links ]
ANDRIOLA, Wagner Bandeira. Doze motivos favoráveis à adoção do Exame Nacional do Ensino Médio (ENEM) pelas Instituições Federais de Ensino Superior (IFES). Ensaio , Rio de Janeiro, v. 19, n. 70, p. 107-25, 2011. [ Links ]
ANDRIOLA, Wagner Bandeira. Evaluación: la vía para la calidad educativa. Ensaio , Rio de Janeiro, v. 7, n. 25, p. 355-368, 1999. [ Links ]
BARROS, Ricardo Paes de et al. Determinantes do desempenho educacional no Brasil. Pesquisa e Planejamento Econômico , Rio de Janeiro, v. 31, n. 1, p. 1-42, 2001. [ Links ]
BELLONI, Isaura. Avaliação institucional: um instrumento de democratização da educação. Linhas Críticas , Brasília, DF, v. 5, n. 9, p. 31-58, 1999. [ Links ]
BOURDIEU, Pierre; PASSERON, Jean-Claude. A reprodução: elementos para uma teoria do sistema de ensino. Petrópolis: Vozes, 2008. [ Links ]
BRYK, Antony; LEE, Valerie. A multilevel model of social distribution of high school. In: DARLING-HAMMOND, Linda (Ed.). Review of research in education . Washington, DC: AERA, 1989. p. 171-267. [ Links ]
CAMARGO, Raphael Vinicius Weigert et al. Desempenho dos alunos de ciências contábeis na prova ENADE/2012: uma aplicação da Teoria da Resposta ao Item. Revista de Educação e Pesquisa em Contabilidade , Brasília, DF, v. 10, n. 3, p. 332-355, jul./set. 2016. [ Links ]
COLEMAN, James et al. Equality of educational opportunity. Washington, DC: U.S. Department of Health, Education, and Welfare, 1966. [ Links ]
CORRÊA, Denise Maria Moreira Chagas. Indicadores de gestão do TCU e o desempenho dos cursos no Enade: um estudo de cursos de graduação. Tese (Doutorado) – Faculdade de Educação, Universidade Federal do Ceará, Fortaleza, 2013. [ Links ]
COTTA, Tereza Cristina. Avaliação educacional e políticas públicas: a experiência do Sistema Nacional de Avaliação da Educação Básica (SAEB). Revista do Serviço Público , Brasília, DF, v. 52, n. 4, p. 89-111, 2001. [ Links ]
CRONBACH, Lee Joseph. Course Improvement through evaluation. Teachers College Record , New York, v. 64, n. 8, p. 672-683, 1963. [ Links ]
FERNANDES, Domingos. A avaliação em educação: uma discussão de algumas questões críticas e desafios a enfrentar nos próximos anos. Ensaio , Rio de Janeiro, v. 21, n. 78, p. 11-34, 2013. [ Links ]
FETTERMAN, David M. Foundations of empowerment evaluation. CA/USA: Sage, 2001. [ Links ]
FRANCO, Creso et al. Qualidade e equidade em educação: reconsiderando o significado de “fatores intra-escolares”. Ensaio , Rio de Janeiro, v. 15, n. 55, p. 277-298, 2007. [ Links ]
GADOTTI, Moacir. Qualidade na educação: uma nova abordagem. Produção de terceiros sobre Paulo Freire. São Paulo: Série Livros, 2010. [ Links ]
GUBA, Egon; LINCOLN, Yvonna. Effective evaluation. San Francisco: Jossey-Bass, 1988. [ Links ]
HAMMERSLEY, Martyn. Social research today: some dilemmas and distinctions. Qualitative Social Work , v. 2, n. 1, p. 25-44, 2003. [ Links ]
HANUSHEK, Eric A. Conceptual and empirical issues in the estimation of educational production functions. Journal of Human Resources , Madison, v. 14, n. 3, p. 351-388, 1979. [ Links ]
JONG, Nynke et al. Blended learning in health education: three case studies. Perspectives Medical Education , Maastrich, v. 3, n. 4, p. 278-288, sep. 2014. [ Links ]
LEE, Valerie. Utilização e modelos hierárquicos lineares para estudar contextos sociais. In: BROOKE, Nigel; SOARES, José Francisco (Ed.). Pesquisa em eficácia escolar: origem e trajetórias. Belo Horizonte: UFMG, 2008. p. 273-298. [ Links ]
LIMBERGER, Jane Beatriz. Metodologias ativas de ensino-aprendizagem para a educação farmacêutica: um relato de experiência. Interface , Botucatu, v. 17, n. 47, p. 969-975, 2013. [ Links ]
LUCKESI, Cipriano Carlos. Avaliação da aprendizagem escolar: estudos e proposições. São Paulo: Cortez, 2014. [ Links ]
MACENO, Nicole Glock et al. A matriz de referência do ENEM 2009 e o desafio de recriar o currículo de química na educação básica. Química Nova na Escola , São Paulo, v. 33, n. 3, p. 153-159, 2011. [ Links ]
MARTINS, Gilberto de Andrade; THEÓPHILO, Carlos Renato. Metodologia da investigação científica para ciências sociais aplicadas. São Paulo: Atlas, 2009. [ Links ]
NEVO, Daniel. Evaluación basada en el centro: un diálogo para la mejora educativa. Bilbao: Mensajero, 1997. [ Links ]
OLIVEIRA, Romualdo Portela; ARAÚJO, Gilda Cardoso de. Qualidade do ensino: uma nova dimensão da luta pelo direito à educação. Revista Brasileira de Educação , Rio de Janeiro, n. 28, p. 5-23, 2005. [ Links ]
PADILHA, Frederica et al. As regularidades e exceções no desempenho do Ideb dos municípios. Estudos em Avaliação Educacional , São Paulo, v. 23, n. 51, p. 58-81, 2012. [ Links ]
PARLETT, Malcolm; HAMILTON, David. Evaluation as ilumination: a new aproach to the study of innovatory programs. In: GLASS, Gene V. (Ed.). Evaluation studies review annual . Beverly Hills: Sage, 1976. p. 141-157. [ Links ]
POLKINGHORNE, Donald. Psychological inquiry and the pragmatic and the hermeneutical traditions. Theory and Psychology , Ontario, v. 10, n. 4, p. 453-479, 2000. [ Links ]
PRIMI, Ricardo; HUTZ, Claudio S.; SILVA, Marjorie C. Rocha da. A prova do ENADE de psicologia 2006: concepção, construção e análise psicométrica da prova. Avaliação Psicológica , Campinas, v. 10, n. 3, p. 271-294, 2011. [ Links ]
PRIMI, Ricardo et al. Competências e habilidades cognitivas: diferentes definições dos mesmos construtos. Psicologia , Brasília, DF, v. 17, n. 2, p. 151-159, 2001. [ Links ]
QUARESMA, Edilan de Sant’Ana. Modelagem para construção de escalas avaliativas e classificatórias em exames seletivos utilizando teoria da resposta ao item uni e multidimensional. Tese (Doutorado) – Escola Superior de Agricultura “Luiz de Queiroz”, Universidade de São Paulo, Piracicaba, 2014. [ Links ]
RISTOFF, Dilvo Ilvo. Apresentação In: DIAS SOBRINHO, José. Universidade e avaliação: entre a ética e o mercado. Florianópolis: Insular, 2002. p. 5-8. [ Links ]
RUIZ, José Maria. Teoría del curriculum: diseño y desarrollo curricular. Madrid: Universiatas, 1996. [ Links ]
SAMPIERI, Roberto Hernandez; COLLADO, Carlos Fernandez; LÚCIO, Maria del Pilar Baptista. Metodologia de pesquisa. 3. ed. São Paulo: McGraw Hill, 2013. [ Links ]
SAUL, Ana Maria. Avaliação emancipatória: desafio à teoria e à prática de avaliação e reformulação de currículo. São Paulo: Cortez; Autores Associados, 1988. [ Links ]
SCHMIDT, Mary R. Grout. Alternative kinds of knowledge and why they are ignored. Public Administration Review, New Jersey, v. 53, n. 6, p. 525-530, 1993. [ Links ]
SCRIVEN, Michael. The logic and methodology of checklists . 2000. Disponível em: < https://wmich.edu/sites/default/files/attachments/u350/2014/logic%26methodology_dec07.pdf> . Acesso em: 14 abr. 2018. [ Links ]
SOARES, Sergei; SÁTYRO, Natália. O impacto de infra-estrutura escolar na taxa de distorção idade-série das escolas brasileiras de ensino fundamental: 1998 a 2005. Brasília, DF: IPEA, 2008. Texto para discussão, Instituto de Pesquisa Econômica Aplicada (IPEA). [ Links ]
SOUZA, Norma Valéria Dantas de Oliveira et al. Perfil socioeconômico e cultural do estudante ingressante no curso de graduação em enfermagem. Revista de Enfermagem UERJ , Rio de Janeiro, v. 21, n. 2, p. 718-722, 2013. [ Links ]
STAKE, Robert E. Case study methodology in social inquire. Educational Researcher , Washington, DC, v. 7, p. 5-8, 1978. [ Links ]
STUFFLEBEAM, Daniel L. The CIPP model for evaluation. In: STUFFLEBEAM, Daniel L.; MADAUS, George F.; KELLAGHAN, Thomas. Evaluation models: viewpoints on educational and human services evaluation. 2. ed. Massachusetts: Kluwer Academic Publishers, 2000. p. 279-317. [ Links ]
STUFFLEBEAM, Daniel L.; SHINKFIELD, Anthony J. Evaluación sistémica: guía teórica y práctica. Barcelona: Paidós, 1987. [ Links ]
TRAVITZKI, Rodrigo. ENEM: limites e possibilidades do Exame Nacional do Ensino Médio enquanto indicador de qualidade escolar. 2013. Tese (Doutorado) – Faculdade de Educação, Universidade de São Paulo, São Paulo, 2013. [ Links ]
TYLER, Ralph Winfred. Princípios básicos de currículo e ensino. Tradução de Leonel Vallandro. 9. ed. Rio de Janeiro: Globo, 1986. [ Links ]
WORTHEN, Blaine R.; SANDERS, James R.; FITZPATRICK, Jodi L. Avaliação de programas: concepções práticas. Tradução Dinah de Abreu Azevêdo. São Paulo: Gente, 2004. [ Links ]
Received: May 18, 2018; Revised: September 26, 2018; Accepted: October 26, 2018
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
