INTRODUCTION

The evaluation of learning has been confused with the evaluation of the education system. Assessing learning means using a variety of tools to diagnose how much learners have learned (or have not learned) and thus make corrections to the direction of the teaching-learning process, if necessary. In turn, evaluating the education system is much more than analyzing students’ grades at a certain level of education. Evaluation must be systemic and take into account factors such as school infrastructure (teaching support equipment, laboratories, libraries and their collections, etc.), the opportunities for continuing education offered to teachers, and the socio-economic and cultural characteristics of students.

According to Santos, “it is increasingly essential to know how to use and analyze data from external evaluation in order to obtain credible information that allows us to improve the curriculum, assessment and didactics, taking into account all the specificities and diversities that characterize our educational system”(^{SANTOS, 2014}, p.151). The intervention of the Ministries of Education of the countries is necessary to ensure the implementation of educational policies as a whole. Therefore, understanding the concepts of evaluation from the perspective of the school and the educational system present in official documents is fundamental.

Those who think that education and the economy are completely separate, detached, are being eluded. The importance of education for the economy is so evident that its relationship is explicit in different official documents from different countries and bodies.

According to ^{Marcon (2015}), various multilateral organizations such as the World Bank (BW), the International Monetary Fund (IMF), the United Nations Educational, Scientific and Cultural Organization (UNESCO), the Organization for Economic Cooperation and Development (OECD), the Inter-American Development Bank (IDB) and the International Bank for Reconstruction and Development (IBRD), created since the end of the Second World War with political and economic objectives, began to act systematically in the field of education, in the broadest sense and, in educational policies, in particular. In addition to these actions, there has been a growing spread of an ideology that strongly links education and economic development of countries.

This brings us to the present day, where the phenomenon of globalization has taken shape and imposes on future employees the need to master Information and Communication Technologies (ICT), to know how to speak a second language, without leaving aside the mastery of the mother tongue (reading and writing), mathematics and sciences.

All this without forgetting the acquisition of transversal competences - increasingly well-informed professionals, with emphasis on group work, peer interaction and group performance assessment, capable of performing multiple tasks with multidisciplinary knowledge. (^{SILVA, 2009}).

In the Biology discipline, the development of scientific literacy (competence) is directly related to the student’s ability to “explain scientific phenomena and draw informed conclusions on issues related to science”. (^{PINTO-FERREIRA, SERRÃO e PADINHA, 2006}, p.6).

At this point, we arrive at two basic questions - How do we achieve the standardization of competencies? How does decision-making about what to teach occur? The answer to both questions is interlinked, because it is easy to see that through the reformulation of curricula, standardization is achieved, with those influenced by public policies in the educational area. Two examples of standardization of curricula to achieve such propagated competencies and thus obtain the so dreamed economic development and improve competitiveness, were implemented in Portugal: Bologna Process (1999); reorganization of the National Curriculum of Basic Education - Core Competencies (2001) and in Brazil - National Curriculum Parameters - PCN and PCN+.

The interrelationship between Public Policies and Curricula brings us back to the context of the economy versus education, where the conduct of these policies is the product of articulations at a global level, as can be seen on the European Commission’s Education page, through the description of the “Role of the EU in the field of education and training”:

[...] European Union policy aims to support national action and contribute to solving common problems such as ageing societies, skills gaps in the workforce, technological change and global competition. (^{EUROPEAN UNION [s.d.]}).

You can also find it on this page:

While education and training systems are the responsibility of Member States, the European Union plays a key role in supporting and complementing efforts to improve and modernize national education systems. (^{EUROPEAN UNION [s.d.]}).

UNESCO itself, in the preface to its 2015 Education for All (EFA) Global Monitoring Report, mentions that “EFA Global Monitoring reports have played a prominent role in the journey to 2015 in supporting countries, providing robust assessments and analysis to underpin new policy development”. (^{UNESCO, 2015} p.4)

These reports are presented in a configuration that involves the evaluation of student learning associated with the quality of educational systems and have as assumptions “to diagnose to correct deviations, realigning local educational agendas according to the adjustments proposed by the agencies that sponsor them, and present legitimations that reaffirm their propositions, not only as being the best, but also the fairest”. (^{CAMPOS, 2005} p.2).

The question then arises: how is it diagnosed and legitimated to intervene? In education it’s been, in part, through exams. We came to a huge amount of Benchmarking tests (internally promoted by schools), National Examinations (eg. ENEM) and international (eg. TIMSS and PISA).

NATIONAL EXAMINATIONS AND THEIR IMPLICATIONS FOR THE EDUCATIONAL SYSTEM

At first, National Large-Scale Assessment examinations were introduced in many countries with the aim of diagnosing and improving the education system.

^{Anderson (2005}), mentions that countries such as China, France, Hong Kong and Japan have begun to employ national assessments to measure student performance and consequently make decisions about student progress. However, as already mentioned at the beginning of this article, education policy makers around the world have used Exams in an accountability approach. This accountability, in turn, has had a major impact on schools, i.e. on managers, teachers and students.

Authors such as ^{Looney (2011}), ^{Nichols e Berliner (2007}), report that when the evaluation system places an importance, a very great weight on external evaluations, teachers and principals perceive these, as a threat.

^{Klieger (2016}) conducted a survey in Israel, which found that only 6 principals (out of 30) and 13 teachers (out of 92) saw the large-scale National and International Exams as a working tool. ^{Scherer and Cruz (2015}) point out that large-scale evaluation policies, in addition to marking/labeling some students and/or schools, hold the actors involved in the learning process (parents, teachers and the students themselves) accountable, without necessary reflection on the causes of these results. Others, such as ^{Gesser and DiBello (2016}), refer to the privilege of areas and ^{Ventura (2014}), to disciplines in detriment of others, as we see in the texts below:

[...] Therefore, in schools, and in particular in disciplinary groups, there is a great concern for the management of the curriculum of the subjects of the years with National Examinations, which does not always happen with those of the remaining years of schooling. (^{VENTURA, 2014}, p. 90).

[...] The level of alignment and standardization that we perceive impact the school curriculum in terms of what should be taught and with which materials (books, ideologies, logic, and other resources) the students will be taught. In addition, it will affect the practice of teachers in terms of how to teach and why to teach certain subjects to the detriment of others available and other ways of learning and knowing. (^{GESSER e DIBELLO, 2016}, p.89).

The direct relationship between National Exams and curriculum content is recurrent in many articles ^{Ventura (2014}); Caria and Oliveira (2015); ^{Gesser and DiBello (2016}). Similarly, the transmission of content remains a recurring concern of teachers.

It can be seen that there are many approaches that can be analyzed when referring to the subject. In this research, the focus was on the test instrument and what is requested there to the student, in order to contribute in some way to its elaboration in order to produce the desired effects upon its application.

METHODOLOGY

RESULTS

Structure of ENEM - Brazil

In the ENEM there is no specific Biology test. This discipline is included in the Nature Sciences and its Technologies exam associated with Chemistry and Physics. All questions (also called items) have as standard three constitutive parts: base text (statement), enunciation (command given to the student) and (alternatives). Usually the basic text brings a problem situation and the items are contextualized with hypothetical or real subjects and themes of the participants’ daily lives. The test is of multiple choice; applied in two days, and on the day of application of the Science of Nature and its Technologies test is also applied the Mathematics test and the duration of the test is five hours.

Table 3 shows the quantification of the questions that require students to know Biology over the years evaluated. It should be noted that Biology issues are very representative, especially in 2011 and 2012. Over the 7 years evaluated, 114 questions demanded knowledge of Biology.

Table 3 Issues that demand knowledge of Biology from students in the ENEM tests of the time series under study 

Year	Total number	Biology number	% Biology
2010	45	17	37,7%
2011	45	20	44,4%
2012	45	18	40%
2013	45	14	31,1%
2014	45	14	31,1%
2015	45	16	35,5%
2016	45	15	33,3%
Total		114

Source: Prepared by the authors, 2019.

Finally, an overview of the Exams applied in Brazil and Portugal is presented in Table 1. The most striking difference between the two Exams is in the fact that the National Examination of Portugal is compulsory, since the exam score contributes to the final score of the respective subject (the assessments make up 30% of the students’ final marks).

Source: Prepared by the authors, 2019.

Frame 1 Characteristics of National Examinations in Brazil and Portugal 

The ENEM application fee waiver is valid for all students in the public network and for participants who claim to be low-income family members or to be in a situation of socioeconomic vulnerability.

In relation to the National Examinations of Portugal, registration and the tests of equivalence to attendance by self-proposed students¹ is compulsory in any of the two phases of the Examinations or tests, being subject to payment of €3 (three euros) per subject, in each phase. Internal pupils and self-proposals enrolling in national Final Examinations or tests of equivalence to attendance to improve their marks are subject to a payment of €10 (ten euros) per subject at each stage and the payment referred to in the previous paragraph does not apply.

CONTENTS PRESENT IN THE QUESTIONS OF BIOLOGY OF THE NATIONAL EXAM - PORTUGAL

The results related to the programmatic contents of Biology, present in the questions of the National Examination, are presented in figures 1, 2 and 3.

Source: Prepared by the authors, 2019.

Figure 1 10th grade biology contents, by teaching unit, present in the National Examination questions (2010-2016) 

As can be seen in the graph in figure 1 of the structural themes of the Biology Program, unit 1 - obtaining matter by heterotrophic beings in the processes of auto and heterotrophy in beings with different degrees of complexity occupies the first place quantitatively in all editions of the Examinations with 25 questions (sum of the editions evaluated), followed by unit 3 - processes of energy transformation, namely in the use of the aerobic and anaerobic pathways by living beings, with 16 questions. Unit 4 - aspects related to the maintenance of the conditions of the internal environment of organisms in the face of the fluctuations of the external environment, through the study of the cases of thermoregulation and osmoregulation in animals and phytohormony in plants come soon after, with 14 questions. Among the topics that have least appeared in the National Examinations are units 2, which focuses on the distribution of matter and its obtaining by beings with different levels of organization, with 12 questions and unit 0, which refers to the conception of the Biosphere (its diversity, organization, extinction and conservation) and the cell as a structural and functional unit (the degree of complexity of organisms), with only 6 questions over the last 7 years.

With regard to the subjects considered in the National Examinations, in addition to their numerical difference, it was found that in certain years some subjects did not even appear, namely: those related to unit 0 in 2010 and 2015, to unit 2 in 2012 and 2015, to unit 3 in 2016 and to unit 4 in 2014. Thus, taking into account the data on the contents of the 10th year, it can be seen that their distribution in the Exams is not uniform, both in relation to the units of the Program and throughout the years of application of the National Exams.

The data for the 11th grade contents (Figure 2) show that there is greater uniformity in the appearance of questions concerning the different teaching units - contents from all units appear in the National Examinations of all the years considered.

Source: Prepared by the authors, 2019.

Figure 2 11th grade biology content, by teaching unit, present in National Examination questions (2010-2016) 

Among the units with the greatest number of questions, unit 5 - cell renewal and growth - explaining the role of mitosis, DNA and protein synthesis is, without a doubt, the most requested with 41 questions over the last 7 years of application of the National Examinations. This is followed by unit 6, which deals with reproduction, understood as a process of information transfer, with 21 questions. On the other hand, unit 8 - levels of biological organization, which addresses the systematics of living beings, was requested in 20 questions. Finally, there was unit 7 - which explains biological evolution, with 19 questions.

When the data on the total programmatic content of the National Examinations are analyzed (Figure 3), it can be seen that the contents of the 10th year appear less often than the contents of the 11th year. In relation to the contents, there are also discrepancies within the contents of the 10th grade of schooling. It is also observed that units 1 (10th grade) and 5 (11th grade) were the most demanded in the 7 years of National Examinations analyzed, which suggests privileging contents that make up the educational curriculum.

Source: Prepared by the authors, 2019.

Figure 3 Total contents of Biology of the 10th and 11th grades, by teaching unit, present in the questions of the National Exam (2010-2016). 

Analyzing the workload - the number of 45-minute classes suggested by the Program - it was observed that some contents with an expected allocation of workload higher or equivalent to others, were not included in many editions of the exam. A good example is the unit 0 of the 10th year with 8 suggested lessons: in the editions of the National Exams of 2011 and 2015 there was no question on the subject. On the other hand, unit 1 of the same year, which appears first, quantitatively, in all editions of the Exams with 25 questions (sum of the editions evaluated), is suggested to be taught in 9 classes. In the case of the 11th year of schooling, although there is greater uniformity in the appearance of issues related to the different teaching units, and a greater number of classes foreseen in the subject Program, there is also a greater number of classes foreseen in the subject Program for less requested content. An example is the 13 lessons foreseen for unit 6, which presented 21 questions, while unit 5 has as a suggestion to be taught in 10 lessons, but its content was required in 41 questions over the 7 years of application of the National Exam.

Given these results, it should be noted that, over the years analyzed, the suggested time to teach the various teaching units, and eventually of cognitive deepening of the topics, is not similar, with a discrepancy between what is suggested in the number of teaching units by the Programs of the subject and the proportion of issues that correspond to them in the National Examinations. Corroborating with these results, studies conducted by ^{Ventura (2014}) showed that there is a great concern in the management of the curriculum of the subjects of the years with National Exams, which does not always happen with those of the remaining years of schooling.

The difficulty in testing the various contents indicated in the Biology-Geology Program (Scientific-Humanistic Course in Science and Technology) for the 10th and 11th years of schooling in Portugal, may be related to the number of questions - only 30 questions on average. There is therefore an impediment to requiring a large number of contents from students on a small number of issues. It is therefore an unrepresentative sample of the contents.

Corroborating this hypothesis, it is possible to find in the report of the National Exams of Secondary Education (period 2010-2016), prepared by the Institute of Educational Evaluation, I.P. (IAVE) the statement that [...] “In all the tests, the contents of all the binding themes, but not of all the themes/units of the program, shall be evaluated”. The same report makes the following reference:

Under the Biology component, the initial module (Unit Zero), Diversity in the Biosphere, was explicitly assessed only in 2010 and 2015. This unit is successively revisited in the teaching and evaluation of the remaining units. In turn, Unit 5, Cell Growth and Renewal, was explicitly evaluated in all the years under review. (^{IAVE, 2017}, p.51)

CONTENTS PRESENT IN THE QUESTIONS OF BIOLOGY OF THE ENEM - BRAZIL

In contrast to Portugal, the contents present in Biology issues in ENEM are not distributed by teaching unit, but by “contents and themes normally addressed in the whole Basic Education” (^{BRASIL/INEP, 2015}, p.62).

When the data in figure 4 are analyzed, it can be seen that the questions associated with the structuring theme “ecology and environmental sciences” were the most requested in the ENEM in the period studied, with 39 questions. Then, with 24 questions, there are questions about the structuring theme “molecules, cells and tissues”. With 20 questions, we find the structuring theme “heredity and diversity of life” and, among the most requested are also the “quality of life of human populations” with 16 questions. “Origin and evolution of life” was the least requested structuring theme, with a total of 7 questions distributed in 2010, 2012, 2014 and 2016.

Source: Prepared by the authors, 2019.

Figure 4 Biology structuring themes present in ENEM (2010-2016) 

The greater number of questions associated with the structuring theme “ecology and environmental sciences” over the analyzed period may be associated with the fact that this theme is more easily inserted into problem situations and contextualized in terms of social and economic relevance. Thus, there is an ease in the construction of the statements of the questions (items) and approach to the student’s daily life.

The theme “Molecules, cells and tissues”, second most addressed in the years evaluated, brings important contents, based on the scientific concept of Biology. Thus, it is expected that a representative number of questions from these contents will be elaborated, since the correct answer indicates the appropriation of scientific concepts by the student (one of the competences recommended in the ENEM and in the PCNEM).

“Heredity and diversity of life” is a structuring theme considered controversial (it involves ethical, political, economic and social debates), because they address issues focused on genetic improvement, transgenic, Human Genome Project, treatment of neurodegenerative diseases, among others. Thus, there is also a facility to approach the theme in a contextualized manner and through problem situations. This is also the probable cause of the relatively high number of issues involving the theme “quality of life of human populations”, since it refers to numerous social issues such as vaccination, sanitation, water resources, etc.

Despite being a very important content, “a theme of central importance in the teaching of Biology” and that should “constitute a guideline for the discussions of all other themes” according to the PCNEM (^{BRAZIL/MEC, 2006}, p. 22), the structuring theme “Origin and evolution of life” was very little addressed in the ENEM tests: only seven questions during the six years of analysis. One likely reason for this finding is that the theme is controversial because it involves religion and science (many students already have a preconceived idea about the subject, which has been infused into their homes and churches). Associated with this fact, in the document “Guide for preparation and review of items” of the Inep there is the recommendation “Avoid approaches to controversial topics” (^{BRASIL/INEP, 2010}) Thus, the different positions of the official documents are evident (that of the preparation of the ENEM exam and the Curriculum Parameter).

The results referring to the frequency of appearance of structuring themes over the years analyzed are presented in table 4. It should be noted that, both in absolute and in percentage terms, four structuring themes dominate “Ecology and environmental sciences”, “Molecules, cells and tissues”, “Heredity and diversity of life” and “Quality of life of human populations”.

A temporal analysis of the data in Table 4 reveals that some themes are favored with a large number of questions (items) per exam, while others are relegated to a secondary role or are not even requested from students. This reveals that in the period the examination prioritized the contents related to Cell Biology, Ecology and Heredity.

END REMARKS

Several authors such as ^{Stecher (2002)}, ^{Koretz (2008}), ^{Gesser and DiBello (2016}), Cária and Oliveira (2015) mention the interference of National Examinations in curricula. It is through the curriculum that we establish what will be taught in schools. Thus, defining what will be taught in schools also defines the profile of the citizen that the school is forming.

The curriculum is also an instrument that guides pedagogical practice when defining what to teach, when to teach, how to teach and how to evaluate. All this is conditioned to the knowledge that the curriculum privileges or secondary.

In relation to the teaching of science, Krasilchik, still in 1988, already mentioned two existing strands in relation to its objectives. One of them aimed at training scientists to meet the need for scientific and technological dominance of nations; the other considered the need to train citizens capable of acquiring, understanding and obtaining scientific information and with the ability to opine about them.

This polarization ends up creating expectations about the teaching and learning of science. There are those who mention that ‘The process of research in science deserves special attention in its approach to education, because it is the process of building science itself’. (^{MAIA and JUSTI, 2008}, p. 432).

In this perspective, it is important to mention that issues related to the research processes have an important role in the Exams, because it is one of the means to ascertain if the student has acquired scientific knowledge, which understands how and why scientific claims are validate, explaining and rationalizing with models. Developing questions to test specific knowledge and criteria of scientific knowledge is a major challenge.

Concurrently with the concern for scientific formation, the role of science education in the formation of the citizen was also postulated. ^{Kolstoe (2000}) mentions the need to promote science for citizenship and, in this context, the capacity of the teacher to develop in future citizens the dimension of attitude is a crucial point.

In turn, the interrelationships between science, technology and society have been highlighted in science education. On this subject, ^{Magalhães and Tenreiro-Vieira, (2006}, p. 86) point out that “Effectively, it is now widely defended a science teaching with a CTS orientation with the purpose of teaching about the phenomena in a way that connects Science with the technological and social world of the student”. With regard to the pedagogical practices that promote the aforementioned assumptions, the same authors mention “the performance of teachers in the classroom tends not to contemplate, at least intentionally and systematically, activities/strategies of a CTS/PC nature”.

Covering all these perspectives, i.e. providing students with solid knowledge for citizen science education and a CTS-based approach, enabling them to pursue a career in science if they so wish, is the school’s duty.

When it comes to assessment, in practice, the role of assessment is to support and encourage student learning. Thus, the type of assessment has to be in line with the way the teacher teaches and the degree of assessment must respect the depth with which the content was addressed in class. There is also an urgent need to link the school’s curriculum with the assessment matrices of education systems.

With these understandings, when resuming the objectives of this study, it was possible to ascertain, through the characterization of the tests, that both (National Examination of Portugal and ENEM in Brazil) have a reduced number of Biology questions to test the various contents indicated by the official reference documents of each country. Objective issues hamper the purpose of testing scientific knowledge. In this situation, the National Examination in Portugal surpasses the ENEM, because it presents in its structure essay questions, while the ENEM presents exclusively objective questions.

In terms of content, in addition to restricting the knowledge tested, both the ENEM and the National Examination tests focused on certain contents. In such circumstances, it became clear that in both countries, National Examinations may standardize, in terms of knowledge, what students have to learn by demanding more questions on a given topic.

Taking into account the information obtained in this research and taking into consideration the question raised by this article - Is the external evaluation adequate to the curriculum proposed in Portugal and Brazil? A discrepancy was found between the Biology-Geology Programs in Portugal, as well as the PCN’s in Brazil and the National Exams of both countries.