Introduction

Public policies cycle might be understood as a sequential logic of didactic purposes, composed by the following steps: 1^st) organization agenda; 2^nd) formulation agenda; 3^rd) implementation; 4^th) evaluation; 5^th) finishing (^{PALUMBO, 1994}). However, in the practice plan, such steps are connected, and they are not regularly sequential. For example, at the same time an idea is considered to compose the governments’ agenda, it may be evaluated whether the pursuit is feasible in fact, coexisting two moments of a cycle (organization agenda and evaluation).

Evaluation step may be present in the three first moments of policies cycle, contributing to (re) think about a public policy. This step is fragmented in three parts, namely: 1) policy analysis (it occurs before the definitive decision making on a policy. Essential knowledge for these studies is on economy, mathematics and statistics); 2) process analysis (it focuses on the process by which policies are translated in programs and how they are administered. The subjects underlying this study correspond to political Science and public administration); 3) program evaluation (it is restricted to the post-decision step which tries to determine whether a program is reaching its goals, what the program is carrying out and who benefits from it. It considers studies from the psychology and sociology areas) (^{PALUMBO, 1994}).

Evaluating a policy, both in its implementation process, which may be understood as monitoring the life of a policy, and verifying its outspreads and impacts when finishing the step correspond to essential actions for a policy success. The opposite is also real, in other words, the lack of periodic evaluation mechanisms and the absence or barely existence of evaluation practices contribute negatively to a public policy success.

Considering the relevance of this policy cycle step, this work aims at presenting a survey on the production of knowledge regarding monitoring and evaluation of education decennial plans, understood here as the National Education Plan (PNE in Portuguese acronym), State Education Plan (PEE - Portuguese acronym) and Municipal Education Plan (PME also in Portuguese acronym).

In order to reach a broader search, two generation of plans are considered, after Democratic Brazil - 1988 (PNE 2001-2010 and PNE 2014-2024 and the respective subnational plans), as well as a temporal clipping covering the entire historical period from the promulgation of the first PNE to the present cover.

Therefore, the state of knowledge about the scientific production contained in the CAPES Thesis and Dissertation Portal is addressed, also in the Annals of ANPED national meetings and in Online Electronic Scientific Library (Scielo Brazil) database, in the period from 2001 to 2017.

For presentation purpose, the issue is structured in four sections. The first one approaches the Brazilian educational plan in a historical perspective. Next, it presents and discusses the production of knowledge on the decennial plan. The third section deals specifically with the scientific production mapping regarding the monitoring and evaluation of decennial plans. Lastly, final considerations are presented, where some notes are described regarding the possible research focuses which may be studied from the gaps verified in the productions analyzed.

National education planning policy

The idea of an education plan of “[...] State, further the governments with national dimension, responsibility of all the federated entities and approaching different education levels and stages” (^{BORDIGNON, 2009}, p. 91), despite dating back to the 1930s - whose historical landmark id the “New Education Pioneers Manifest” - and advocated by important legal diplomas, always found strong obstacles for its effective materialization in the country as public policy. As ^{Dourado (2017, p. 12)} points:

[...] the relation between the proposition and approval is not linear to its materialization process as public policy. In other words, PNE materialization implies actions and policies which are effective from several clashes and conjunctures, which contribute to the plan realization or put it in a second plan.

An evidence of the little tradition of the Brazilian state to guide our education through long-term educational planning is that

Only in 1988, with the approval of our Federal Constitution, the Union was obliged to draw up a law establishing a multiannual duration National Education Plan (art. 214), after redefined as decennial, with the aim at articulating the national education system (EC 59/2009). The first decennial education plan in Brazil was born then in 2001 by the Law 10,172, January 9^th 2001 (^{NASCIMENTO; GROSSI JÚNIOR; PEREIRA, 2017}, p. 110).

Regarding the elaboration process ad approval of the Brazilian first National Education Plan (PNE in Portuguese acronym) instituted by specific law, ^{Scaff and Oliveira (2018, p. 143)} assert:

The first PNE elaborated under the aegis of the 1988 Federal Constitution was marked by a wide participation of Brazilian society, through two National Education Congresses (CONED) and wide debates held across the country, coordinated by National Forum in Defense of the Public School (FNDEP). However, the Project elaborated by the Brazilian failed because it challenged the ongoing federal government neoliberal project. After a long debates and confrontations, the document approved and sanctioned by Law number 10,172, d January 9^th 2001 (^{BRASIL, 2001}) remained committed to the budgetary constraints that prevented the achievement of its goals.

When evaluating PNE 2001-2010 already expired, ^{Dourado (2010, p. 685)} points the limits for its concretizations, which are from “[...] restrictions to the education management and financing, because of nine presidential vetoes to what was outlined”, what configures a “formal plan, marked by absence of financing concrete mechanisms”. Thereunto, it was not made effective as guiding for policies, planning and management of national education. An example is on the reduced number of state and municipal education plans approved due to such PNE, despite it being a legal requirement.

Internal organicity needy, exaggerated as to the number of set goals - 297 in total, denoted “[...] a high index of dispersion and loss of distinction sense between which is principal and what is accessory” (^{SAVIANI, 2008}, p. 275) and did not prevent concrete and effective mechanisms to the corresponding monitoring and evaluation of its implementation. PNE 2001-2010, in practice, did not was translated as a regulation mechanism able to “[...] lead the guidelines for planning, management and effectiveness of educational policies, as expected from a national plan that contributes to the federative pact” (^{DOURADO, 2010}, p. 688).

However, despite all the problems, limitations and weaknesses of PNE 2001-2010, such Plans aroused the hope of civil society about the approval of a forthcoming Plan, notably its most democratic construction. Such wish may be observed in several works published near the end of the Plan 2001-2010 validity, among those we can indicate: ^{Dourado (2010)}, ^{Aguiar (2010)}, ^{Di Pierro (2010)} and ^{Sampaio (2011)}.

Amidst this context of expectation to overcome the impasses in education and searching for a path for a social quality education, the Plan that replaced PNE 2001-2010 started to be elaborated from the end of 2010, taking about three years and a half for its processing and final approval.

Regardless the marches and countermarches during the long, disputed and complex processing, it was approved by the Law number 13,005, June 25th 2014 (without any veto from then President Dilma Rousseff) PNE 2014-2024, composed by 20 Goals and 254 strategies addressed to the all the teaching levels, stages and modalities, in order to represent a new encouragement in terms of educational planning as a state policy.

Differently from that occurred with PNE 2001, elaboration of the current PNE also advanced because it resulted from the broad participation of civil society, notably in the different stages of the National Education Conference (CONAE), which took place in 2010 and 2014. Other important advance is regarding the structure of the final document approved, leaner and more objective. To ^{Saviani (2014)} it is a positive aspect, because facilitates the PNE 2014-2024 monitoring process and evaluation.

However, in a contemporary scenery with significant political setbacks and impactful “fiscal adjustment”, whose greater expression was the Constitutional amendment number 95/2016 approval by an illegitimate and unpopular federal government by Michel Temer “[...] installed as a result of a legal-media-parliamentary coup [...]” (^{SAVIANI, 2018}, p. 45), PNE 2014-2024 effective materialization is significantly compromised.

Thus, an “[...] effective mobilization and participation by the civil society and monitoring, evaluation and materialization policies of the new PNE (2014-2024), in order to constitute thus plan in an effective state policy for education” (^{DOURADO, 2017}, p. 19) is imperious.

Considering the decennial plans relevance to design the directions of national education, the nest section presents Brazilian production of knowledge mapping at master and doctor degrees and issues/scientific works from independent researches regarding the theme.

PNE, PEE and PME: production of knowledge in Brazil

This research is characterized as qualitative and quantitative (^{TRIVIÑOS, 2007}), with bibliographic and documental character, because it is an orderly, analytical, descriptive and critical survey (^{TEIXEIRA, 2006}) on the production of knowledge about the decennial education plans focused on the monitoring process and evaluation of this policy.

Such approach was chosen because it is consistent with the demand required by the object of study, as well as the relationship established with the researcher. Furthermore, this strategy was adopted because it comprehends an “inventor and descriptive methodology of academic and scientific production” (^{FERREIRA, 2002}, p. 259).

The research started by surveying everything that was produced on decennial plans (PNE, PEE and PME) from 2001 to 2017, in the portal of theses and dissertations of CAPES, searched by the descriptors “national education plan”, “state education plan” and “municipal education plan”. In the first moment, 582 results were obtained¹. Next, the works which really approached decennial plans as central matter were separated through reading their titles. When only the title did not make the scope of work clear, the abstracts were also read.

Thereunto, 85 researches remained from the universe of 582 works published in the portal of theses and dissertations of CAPES, from 2001 to 2017, according the Table 1.

The data from the Table 1 show that after a triennium (2001-2003) in complete silence, only after 2004 the first researches at stricto sensu postgraduation level specifically addressed to theme decennial education plan started to be defended. However, the amount of works dedicated to the theme in approach remains very incipient until the early years of the next decade.

The very incidence of this theme on the academic production of the area is possibly related to the historical failure and low priority to guide the Brazilian education in face of the decennial plan and to the corresponding researchers’ lack of interest in approaching such a theme. We highlight that only from the last re-democratization of the country (1988) that the matter of long-term educational planning takes centrality and materiality, ceasing to be a mere specter, especially after 2001, with the promulgation of the first PNE.

Still through the Table 1 we observe a large increase in the amount of postgraduation level works focused on decennial plans from the year 2015, what denotes a significant maturation of reflections/interest on the theme, especially after the CONAE editions (2010 and 2014). Thus, from 2015 to 2017, the production of knowledge regarding the theme did more than doubled, because there are 11 works published in 2015 and 25 researches in 2017. Notwithstanding such growth, even so, the amount of research focused on the educational planning matter at the systemic level may be considered very small, a highly relevant theme, given the importance of planning.

The scarcity of research on the theme found in the CAPES Theses and Dissertations Portal is also observed in relation to the ANPED website, which houses scientific papers presented at national meetings. Following the same rigor and methodological course to survey and treating the data from the Portal of CAPES, searches were carried out in the website of ANPED, especially from the GT-5 (work group dedicated to educational public policies). The amount of works produced and published by GT-5/National ANPED in the researched period correspond to 264 issues, of which only 5 remained focused on the decennial plans (Table 1).

Table 1 shows that, despite the first work focusing on decennial plans published by GT-5/National ANPED in 2001, for eleven uninterrupted years (from 2002 to 2012) the theme remains completely absent from publications of the GT above.

This finding undoubtedly expresses the great lack of interest and little investment in the theme by researchers, especially those linked to the field of educational policy research, usually gathered in GT-5.

Data still show that the amount of works/publishing from the National ANPED meetings focused on the theme only takes momentum and constancy from the year 2013, reaching 4 in total until 2017. Even so, the scientific production on education decennial plans found in GT-5/National ANPED is small, including inferior - both in absolute and relative terms - to those registered in the Portal of Theses and Dissertations of CAPES in the corresponding period.

The scarce production of knowledge on decennial plans lead the authors to the initial questioning on the amount of researches addressed to the policy cycle steps, posteriorly by considering that from the theme “decennial plans”, other relevant themes to the education area emerge to be investigated. Thereunto, the distribution of works was by thematic categories.

Elucidating that such categories were not randomly created by the researchers is important, because they emerge from the central focus of works identified when reading the titles and abstracts of all the investigations surveyed through the research in the Portal of CAPES (85 works) website of National ANPED (5 works), which approached the plans as pivotal aspect.

The result of thematic distribution of works from master and doctor degrees may be followed through the Table 2.

Distribution of works by thematic category resulted in 24 different categories, in other words, each work analyzed when treating decennial plans as central matter approached it within a different thematic perspective. In some cases, despite the particularities in the research, there are recurrence themes. Thereunto, some thematic categories are highlighted in terms of frequency of appearance in the set of works researched, namely: Elaboration plans process (34), Comparison between plans (8), Implementation plans process (5), Management concept in the plans (4), Social representation (4), Financing (3), Quality of education (3), Teacher training (3), Gender (3), Plans monitoring and evaluation (2), Right to education; and (2) Political conciliation.

Considering that all the thematic approached by the researchers’ present relevance to the education field, certain quantitative research imbalance by theme is registered. Thereunto, we may say that these themes regarding the contemporary education field has not been appropriately approached by researches in the area, such as education financing, quality of education and teacher training.

Regarding the financing education plans scope, we may assert that this theme has not yet been sufficiently approached, even more if we consider it is an indispensable aspect for the realization of long-term educational planning, which demands significant resources. Moreover, as ^{Saviani (2008, p. 317)} highlights, in a decennial plan, education financing is “[...] the goal of the goals [...], as it is the precondition and feasibility mechanism of all other targets.”.

Other two central themes for Brazilian education advance that need to be more explored in researches on the plan are: quality of education and teacher training.

Quality of education is not a new theme, and it is constituted as a “[...] complex and embracing phenomenon, and comprehend several dimensions” (^{DOURADO, 2007}, p. 9). Retaken it with greater emphasis is necessary “[...] in the scope of debates on education public policies, considering the place that PNE occupies” (^{ARANDA; LIMA, 2014}, p. 308).

Teacher training, in its turn, is undoubtedly one of crucial dimensions that reflects on quality of education, because “[...] there is no education quality without teaching quality” (^{LEITE et al., 2010}, p.102).

Qualitative imbalance also reoccurs when analyzing policy cycle investigations. We observe that the greater number of works is based on the policy formulation and elaboration process (34 works), followed by those which approached the implementation process (5 works). Lastly there are researches addressed to the monitoring process and evaluation of policies (2 works).

In the scope of works in the website of National ANPED - GT-5, we noticed out certain dissonance in some respects with the categorization of stricto sensu postgraduation studies.

Thematic categories that emerge from the published issues in annals of Nacional ANPED are proportional to the quantitative of works and differ from the categories contained in Table 2 - the only recurring theme is teacher appreciation². It shows that the focus of interest of this group of researchers is distinct from that coming from the CAPES Portal, which be understood as a favorable aspect. The reason is the focus on the decennial plans is broadened, producing knowledge on multiple themes about decennial plans and pointing to indicative gaps for other researchers to advance the discussions, enriching the study field.

Sparse works that are revealing for the educational field may be found from the 1920s, but it is only from the mid-1980s that academic productions intensify, as well as thematic diversification of works, thanks to the beginning of the expansion and solidification of higher education and postgraduate programs in Brazil (^{GATTI, 2010}).

At the same time, researches in the public policy area has reached quantitative growth since the 1960s, a period in which the area of public policy has become a field of scientific investigation, due to a number of historical factors, such as the creation of ANPAE (1961), implementation of postgraduate studies in Brazil (1965), creation of education journals (1971), ANPED (1976), creation of GT 5 - State and educational policy (1986/1987). However, the real expansion of this research field was after the 1990s, with the creation of educational politicy lines and research groups, of the subject educational policy, of journals in educational policy; and the strengthen from the 2000s, due to the growth of specialized scientific events and creation of research networks (^{STREMEL, 2016}).

These historical facts point that research in education, in general way, have been growing in recent years and this growth, as well as the strengthening, especially in the public policy research field is very recent, which implies few productions in the area. A reflection of this is the small amount of research focused on decennial plans, even though it is a relevant theme for the educational area, as well as for the public policies field. It indicates that investment in research addressed to this theme is necessary, and it constitutes a fruitful scope of research.

In order to deepen the discussions on the production of knowledge about the decennial plans, next section deeply approaches the matters regarding the productions addressed to the monitoring and evaluation of public policies.

Scientific production mapping: decennial plans monitoring and evaluation

The act of evaluating a planning policy does not correspond to an isolated step, but in “[...] one of the components of the evaluation process” (^{KIPNIS; ALGARTE, 2001}, p. 153). In other words, decennial plans evaluation should not be understood as an activity tight to the planning process, because one of the objectives of monitoring is verify possible blanks to present solutions, understand necessary adjustments and fully achieve the objectives.

Furthermore, monitoring and evaluation of plans become understood not only as an articulated instrument for correcting their directions by the government, but as a policy way addressed to accountability of governments to society with a view to knowing their performance or results, impacts and effects (^{DRAIBE, 2001}).

Recognizing the relevance of monitoring and evaluation of education decennial plans, although lightly flagged in the PNE/2001 (Art. 3, paragraphs 1^st and 2^nd), gained greater repercussion with the PNE/2014. Article 5 of this document establishes the obligation of continuous monitoring and periodic evaluation of the Plan's goals, carried out by the Ministry of Education (MEC), the Education Committee of the House of Representatives and the Education, Culture and Sport Committee of the Federal Senate, National Council of Education (CNE); and National Education Forum.

Due to the legal determination contained in the PNE / 2014 regarding uninterrupted monitoring and evaluation at determined time intervals in relation to the document's goals, the states and municipalities also had to organize to establish their respective monitoring and evaluation committees of the Plans.

Because of the relevance of the monitoring and evaluation actions of decennial planning, in this stage of the research, we sought to collect data that met the criteria listed for further study, now focusing on the production of knowledge on monitoring and evaluation of decennial plans in their complexity.

Added to the localized results about the systemic planning policy evaluation stage on the portal of thesis and dissertation CAPES portal and on the National ANPED website - GT 5, the number of works was expanded through research undertaken on the Scielo Brazil digital website, equivalent to the period from 2001 to 2017.

In total, 4 works were found on the CAPES portal, 1 work on the ANPED website (focused on teacher appreciation, but in the bias of the PNE / 2014-2024 goals evaluation); and 4 works at Scielo Brazil. Once the material was selected, the data lapping process was performed. Thereunto, readings of the works were undertaken in their entirety, not just the abstracts. The first reading had as main analyzing whether that production really could be inserted in the researched thematic. At the end of this stage, only 2 dissertations were excluded, because although the abstract appeared to be related to the decennial evaluation policy matter, the full paper content did not signal the same.

To perform the second reading stage, the authors created a reading sheet containing essential criteria to answer the investigation proposed, such as: identifying types of research (master degree, doctor degree, parallel research, etc.), database from, belonging theme, study object, temporal clipping, research focus, regional distributions, existing trends in terms of adopted research methodologies and notes of possible focuses of scientific investigation from the existing production.

Criteria listed to subsidize the analysis of the productions located in the CAPES portal, National ANPED and Scielo Brazil are presented in Chart 1.

Through data surveyed and made explicit by the Chart 1, production of knowledge focused on monitoring and/or evaluation is still minimal. At stricto sensu postgraduation level, only two (or 28.57% from the total) researches are registered, both of master degree. The absence of investigations at doctorate level approaching the theme studied probably are related to the fact that Brazilian postgraduation offers “[...] many more masters than doctoral courses, with shorter duration and more vacancies offered, which explains the greater production of dissertations [...]” (^{SILVA; ALMEIDA; CAIADO, 2017}, p. 865).

These researches, despite performed in different states (Pará and São Paulo), present similar characteristics, because both selected as study object the education municipal plans. Furthermore, these two works study a purposeful analysis, in other words, they search for signal monitoring mechanisms and plan evaluation.

Regarding the other searches found (5 or 71.43% from the total), they are scientific issues, and three of them make a general analysis of the document, and other two analyze specific goals of the Plans. From the total of issues found, only one of them is about the current Plan, other ones perform an analysis on the PNE/2001, what indicates the lack of studies published on the current decennial plan.

Table 4 shows the qualitative distribution of works produced that correspond to the theme, organized by Brazilian region.

Table 4 demonstrates that the greater quantity of academic productions published on the theme approached is from institutions localized in the Southwest region of Brazil, responsible by 4 (or 57.14%) production from 7 works surveyed in total. This high concentration of publications in the Southwest is because this geographic region gathers brings together most postgraduate programs and leads the country's knowledge production ranking.

Also noteworthy is the lack of expressiveness of the works produced in the other regions of the country (responsible for only 1 or 14.28% of the total works), the lack of academic production regarding the theme in focus in the South, the second most prominent in the country in terms of concentration of the national postgraduate.

Regarding the main methodological approaches, research tips and data collection procedures adopted by the analyzed works, Table 5 is illustrative.

Based on information presented by the Table 5, we observed that all the scientific production under study are grounded by the research qualitative approach, defined by ^{Esteban (2010, p. 127)} as

[...] a systematic activity oriented to in-depth understanding of educational and social phenomena, to the transformation of socio-educational practices and sceneries, to decision making and to the discovery and development of an organized body of knowledge.

The predominance of qualitative studies found proves, therefore, the growing “[...] interest that researchers in education area have demonstrated to the use of qualitative methodologies” (^{LÜDKE; ANDRÉ, 1986}, p. 11).

According to information in Table 5, bibliographic research is also prevalent in the scientific production analyzed. These data are aligned to the nature of this research type, which “[...] constates the first step for all the academic activities” (^{ANDRADE, 2005}, p. 39).

Regarding the data collection, documental analysis is the most frequent and stand out the other procedures employed. Its adoption is related to the fact that both may complement “[...] information obtained by other techniques [...]” and because allows unveiling “[...] new aspects from a theme or problem” (^{LÜDKE; ANDRÉ, 1986}, p. 38). Other reason by its supremacy is probably because it is not possible monitoring and/or evaluate thoroughly assess the decennial education plans without carefully approaching them.

Final considerations

In the last seventeen years the production of knowledge in Brazil about decennial plans has been very scarce. From 2001 to 2017, only 85 researches at postgraduation level had as central object of analysis the education plans (PNE, PEE or PME) and only 5 works were published by GT5 from the national meetings of ANPED which deal the matter. In addition to scarce, such production is also characterized because it is recent, as most works were published in the current decade.

Whether considering only the production of knowledge focused exclusively on decennial education plan monitoring and evaluation, it is even more minimal. From the CAPES portal, ANPED (GT-5) and Scielo Brazil websites, only 7 researches which approach the decennial education plans monitoring/ evaluation stage were found. These data show the lack of studies on the educational planning area, especially as regards the evaluation and monitoring of these documents.

Although the amount of research focused on monitoring/evaluation of the Plans is very low, the few investigations found bring important contributions to the area and raise new investigation points, especially when analyzed in the current context and following the demands contained in the PNE 2014-2024, such as the organization of committees to monitor and evaluate the Plans, as well as to propose public policies to meet the existing goals in Decennial Plans.

Thus, we believe that the decennial plan evaluation scope is an open and fruitful field for further research, especially regarding the process of monitoring and evaluation of the Plans, whether at federal, state or municipal levels. The need for studies that have the Plan follow-up commissions as the central object of the research is also highlighted, as well as the actions of the Government from the notes made by these commissions, in order to overcome the obstacles and reach the goals contained in the documents.

Finally, there is also the need for investigations within the scope of Plan evaluations that have a more specific focus, that is, studies that analyze, monitor and evaluate the process of implementation of specific goals, because according to the data, only two works elaborated in the period 2001-2017 focused on a specific goal (goals 7 and 17, respectively).