Quality evaluation parameters in early childhood education

This section will address parameters and concepts raised by academic research, as well as found in official documents.

These are relevant points to be considered in evaluating the quality of ECE: teacher training; the structure of the institution; the curriculum; pedagogical practices; early childhood policies; pedagogical work through the offer of times, spaces, materials, which promote the expansion of repertoires and pedagogical practices that enhance learning (MEC, 2012).

^{Bondioli (2014}) presents a reflection on the objectives and purposes of the evaluation. The author does not recommend that the evaluation be based on levels of development or on children’s proficiency tests, investigating by testing and specific tests is not advisable. “In daycare there are no fields of knowledge, no specific learning content, the continuous proposition that enhances experiences, verifying its effect on the child’s capacities and development”⁵ (Bondioli, 2014, p. 59).

^{Sousa and Pimenta (2018}, p. 5) reaffirm this concept, pointing out that

. . . measuring through external and large-scale tests is not in line with the purposes of early childhood education or with the meaning of the evaluation of the child contained in the current legal regulations, which do not prescribe the evaluation for classification or selective purposes.⁶

The document Educação infantil: Subsídios para a construção de uma sistemática de avaliação (MEC, 2012) presents guidelines for the establishment of a systematic evaluation, considering the diversity of the context and the specificities of the institution, capable of supporting educational policies and programs. According to this document, the evaluation shall:

The document also presupposes a set of aspects that are the subject of the evaluation:

The evaluation of early childhood education needs to provide for participation, dialog, negotiation for collective decision-making. ^{Bondioli and Sávio (2013}, p. 23) stress the importance of participation as a necessary criterion for quality, “all parties that are in any way involved and that work to consensually explain and define values, objectives, priorities, ideas on what the institution is like and how it should or could be”.⁷

In short, the importance of socializing, debating and making decisions about the evaluation instrument with teachers and managers who work directly in delivering the services stands out. For ^{Bondioli (2014}, pp. 51-52), the “assessment methodology should be formative and participatory, starting with data collection”.⁸

^{Festa (2019}) corroborates by emphasizing the importance of promoting an assessment of ECE that actually respects the child’s right, the specificities of the educational stage, participation, improvement of the quality and delivery of public policies.

A context evaluation, according to ^{Bondioli (2014}, p. 65), “in order not to contradict the principle of negotiation - even when dealing with pre-made instrument - requires a choice consistent with a given reality, based on the critical reading of this reality and of the value systems”.⁹

However, when dealing with an external evaluation instrument in early childhood education, the data are collected and interpreted by the external evaluator. The context and subjects are observed and do not interfere in decisions about the results and subsequent referrals. In order to be trustworthy, the instrument needs to be validated conceptually and by statistical testing.

Description and comparison of early childhood education evaluation models

The research, as previously pointed out, is qualitative in nature and is characterized as bibliographic and documental. It is also configured as exploratory, through searches in websites and literature, not restricted to authors or education systems, given that the instruments for assessing the quality of ECE have different characteristics, and may or not be linked to educational networks, be recent or under study, drafting or review and are even being tested in the contexts.

In 2018, Sousa and Pimenta undertook a bibliography survey of production dealing with the theme of the ECE quality assessment. The researchers point out the lack of consensus among the authors regarding approach, as well as linearity in type and purpose. The evaluation models are diverse and present different conceptions and may be in connection with the evaluation of children, institutions, faculty and programs, for instance.

^{Pimenta (2017}) presents a detailed review of the main models for assessing the quality of ECE. In addition to the author, the works of ^{Becchi et al. (2014}), ^{Bondioli (2009}), ^{Harms et al. (2006}), Harms (2013), ^{Marcuccio and Zanelli (2013}), ^{Moro (2018}), Moro and Souza (2016), ^{Motiejunaite et al. (2014}), ^{Myers (2011}), ^{Martínez Preciado (2010}), Souza et al. (2017) and ^{Tayler (2014}) were found in the literature.

^{Pimenta (2017}) classifies the evaluation instruments into two categories of analysis: focusing on the development of the children and focusing on environments, inputs and processes.

In models focused on the children’s development, the evaluation assigns the concept of quality from the level of learning achieved by the children. Among the instruments that follow this conception, the following can be mentioned: Early Years Foundation Stage; COR - Child Observation Record;¹⁰ ASQ - Ages & Stages Questionnaires; iPIPS - Performance Indicators in Primary Schools: Baseline Assessment; and PISA Baby - Program for International Student Assessment.

The second category of analysis, which features quality evaluation in early childhood education with a focus on environments, inputs and processes, evaluating the service delivery conditions. The following are aligned with this concept: ITERS-R and ECERS-R; ISQUEN and AVSI; ECCP (Escala de evaluación de la calidad educativa en centros pré-escolares); NQS (National Quality Standard); IDEA (Evaluation proposal by the Instituto de Evaluación y Asesoramiento Educacional); Qualità Lúdica (Strumento per l’analisi della Qualità Lúdica del Nido); ERVIS (Elementi Per Rilevare and Valutare L’integrazione Scolastica); SPRING (Strumenti per lo Sviluppo di Processi Riflessive e Indagini valutative Nei Nidi de parte dei Gruppi di Lavoro Educativi); and Indicadores da qualidade na educação infantil.

The Indicadores da qualidade na educação infantil served as a reference in building evaluation instruments adapted to different networks and education systems, such as: the Indicadores de qualidade da educação infantil paulistana [Quality indicators of early childhood education in São Paulo], which cover two additional dimensions in relation to the base document (Secretaria Municipal de Educação [Municipal Secretary of Education], 2016); the Indicadores da qualidade na educação infantil da rede municipal de ensino de Salvador (Indique) [Early childhood education quality indicators of the municipal school network of Salvador], which assign responsibilities for managing the evaluation result beyond the institution (Secretaria Municipal da Educação [Municipal Secretary of Education], 2016); and the Indicadores de qualidade da educação infantil do Distrito Federal [Early childhood education quality indicators of Distrito Federal], in line with the Base Nacional Comum Curricular [National Common Curriculum Base] (BNCC) (Secretaria de Estado de Educação [State Department of Education], 2019).

Another evaluative model identified in the literature is the MELQO - Measuring Early Learning Quality and Outcomes (United Nations Educational, Scientific and Cultural Organization [Unesco], 2017), with a hybrid methodology, focusing on both, the children’s development and in the learning environments.

Next, the models based on environments, inputs and processes will be described in a detailed and structured way, with a view to supporting the comparison between them and the subsequent verification of their suitability to the reality of the RME of Florianópolis. Models involving assessing the child’s development will not be addressed in the next section, since these tend to disregard the existence of different childhood realities and the non-linearity of child development pace, as described in the second section.

Discussion of results

As observed in the comparative analysis of the evaluation models, the Indicadores da qualidade na educação infantil (MEC, 2009) are aligned with the parameters of the educational legislation and meet most of the desirable aspects in an instrument for evaluating the quality of early childhood education, therefore, this evaluation model is the one in the higher alignment with the RME of Florianópolis; although it needs to be anchored by an evaluation policy capable of ensuring formative evaluation and provide for an intersectoral policy.

Since the 1990s, in the construction of guidance and guidelines, the RME of Florianópolis has been in line with the legislation and concepts announced in national documents and, in particular, with the Diretrizes Curriculares Nacionais para a Educação Infantil [National Curriculum Guidelines for Early Childhood Education] (MEC, 2010).

The Indicadores da qualidade na educação infantil dialog with the curricular documents¹³ of the RME of Florianópolis by promoting participation and debate, considering the specificities, identity and culture of the historical, social and economic contexts. They provide for dialog around the delivery of services based on structural, pedagogical and relational conditions, as promoters of the expansion of scientific knowledge through access to material and pedagogical conditions, contributing to the children’s learning and development. This instrument is capable of promoting a viable, valid and reliable evaluation for mapping and management of outcomes.

The evaluations developed and adapted in the different countries show the need for constancy in quality mapping. Italy presents an evaluative trajectory, which began in the adaptation of North American instruments, up to the construction of its own evaluation format. The Italian education system does not establish a single national model, but its common criterion is to address the elements of local reality as well as participation, taking into account the contextual diversity of institutions.

The Italian evaluation models are also close to the pedagogical proposal of the RME of Florianópolis, given the influence of Italian pedagogy on its curriculum base and guiding documents, in the training of professionals and, additionally, their relation to the values and purposes of the context. This finding raises the second evaluative possibility for the reality of Florianópolis: the context evaluation. Among the Italian models studied, SPRING is one of the best suited for the application in the context of Florianópolis. Nonetheless, it requires some adaptations, in particular in the analysis of results that, because of being descriptive, impair optimizing access and evaluation management.

Similar to Italy, there is a need for managers and faculty to know the different evaluation tools, in an action-research process, to then make choices, adapt these to the context and develop specific evaluation instruments based on the ECE principles and guidelines. However, the Florianópolis RME, even without an instituted evaluation policy, is in the process of constructing its own evaluation instrument, context evaluation.

The initiative is linked to the processes of continuing education on its curricular basis, which is a space for formative and democratic debate around its own outcomes. Thus, this attitude is not only configured as a field of study proposed by the evaluative experiences of the quality of ECE, but there is also the possibility of constructing the instrument itself, from the comparison of curriculum documents.

The WG for the compilation of the evaluation instrument was established in 2018 by early childhood education professionals, representing the five regions that geographically make up the municipality, with contributions from eight advisors from the Núcleo de Formação, Pesquisa e Assessoramento da Educação Infantil [Center for Training, Research and Counselling for Early Childhood Education] (NUFPAEI), with the advisory participation of Catarina Moro and Angela Scalabrin Coutinho, from UFPR. Some actions were carried out in this route. Initially, the WG participated in meetings in relation to: theoretical and conceptual studies¹⁴ on the evaluation of early childhood education; quality and participation; context evaluation; quality indicators; dimensions and analyses of the curricular documents of early childhood education of the RME of Florianópolis and some national and international evaluation instruments. In September 2018, organized by the NUFPAEI, the first seminar for Avaliação da e na Educação Infantil [Evaluation of and in Early Childhood Education] took place. This was the beginning of sharing with representatives of the educational units in the process ofestablishing the quality assessment instrument.

Based on the methodological choices and concepts mapped and developed by the WG in building the context evaluation document, a proposal was drafted for the dimensions comprising the instrument: educational-pedagogical relations; pedagogical action strategies; relationships; management and fundamental contributions. In 2019, study groups - of a participatory formative and representative nature - were held in the regions of the municipality to share the path of drafting the instrument. In turn, the representatives of each educational unit had the task of socializing in the collective, at their workplace, the information presented in the regional ones. The WG also had the task of socializing, in the formative meetings, the debate and the demands presented in the regional ones.

The second seminar − Avaliação de Contexto na Educação Infantil [Context Evaluation in Early Childhood Education] - took place in November 2019, with the objective of going more in-depth into the theoretical perspective and learning and dialoguing about the methodology of the context evaluation in ECE, developed in the Romagna region of Italy, coordinated by Antônio Gariboldi. This seminar disclosed the methodological choice of the instrument and the main dimensions comprising it.

In 2020 and 2021, the process of drafting the evaluative instrument was given continuity and has been establishing itself as the WG outlines the methodological principles, writing and strategies of its operationalization are outlined.¹⁵ Currently, the first dimension is in the finalization processes. Thus, the estimated date for its conclusion depends on numerous variables, among which the feasibility of applying the instrument in educational units, given the current pandemic scenario and the health protocols that focus on the organization of care in early childhood education in the municipality.

The discussions presented throughout this article enabled verifying that the implemen- tation of a quality assessment system in early childhood education requires inseparability between evaluation policy and evaluation instrument.

The evaluation policy is comprehensive because it is a broader planning focused on public policies, which guarantee actions through educational legislation, as well as it includes provision for funding for the quality of offering, efficiency in the management of systems and, also, faculty training. For an evaluative policy to be efficient, an intersectoral support network is needed, which refers to the guarantees provided for in public policies, in the social commitment to education and the need for articulation and planning of the different institutions.

The purpose of the evaluation instrument, of a context-driven approach, is to collect, analyze, and define the quality of the offer, considering the evaluation methodology. In addition, this instrument proposes that the focus of the evaluation be based on inputs, environments, and process, with a view to addressing institutional specificities, in addition to this it also provides for system-level coverage. Moreover, it marks out the importance of considering the physical and curricular environments with quality parameters related to infrastructure, materials, service management, pedagogical practices and the guarantee of the child’s right as the main purpose of the educational unit; connected to this are the relational and social environments with parameters that provide for the participation of the educational community, anchored in the social purpose, which the institution establishes in the community in which it delivers the services.

^{Moro (2017}) highlights the importance of a system for evaluating the quality of early childhood education that considers the physical, relational and social environments, in which the instrument emphasizes pedagogical practices and the relationships with the educational community.

Further, the instrument should include the management of the evaluation, which involves the establishment of an improvement plan, with goals, deadlines, persons responsible and periodic self-evaluation for monitoring outcomes management.

The elements described above characterize the evaluation of early childhood education as a process and not only as an end determining the level of quality. Each element has its important role in mapping and promoting the quality of care.

Final considerations

This study identified, described, and compared quality evaluation models in early childhood education of national and foreign origin. The analysis showed how appropriate evaluative instruments based on access, inputs, and processes are, as well as the models that present methodological coherence, are adaptable to the context under evaluation and foresee participation. The evaluation instruments that most closely approached the reality of Florianópolis were Indicadores da qualidade na educação infantil and the Italian instruments of context evaluation.

Regarding the Indicadores da qualidade na educação infantil, the municipalities of São Paulo, Salvador and Distrito Federal have already shown the adequacy of the instrument, adapting it to their pedagogical proposals. This evaluation instrument is part of a national standard, representing a common core, but can also be adapted to addressing elements relevant to the context and specificities of specific educational units. This instrument has the potential to promote the evaluation of the quality of early childhood education in a systemic and comprehensive way, considering the need to have a support network for managing outcomes.

Concerning the context evaluation, research undertaken in Brazil, such as ^{Bondioli (2014}), ^{Souza et al. (2017}) and ^{Castro and Martins (2018}), confirm the pertinence and potentiality of this evaluative concept in promoting the improvement of the quality of Brazilian early childhood education. This evaluative strategy promotes participation and reflection around the services offered to children (Souza et al., 2017). The challenge of a context assessment is quite broad, as it covers relational, pedagogical, and structural dimensions.

A quality evaluation system in early childhood education should provide guarantees, such as State public policies, targeted at training faculty and education managers, qualifying them for the mastery of processes and the evaluation instrument, as well as assigning responsibilities to indirect managers. The possibility for each municipality to think and build its own evaluation instrument based on the guidelines that guide its pedagogical proposal - having an evaluation policy with financial support and mutual collaboration between the union and the federal entities (states and municipalities) - with the technical and pedagogical advice of the MEC for a systemic and integrated evaluation among Brazilian municipalities of the instrument to its context must be considered.