SciELO - Scientific Electronic Library Online

 
vol.18 número49PERFIL DOS ESTUDANTES PÚBLICO-ALVO DA EDUCAÇÃO ESPECIAL NA EDUCAÇÃO SUPERIOR BRASILEIRA ANTES DA LEI DE RESERVA DE VAGAS índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Compartilhar


Revista Práxis Educacional

versão On-line ISSN 2178-2679

Práx. Educ. vol.18 no.49 Vitória da Conquista  2022  Epub 18-Fev-2022

https://doi.org/10.22481/praxisedu.v18i49.9388 

ARTIGO

SYSTEM OF EVALUATION OF JOURNALS IN BRAZIL: IMPACTS OF THE EVOLUTION OF THE QUALIS-PERIÓDICOS CRITERIA IN THE AREA OF EDUCATION

1Universidade Estadual Paulista - Araraquara, São Paulo, Brasil; lzsene@unesp.br

2Universidade Estadual Paulista - Araraquara, São Paulo, Brasil; jose.bizelli@unesp.br


ABSTRACT:

The present article analyzes the production of the Qualis-Periódicos in Education, in the period from 2010 to 2018 - triennium 2010 to 2012; quadrennium 2013 to 2016 and mid-term evaluation on the years 2017 and 2018. The instrument serves to evaluate the production of faculty members of the Graduate System in Brazil, although it is often used for other purposes. It is possible to determine two types of change in the evaluation criteria: either incremental, between the triennium and the quadrennium; or disruptive, wherein in 2019, in the mid-term evaluation, a new factor is introduced to measure impact or citation. The research analyses are based on documental sources: the Qualis itself and the guiding and evaluative documents of the education area. The data show a considerable increase in the stratification of the journals caused by the objectification of the criteria in the first periods and a rupture in the last one; a drastic decrease of the evaluation universe with the introduction of the concept of parent area; and an uncertainty about the evaluation that will come in 2021.

Keywords: qualis-periódicos; scientific production; evaluation system

RESUMO:

O artigo analisa a produção do Qualis-Periódicos da área de Educação, no período de 2010 a 2018 - triênio 2010 a 2012; quadriênio 2013 a 2016 e avaliação de meio termo sobre os anos 2017 e 2018. O instrumento serve para avaliar a produção de docentes do Sistema de Pós-graduação no Brasil, embora seja frequentemente utilizado para outros fins. É possível determinar dois tipos de mudança nos critérios de avaliação: uma incremental, entre o triênio e o quadriênio; outra de ruptura, quando em 2019, na avaliação de meio termo, introduz-se um novo fator para medir impacto ou citação. As análises da pesquisa estão fixadas sobre fontes documentais: os próprios Qualis e os documentos orientadores e avaliativos da Área da Educação. Os dados demonstram um aumento considerável na estratificação dos periódicos ocasionada pela objetivação dos critérios, nos primeiros períodos, e uma ruptura no último; uma diminuição drástica do universo de avaliação com a introdução do princípio de área mãe; e uma incerteza quanto à avaliação que virá em 2021.

Palavras-chave: qualis-periódicos; produção científica; sistema de avaliação

RESUMEN:

El artículo analiza la producción del Qualis-Periódicos del área de Educación, en el período desde 2010 hasta 2018 - trienio 2010 hasta 2012; cuadrienio 2013 hasta 2016 y evaluación del medio término sobre los años 2017 hasta 2018. El instrumento sirve para evaluar la producción de docentes del Sistema de Posgrado en Brasil, aunque se utilize frecuentemente para otras finalidades. Es posible determinar dos tipos de cambios en los criterios de evaluación: una incremental, entre el trienio y el cuadrienio; otra de ruptura, cuando en 2019, en la evaluación del medio término, se introdujo un nuevo factor para mensurar impacto o citación. Los análisis de la investigación están fijados sobre fuentes documentales: el propio Qualis y los documentos orientadores y evaluativos del Área de Educación. Los datos demuestran un incremento considerable en la estratificación de las revistas científicas ocasionada por la objetivación de los criterios, en las primeras revistas científicas, y una ruptura en el último; una disminución drástica del universo de evaluación con la introducción del principio del área madre; y una incertidumbre en lo que respecta a la evaluación que se llevará a cabo en 2021.

Palabras clave: qualis-periódicos; producción científica; sistema de evaluación

Introduction

The Qualis-Periódicos is much more than a portal organized by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)1 to sanction a national ranking of journals in each area of knowledge, since, by classifying and valuing the scientific communication vehicles, it is used to qualify the production of teachers linked to the Brazilian Postgraduate System, interfering directly in the quadrennial score of the Postgraduate Programs. Thus, the well evaluated journals transfer value to their authors, improving the score of their programs and allowing greater resources from funding agencies and better management autonomy over the expenses incurred.

CAPES, therefore, creates a national metric to face this specific assessment issue - regardless of regional or historical differences -, so that it returns to society the investment made in the academy, through the indication of the Postgraduate results in the generation of human resources and in the development of new products and processes (MONTEIRO; FURLAN; SUAREZ, 2017).

However, the management of journals facing the metrics adopted by the evaluation system - by Qualis-Periódicos - has as a result an extremely competitive environment due to the imposition of statistical locks that perpetuate inequalities or declassify journals through the evaluation strata fixed, on behalf of preserving the quality of national scientific production and scientific vehicles, increasingly internationalized. Although it is possible to recognize that Qualis has been one of the elements responsible for the improvement of national journals, the competitive climate makes it difficult for the competencies of the editorial team to be totally focused on the selection and circulation of relevant scientific knowledge produced in the academy or in laboratories and research centers.

Taking the Education Area as a reference, several authors have addressed the Qualis production policy by CAPES, focusing on different aspects: the impacts on the editorial management process of journals (SANTOS CRUZ, 2020; PONCE et al., 2017); the most appropriate indicators for the evaluation metric (SANTOS CRUZ; SANTOS; BIZELLI, 2019; SOUZA et al, 2018); the effects of using an internationalized impact factor on the graduate system (BIZELLI, 2017); the evaluation process of journals in specific strata and determined periods (RIBEIRO; ALENCAR; COUTO, 2016); the contours and limits of the Qualis process (BARRADAS BARATA, 2016).

The present article, however, traces a comparison between the evaluative processes that go from 2010 to 2018 - that is, it comprises the triennium 2010-2012; the quadrennium 2013-2016 and the mid-term evaluation that happened in 2019 -, going through indications that point to the reason for the changes made in the classification criteria and their consequences for the journals. Thus, it seeks to understand the trend in internationalizing Brazilian journals to consolidate a scientific policy model, as Frigeri and Monteiro (2014) point out.

According to Gil (2002, 2008), it is possible to say that the research is explanatory, i.e., it seeks to identify the reasons for the occurrence of facts; descriptive in nature; and uses documentary sources of information: CAPES official documents (MARCONI; LAKATOS, 2002). The purpose, therefore, is to contribute to the understanding, strengthening and improvement of the evaluation system of academic production in Brazil, towards the improvements that may be necessary, from the discursive perspective built by this text.

Why do journals need to be evaluated?

Between December 2019 and November 2020, the Brazilian scientific production exceeded 275 thousand published articles (NATURE INDEX, 2021). The numbers aggregate an expressive set of institutions such as universities, research centers, with or without support from funding agencies, scattered in thousands of scientific journals that publish the built knowledge.

This entire universe is subject to rules that measure the regularity of the vehicles of publication; the standardization of articles; the verification of the originality of the production, among other procedures. Therefore, it is necessary to evaluate in order to guide public policies to encourage the academic sector. However, evaluating the production of knowledge can be abstract and difficult to execute (YAMAMOTO, 2001); or, still, evaluate provides different horizons: it can predict punishment for strategic actions deployed in the past; but it can also promote positive reflections on how to act in the future, offering stability to the evaluation system itself, as long as it is visible2 and allow the necessary corrections for adjustments and improvements (PONCE et al. , 2017).

The guarantee that a published work contributes to the construction of scientific knowledge, that it follows recognized research methods in its execution and that it is presented within the national or international standards of standardization, is given by the reliability and prestige that the journal achieves in the evaluation system, giving back to the authors recognition for their findings. It is essential that the system has rigorous criteria to assess the quality of the scientific vehicle, although the evaluation metrics are historically in constant improvement.

In Brazil, the journal evaluation system is the responsibility of CAPES, a foundation of the Ministry of Education (MEC), which has among its institutional roles to evaluate, expand, and consolidate the stricto sensu postgraduate. It is part of CAPES attributions:

  • evaluation of stricto sensu postgraduate;

  • access to and dissemination of scientific production;

  • investments in the training of high-level human resources in the country and abroad;

  • promotion of international scientific cooperation;

  • induction and promotion of initial and continuing education of teachers for basic education in face-to-face and distance formats (BRASIL, 2020, our translation).

The Qualis-Journals, therefore, is one of the evaluation instruments of the National Post- Postgraduate System (AZEVEDO; OLIVEIRA; CATANI, 2016), since it is the main gauge of the production of professors who belong to the Postgraduate faculty (CAPES, 2021). By principle, the Qualis measures the quality of scientific articles, as it was said, by the prestige of the vehicle of disclosure used by the authors.

The stratification proposed by the agency was modified within the period of analysis of this article. In the triennium 2010 to 2012 and in the quadrennium 2013 to 2016 the strata were: A1; A2; B1; B2; B3; B4; B5 and C, and in the mid-term evaluation (2019) the same classification was used that will guide the period 2017 to 2020 - to be published in 2021 -, thus established: A1; A2; A3; A4; B1; B2; B3; B4 and C. In fact, the A1; A2 and B1 strata of the former Qualis were transformed into A1; A2; A3 and A4, while the other strata became B1; B2; B3; B4 and C, as will be percentually explained below.

Another interesting change to be noted about the periods cited - 2010 to 2016 and 2017 onwards - was about the sample classified in the areas. It is part of the sample analyzed by the committee of each knowledge area all the journals cited by the Postgraduate professors of that area, in the period evaluated3. Until 2016, the journals had different Qualis in different areas of knowledge. After that, the concept of parent area was created, i.e., the journals would be analyzed in the area where they were most cited - this would be the mother area of the journal - and this classification - Qualis-unique - would be valid for all other areas4.

It is important to reaffirm the need for quality indicators to be clear and stable: in the case of Education, the more qualitative - and not quantitative - they expand the possibility of dialogue with the Evaluation System, facing issues such as statistical locks (SANTOS CRUZ, 2020). The results of the evaluation of a given period do not work only as an indicator of prestige of a vehicle but transfer prestige to authors and articles published in journals of higher Qualis5 (SILVA, 2009). According to Frigeri and Monteiro (2009, p. 311, our translation), the Qualis is increasingly used as an indicator:

[...] which assists in the granting of funding, in the inclusion of titles in libraries and indexers, in guiding researchers and readers during the choice of titles, in the submission of works and in the search for bibliographic material, besides stimulating publishers to raise the quality standard of their journals.

Another point to be remembered is that all Qualis requirements influence the management processes of the journals, both in their production practices, as in the quality of publication, or even in the search for meeting the demands of indexing, instilling in editors’ standards to keep the journal in prominence in the area. Besides that, the maintenance of the criteria of the evaluation system of CAPES is unstable, which reduces the collaboration among editors, because it fosters a competitive environment (BIZELLI; SANTOS CRUZ, 2016; PONCE et al., 2017).

In Brazil, however, scientific journals do not have the same working conditions and especially do not have the same budgets: Funding Agencies require, for example, that the journal be at the top of the ranking to receive resources, strengthening the strongest and discouraging the weakest that need to grow. The opening of a permanent dialogue with the agencies could create forms of support so that the editorial teams do not feel so prejudiced in the evaluation processes.

CAPES, in turn, minimizes the role of Qualis in the Brazilian scientific evaluation framework, emphasizing its origin and function. It clarifies, therefore, that Qualis is not, for example, an indexer - although many publishers want to see their journals “indexed” in the evaluative instrument -, it is not a bibliometric base, but it concerns the teaching production in the professors who belong to the Postgraduate courses. Thus, it is not the purpose of Qualis to classify journals or attest their quality and much less to be used outside the scope set by the Brazilian Postgraduate Evaluation System.

As Barradas Barata (2016) notes, the submission of articles in journals considers aspects related to the journal, in addition to its Qualis stratum, such as: target audience, credibility and scope. The author reiterates that the Qualis was not developed to assess the individual scientific performance of authors - it does not have the capacity for such -, since it was designed to evaluate and analyze the collective production of a Postgraduate program. It also points out that the measuring instrument is not very sensitive to journals that change their indicators in an accelerated manner, since the period of use is dated (three or four years), fitting corrections only for the next evaluation.

Hence the volume of complaints about the lack of dialogue between CAPES and editors, who usually seek to obtain a better classification to positively increase their ability to reach sources of funding. It is important to emphasize that the Brazilian Funding Agencies are the first to encourage this competitive behavior by demanding, as already mentioned, that the journals awarded by their edicts are at the top of Qualis.

Analyzing the data

According to the objectives outlined by the research, we set as object of analysis the Qualis-Periódicos, in Education, working with the constant transformations undergone by the indicator towards reducing the differences between the criteria of the various areas of evaluation and towards the internationalization of national scientific production. Methodologically, it was about explaining and describing the phenomena that involved the object of study, identifying determining factors and assumptions (GIL, 2002), with comparative research instruments, evaluating similarities and dissimilarities (GIL, 2008). The main source of data was the official CAPES documentation (MARCONI; LAKATOS, 2002).

The data refer to the evaluative periods of the triennium 2010-2012, moment of a tendency to mild internationalization; the quadrennium 2013-2016, when internationalization is sought with greater rigor; and the mid-term evaluation (2017-2018), when internationalizing starts to mean finding an impact index. Once collected, the results of the application of the criteria were transformed into graphs to facilitate the analyses.

The entire discussion observed in the triennium 2010 to 2012 results in a set of guidelines that seeks to move away from the determination of an impact factor6 that could, as a single criterion, guide the evaluation of scientific production published in journals. There was an orientation towards internationalization, but that was materialized in demanding from the journals an internationalized editorial board, referees who were affiliated with international institutions and a percentage of articles with foreign authors. As there was already a movement for institutional diversity of the vehicles of scientific dissemination, it is possible to perceive movements of cooperation with internationalized groups.

Although there was already an incentive to index the journals, from the quadrennium 2013-2016, the Qualis upper strata began to require numerical quantities of certain indexers:

Web of Science; Scopus; Scielo BR; Educ@; Social Sciences Citation Index; Redalyc; DOAJ; IRESIE; BBE; Latindex and Clase. In the mid-term evaluation (2017-2018), the criterion of number of indexers remains, but for the A1 stratum they become more restricted: Web of Science; Scopus and Scielo BR. At the same time, an impact indicator is used for the first time: the h5 of Google Scholar.

Alongside the stricter indicators, good publishing practices began to be valued in the classification of the triennium 2013-2016, such as: articles registered in the DOI (Digital Object Identifier) system, composed of a series of numbers and letters that identify and legitimize the publication; the mode of ahead of print publication, which gives agility in the publication and increases the exposure time of the article for citations, allowing the approved material to gain dissemination and subsequently accompany an issue published in issue; editions with low endogeneity and high proportion of articles in foreign language or, more specifically, English. As for the mid-term classification (2017-2018), the issue of good editorial practices was deepened - graphic quality, transparency in editorial management and in the historical series of the journal - and began to require the identification Orcid (Open Researcher and Contributor ID) of the authors.

The evaluation committee of educational journals of the 2013-2016 quadrennium (CAPES, 2017) noted that there was inclusion of indexers in the stratification criteria because in the previous classification A1 or A2 journals needed to be indexed in Scielo, a measure considered very restrictive. In fact, in the triennium 2010-2012 there was no requirement regarding specific indexers. Regarding the other criteria, the 2013 and 2017 evaluation remained unchanged. Until 2016, the journals were classified based on the mandatory locks that determined that 25% should be A1 + A2, where A1 should be less than A2; 25% should be B1 and 50% were directed to the strata: B2, B3, B4 and B5 (CAPES, 2021). For the 2017-2020 classification - and for the analysis made in 2019, considering the period 2017 and 2018 - there is a reorganization of the strata, as we said, and it is proposed that A1 < A2; A1 + A2 < A3 + A4; A1 + A2 + A3 + A4 ≤ 50%, thus, the remaining journals would be in the strata B1, B2, B3 and B4. Those that were not considered scientific journals would be in stratum C (CAPES, 2019).

From all these changes - whether in indicator criteria or in the most recent structuring standards: Qualis reference for each parent area and new stratification - it is possible to visualize and compare the impact of this change in the classification of journals in Table 1.

Table 1: Comparison of the quantity of journals evaluated by Qualis-Periódicos Education 2010-2012, 2013-2016 and 2017-2018. 

Strata Qualis-Periódicos Education 2010-2012 (a) Qualis-Periódicos Education 2013- 2016 (b) Qualis-Periódicos Education 2017- 2018 (c) Difference between a and b Difference between b and c
A1 115 121 37 + 6 journals - 84 journals
A2 170 380 39 + 210 journals - 341 journals
A3 Did not have Did not have 47 + 47 journals
A4 Did not have Did not have 65 65+ journals
B1 322 542 74 + 223 journals - 468 journals
B2 378 425 89 + 47 journals - 336 journals
B3 390 357 62 + 33 journals - 295 journals
B4 455 307 115 + 148 journals - 192 journals
B5 485 782 Does not have + 297 journals - 782 journals
Total 2315 2914 528 + 599 journals - 2386 journals

Source: Prepared by the authors from Qualis-Periódicos Education Reports 2013, 2017 and 2019

It is possible to observe that many journals achieved better classification during the quadrennium 2013-2016, even with more objective requirements, with criteria that became more specific. However, many journals disappeared from the evaluated set in Education in the middle term (2019): with the idea of the parent area, they were evaluated in other areas of knowledge. In this sense, Chart 1 illustrates the impact of the measure for journals classified in A1 and A2, strata that identify the best journals throughout the period 2010 to 2018.

Source: Prepared by the authors from Qualis-Periódicos Education Reports 2013, 2017 and 2019

Graph 1:  Comparison between A1 and A2 strata in the classification of the three-year period 2010-2012; four-year period 2013-2016 and mid-term period 2017-2018 

It is worth remembering that in the mid-term evaluation (2017 and 2018), the h-index (PUCRS, 2019) of Google Scholar was used for the area of Education as a measure of impact or citation. The choice fell on this indicator since the national journals of humanities have low presence in the Scopus and Web of Science bases. Thus, for the evaluation carried out in 2019 it was accepted the calculation of impact by h 5 of Google Scholar, while it is discussed in the College of Humanities a standard method of evaluation for the end of the current quadrennium (2017-2020), approximating methodology used by all areas (CAPES, 2021).

For the final evaluation of the four-year period 2017 to 2020, there will also be a linguistic cut for the use of the h index: the predominant language of the journal starts to separate the national journals from the others7, i.e., it meets the difference, for example, between a journal predominantly consisting of articles in Portuguese - which must meet the h 5 - and another consisting of articles in English - which must meet the Impact Factor. The h 5 of national journals are now corrected by Harzing's Publish or Perish software. It was also determined that journals with h index equal to zero would be classified in C, and the other strata were divided into eight groups with an average of 12.5% per group, in which the highest was the one with the highest h index (CAPES, 2021).

In short, in the evaluation proposed for the quadrennium 2017 to 2020, besides the journals being evaluated by their parent or sister areas, they will be classified according to their impact, by the language in which they publish, reducing considerably the number of journals evaluated and, therefore, reducing the number of journals in the A1 and A2 strata, creating high expectations about the results of Qualis 2021.

Conclusion

For all that has been said, it is configured the importance of Qualis-Periódicos for the evaluation of the teaching production linked to the programs of stricto sensu postgraduate, in Brazil. Even assuming many functions for which it was not created, the instrument remains influencing the direction of scientific research and interfering in the whole publishing field, since it induces practices of how the quality scientific knowledge should be published (FRIGERI; MONTEIRO, 2014).

Thus, the comparative analysis of the data on the rankings made in the three periods - in the three-year period 2010-2012, in the four-year period 2013-2016 and in the mid-term evaluation (2017 and 2018) - shows that changes in criteria between the first and the second period made the ranking more objective, facilitating the stratification of journals and improving the relative position of many of them. Differently, the transition from 2013-2016 to 2017-2018 was carried out with a new differentiator - an impact meter - that although it was not used as the sole factor, created conditions to conduct the next evaluation (2021).

It remains to evaluate the effects of the change, precisely in 2021. The h index is here to stay, and its effects will be materialized, while editorial teams begin to worry about the possibility of dialogue after the classification and postgraduate coordinators begin to worry about the effects of the evaluation of faculty production in the result of the evaluation of programs. The entire scientific community, however, looks with concern to the Qualis score, since it guides choices in other agencies, such as when a teacher asks for a Research Grant to CNPq.

Although it is possible to think that evaluation parameters such as Qualis-Periódicos are recent - the historical series assembled for this article counts nine years - and that the criteria are still in an incremental phase, it is not so: on the one hand, 2021 will bring a rupture, a new gauge, abandoning the incremental trend; on the other hand, it is perceived a movement within the evaluation sectors for the extinction of Qualis. It is necessary to understand and reflect more on the system of evaluation of journals and on the indicators that value the teaching production in the country, pushing aside the interests of interest groups or individuals who wish to guide Brazilian research.

REFERENCES

AZEVEDO, Mário Luiz Neves; OLIVEIRA, João Ferreira de; CATANI, Afrânio Mendes. O Sistema Nacional de Pós-graduação (SNPG) e o Plano Nacional de Educação (PNE 2014-2024): regulação, avaliação e financiamento. Revista Brasileira de Política e Administração da Educação, v. 32, p. 783-803, 2016. Disponível em: Disponível em: https://seer.ufrgs.br/rbpae/article/view/68576 . Acesso em: 01 abr. 2021. [ Links ]

BARRADAS BARATA, Rita de Cássia. Dez coisas que você deveria saber sobre o Qualis. Revista Brasileira de Pós-Graduação, v. 13, n. 30, 22 dez. 2016. Disponível em: Disponível em: http://ojs.rbpg.capes.gov.br/index.php/rbpg/article/view/947 . Acesso em: 02 abr. 2021. [ Links ]

BIZELLI, José Luís; SANTOS CRUZ, José Anderson. Educación e Internacionalización: competencia o colaboración? In: MARTIN BRIS, Mario (org.). Internacionalización de la educación superior en Iberoamérica: miradas y perspectivas. Alcalá de Henares: Servicio de publicaciones de la Universidad de Alcalá de Henares. 2016. v. 01, p. 77-90. [ Links ]

BIZELLI, José Luís. Internacionalización: reflexiones a partir de la experiencia de un programa de postgrado en Educación Escolar brasileña. In: MARTÍN BRIS, Mario; JABONERO BLANCO, Mariano. (org.) Internacionalización de la educación en Iberoamérica: reflexiones y proyecciones. Madrid: Ed Santillana/UAH, 2017. p. 35-41. [ Links ]

BIZELLI, José Luís; SANTOS CRUZ, José Anderson. Indexadores internacionais para medir a produção docente na área de educação. In: LAMOTHE ZAVALETA, Carlos; ÁLVAREZ, María Cristina; MARTÍN BRIS, Mario. (org.). Práctica docente: experiencias, reflexiones y propuestas desde Iberoamérica. Veracruz: Universidad Veracruzana/México Academia Journals, 2019. v. 1, p. 137-148. [ Links ]

BLOG: O que é DOI? Periódicos de Minas, 2018a. Disponível em: Disponível em: https://www.periodicosdeminas.ufmg.br/o-que-e-doi/ . Acesso em: 03 abr. 2021. [ Links ]

BLOG: Entenda a modalidade de publicação Ahead of Print (AOP). Periódicos de Minas, 2018b. Disponível em: Disponível em: https://www.periodicosdeminas.ufmg.br/entenda-a-modalidade-de-publicacao-ahead-of-print-aop/ . Acesso em: 03 abr. 2021. [ Links ]

BLOG: O que é Orcid? Periódicos de Minas, 2018c. Disponível em: Disponível em: https://www.periodicosdeminas.ufmg.br/o-que-e-orcid/ . Acesso em: 03 abr. 2021. [ Links ]

BRASIL. Ministério da Educação. CAPES: História e Missão. Brasília: MEC, 2020. Disponível em: Disponível em: https://www.gov.br/capes/pt-br/acesso-a-informacao/institucional/historia-e-missao . Acesso em: 01 abr. 2021. [ Links ]

BRASIL. Ministério da Educação. CAPES: Sucupira. Brasília: MEC , 2021. Disponível em: Disponível em: https://sucupira.capes.gov.br/sucupira/ . Acesso em: 01 abr. 2021. [ Links ]

NATURE INDEX. 2021. Disponível em: Disponível em: https://www.natureindex.com/country-outputs/Brazil . Acesso em: 29 mar. 2021. [ Links ]

CAPES. Ministério da Educação. Coordenação da Área de Educação na CAPES. Relatório de Avaliação 2010-2012 Trienal 2013: Educação. Brasília, MEC, 2013. 48 p. [ Links ]

CAPES. Ministério da Educação. Coordenação da Área de Educação na CAPES. Relatório da Avaliação Quadrienal 2017: Educação. Brasília: MEC , 2017. 93 p. [ Links ]

CAPES. Ministério da Educação. Coordenação da Área de Educação na CAPES. Relatório do Qualis Periódicos: Área 38, Educação. Brasília: MEC , 2019. 11 p. [ Links ]

CAPES. Ministério da Educação. Coordenação da Área de Educação na CAPES. Notas sobre o qualis periódicos: área de educação. Brasília: MEC , 2021. Disponível em: Disponível em: https://sbhe.org.br/uploads/information_files/8/56a8d7de9cf0ec0f9ce831af2d11ba3d.pdf . Acesso em: 01 abr. 2021. [ Links ]

CARIBÉ, Rita de Cássia do Vale. Comunicação científica: reflexões sobre o conceito. Informação & Sociedade: estudos, João Pessoa, v. 25, n. 3, p. 89-104, 28 dez. 2015. [ Links ]

SANTOS CRUZ, José Anderson; SANTOS, Gildenir Carolino; BIZELLI, José Luís. Fatores de qualificação e boas práticas nos periódicos brasileiros em educação: indexação versus fator de impacto. In: SANTOS, Gildenir Carolino; MARTINS, Valéria dos Santos Gouveia. (org.). Ciência aberta, sistemas e ambientes de informação: do acesso às boas práticas de pesquisa. Campinas: Unicamp, 2019. v. 1, p. 239-2631. [ Links ]

SANTOS CRUZ, José Anderson. Gestão do conhecimento e gestão editorial: qualificadores da avaliação de periódicos da área de educação. 2020. 282 f. Tese (Doutorado em Educação Escolar) - Faculdade de Ciências e Letras, Universidade Estadual Paulista, Araraquara, SP, 2020. [ Links ]

FRIGERI, Mônica; MONTEIRO, Marko Synésio Alves. Qualis periódicos: indicador da política científica no Brasil? Estudos de Sociologia, v. 19, n. 37, p. 299-315, 2014. Disponível em: Disponível em: https://periodicos.fclar.unesp.br/estudos/article/view/6266 . Acesso em: 31 mar. 2021. [ Links ]

GIL, Antônio Carlos. Como elaborar projetos de pesquisa. 4. ed. São Paulo: Atlas, 2002. [ Links ]

GIL, Antônio Carlos. Métodos e técnicas de pesquisa social. 6. ed. São Paulo: Atlas , 2008. [ Links ]

MARCONI, Marina de Andrade; LAKATOS, Eva Maria. Técnicas de pesquisa: planejamento e execução de pesquisas, amostragens e técnicas de pesquisas, elaboração, análise e interpretação de dados. 5. ed. São Paulo: Atlas , 2002. [ Links ]

MONTEIRO, Adriano Lisboa; FURLAN, Maysa; SUAREZ, Paulo Anselmo Ziani. Sistema nacional de pós-graduação e a área de química na CAPES. Quím. Nova, v. 40, n. 6, p. 618-625, jul. 2017. Disponível em: Disponível em: http://www.scielo.br/scielo.php?cript=sci_arttext&pid=S0100-40422017000600618&lng=en&nrm=iso . Acesso em: 28 mar. 2021. [ Links ]

PEREZ, Olivia Cristina. O novo Qualis Periódicos: Possíveis diretrizes, Impactos e resistências. Novos Debates - Fórum de Debates em Antropologia, v. 6, p. 1-8, 2020. Disponível em: Disponível em: http://novosdebates.abant.org.br/e6212/ . Acesso em: 03 abr. 2021. [ Links ]

PONCE, Branca Jurema et al. Sobre a melhoria da produção e da avaliação de periódicos científicos no Brasil. Ensaio: aval.pol.públ.Educ., Rio de Janeiro, v. 25, n. 97, p. 1032-1044, dec. 2017. Disponível em: Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-40362017000401032&lng=en&nrm=iso . Acesso em: 31 mar. 2021. [ Links ]

PUCRS. Índice h. Porto Alegre, 05 jul. 2019. Disponível em: Disponível em: https://biblioteca.pucrs.br/apoio-a-pesquisa/bibliometria/indice-h/ . Acesso em: 03 abr. 2021. [ Links ]

RIBEIRO, Daniele Alves; ALENCAR, Maria do Socorro Nunes Macedo; COUTO, Euclides de Freitas. Critérios Qualis A1 da Educação: a avaliação sob a perspectiva da disputa acadêmica. Linhas Críticas (UnB), v. 22, n. 47, p. 229-248, 2016. Disponível em: Disponível em: https://periodicos.unb.br/index.php/linhascriticas/article/view/4816 . Acesso em: 28 mar. 2021. [ Links ]

SILVA, Antonio Ozaí da. Sua revista tem Qualis. Revista Mediações (UEL), v. 14, n. 1, p. 117-124, 2009. Disponível em: Disponível em: http://www.uel.br/revistas/uel/index.php/mediacoes/article/view/3350 . Acesso em: 01 abr. 2021. [ Links ]

SOUZA, Ângelo Ricardo de. et al. Qualis: a construção de um indicador para os periódicos na área da Educação. Práxis Educativa, v. 13, n. 1, p. 219-231, 2018. Disponível em: Disponível em: https://www.researchgate.net/publication/324839153_Qualis_a_construcao_de_um_indicador_para_os_periodicos_na_area_da_Educacao . Acesso em: 28 mar. 2021. [ Links ]

YAMAMOTO, Oswaldo Hajime. Vale a pena avaliar periódicos científicos? Estud. psicol. (Natal), Natal, v. 6, n. 2, p. 129-131, 2001. Disponível em: Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-294X2001000200001&lng=en&nrm=iso . Acesso em: 31 mar. 2021 [ Links ]

1Coordination for the Improvement of Higher Education Personnel.

2The concept of visibility replaces that of transparency, since the evaluation structures must be amenable to analysis, that is, concretely they must be “visible” to the eyes of the evaluated.

3A distortion arising from this fact is that the Qualis of an evaluative period is only known a posteriori.

4The number of journals analyzed by area has decreased intensely.

5The most glaring distortion generated is that better journals end up receiving better and larger flows of articles, and the worst evaluated are not sought by authors. A vicious circle is formed.

6It is necessary to distinguish the search for an impact indicator from the use of the impact factor, a term that identifies international measures linked to the Scopus and Web of Science indexers.

7We tried to avoid that national journal cease to be attractive to researchers, compromising the Brazilian research agenda, turning it to international interests and excluding local issues (PEREZ, 2020).

Received: August 25, 2021; Accepted: January 31, 2022

Laís Zaccaro Sene. Mestranda, Faculdade de Ciências e Letras (FCLAr/UNESP) - Brasil; Programa de Pós-Graduação em Educação Escolar. Lattes: http://lattes.cnpq.br/1359753390714803.

José Luís Bizelli. Livre docente, Faculdade de Ciências e Letras (FCLAr/UNESP) - Brasil; Programa de Pós-Graduação em Educação Escolar. Pesquisador Bolsista de Produtividade em Pesquisa 2 - CNPq. Coordenador Nacional FEPAE. Lattes: http://lattes.cnpq.br/3751287338655685.

Creative Commons License Este é um artigo publicado em acesso aberto sob uma licença Creative Commons