Introduction

Despite the plentiful criticism in academic literature (Altbach; Hazelkorn, 2017) regarding the methods adopted by academic rankings and their inability to measure the quality of teaching and learning or the quality of the student experience, ^{Altbach (2006)}, released in 2003, remains current and still resonates when stating that rankings are in vogue all over the world, achieving public legitimacy and an aura of credibility, as respectable research and political organizations use them to survey data and benefit from them.

From this perspective, higher education assessment instruments provoked fascination in the daily life of the educational community searching for a certification of excellence (^{Théry, 2010}; ^{Dametto; Esquinsani, 2015}; ^{Hazelkorn, 2011}; ²⁰¹⁹). Its power of attraction involves government officials, academic managers, school directors, teachers and extends to college students, parents, marketers, among other actors in the expansion of this managerial context in evaluating the quality of higher education (^{Post et al., 2013}), as well as in the battle for excellence (^{Hazelkorn, 2011}) in a global education market based on university prestige.

Diverse definitions of academic rankings could be reached, depending on the theoretical perspective adopted or the place occupied in the educational scenario, public policies and the market. They can be idyllically exalted or demonized, subject to the lens through which they are approached. In this article, academic rankings are considered as a set of league tables with a quantitative, empirically rational approach, which, based on the combination of multiple indicators, weights and metrics, with their potential and limits, contribute to the assessment of the quality of Higher Education Institutions (HEIs) and higher education systems, at national or international levels, serving as a seal of excellence and quality, with the World Class University (WCU) seal being the most valuable and desired in the knowledge economy (Calderón; França, 2018; ^{Calderón, 2022}).

In this approach, academic rankings “cannot and should not be considered as the only indicators or instruments of institutional evaluation” (Calderon, 2022). The so-called “Berlin Principles on Rankings of Higher Education Institutions”, drawn up in 2006 by the International Ranking Expert Group (IREG), explicitly state that rankings and league tables should:

be one of a number of diverse approaches to the assessment of higher education inputs, processes, and outputs. Rankings can provide comparative information and improved understanding of higher education, but should not be the main method for assessing what higher education is and does. Rankings provide a market-based perspective that can complement the work of government, accrediting authorities, and independent review agencies (^{Ireg, 2006}, p.1).

As can be seen, the approach adopted in this article distances itself from theoretical-critical approaches, framed in the conflict paradigm (^{Sander, 1984}), which overvalue the negative aspects of rankings as a result of neoliberal strategies, within the scope of governance policies in higher education, which are still hegemonic in Brazilian academic literature, mainly in Education Sciences (^{Calderón; Franca, 2018}).

In technical terms, as a field of study these assessment instruments encompass different types of league tables, which can take the form of rankings or ratings, the same ones that are used, in multidisciplinary terms, both in the academic-scientific scope for the production of knowledge (^{PINHO, 2023}) and within the scope of market and platform economy (^{GARCIA RUIZ, 2023}), key players in the emergence of the metric society, that is, “a society of scores, ratings, likes, stars and grades”, in which there are “data and indicator-based methods of evaluation and monitoring which are encouraging a wholesale quantification of the social sphere” (^{Mau, 2019}, p. 2). In fact, it is precisely in this society where the practices of measurement, evaluation and comparison are expanding, amidst the “popularization of concepts such as transparency, accountability and evidence-basing”, in which “ratings, rankings and quantitative forms of evaluation play a central role.” (^{Mau, 2019}, p. 3).

In the rankings, HEIs or courses are classified hierarchically, from best to worst performance, remaining in a specific position, with reduced possibility of a tie. For example, the 2019 edition of Ranking Universitário Folha (RUF) ranked 197 Brazilian universities, distributed from 1st to 197th place. In the ratings, HEIs or courses are classified into quality segments/ranges, and it is not possible to distinguish, within the group, whether one is better than another, since they are in the same range. For example, Guia do Estudante (GE), issued in Brazil from 1986 to 2018, classified undergraduate courses into three quality ranges: a) excellent (five stars), b) very good (four stars) and c) good (three stars). It is worth noting that there are league tables that mix elements of rankings and ratings in their designs; for example, QS World University Rankings classifies the first 500 best universities by ranking and groups the others after this position into the form of rating, using performance ranges. Furthermore, it is noteworthy that ratings that classify courses and universities using stars, with five stars as a synonym for excellence, is a common practice in the global knowledge economy, such as the national ratings promoted by The Good Universities Guide on the quality of university courses in Australia or, internationally, QS Stars University Ratings, which allows you to find the best universities based on specific areas of knowledge.

Studies show that academic rankings in Brazil have emerged as a multidisciplinary field of study, through an embryonic and expanding scientific literature produced by researchers from different areas of knowledge concerned with understanding how these instruments of external evaluation operate (^{Calderón et al., 2015}). In this field, Educational Sciences stands out due to its minimal scientific production, a fact that signals the reduced interest around the subject, marked by a theoretical-critical literature of essayistic nature, which predominantly questions epistemological aspects referring to a conception of education as a commodity, which is supported by the culture of performativity (^{Ball, 2005}; ^{Dametto; Esquinsani, 2015}) and evaluation through ranking (^{Calderón et al., 2015}).

A survey carried out by Calderón and França (2018, p. 462) on the state of the art of Ibero-American scientific production reveals that, after a decade of studies characterized by criticism of the emergence of national and international rankings, from 2010 onwards “conceptual theoretical studies that are more aligned with the consensus paradigm prevail and tend to see the ranking phenomenon as a factor that induces the improvement of higher education institutions and systems”. Parallel to critical theoretical studies, aligned with the conflict paradigm, in which rankings are approached through the prism of the commodification and privatization of higher education, studies of an empirical nature are emerging concerned with understanding more objectively,

[...] aspects related to the methodology used to construct the indicators, to carry out comparative studies between different rankings and/or on the performance of different universities in a country based on certain rankings (Calderón; França, 2018, p. 462, translated by the authors).

In this sense, academic rankings as a field of study are strengthened by the existence of an Ibero-American scientific production that reveals the “trend towards the study of institutional and country performativity, understanding its limitations, but also its potential” (CALDERÓN; FRANÇA, 2018, p. 462, translated by the authors).

This article is included in this last field of studies, specifically characterized by concern in analytical, descriptive and comparative terms and by understanding indicators, methodologies and performativity of HEIs linked to rankings, with support and funding from the National Council for Scientific and Technological Development (CNPq), thus contributing to the incipient literature on the trajectory of academic rankings.

Therefore, this article aims to understand the trajectory of Guia do Estudante (GE), from Editora Abril, one of the first league tables in Brazil, which operated in the form of course ratings and rankings of higher education institutions (HEIs) for thirty-two years (1986-2018).

To understand the importance of GE in the study of academic rankings, it must be considered that there were two pioneering league tables in Brazil: As melhores Faculdades do Brasil (The Best Colleges in Brazil), promoted by Playboy Magazine, issued between 1982-2000 (^{Calderón et al., 2014}) and the GE rating created in 1984, which, from 1986 onwards, began classifying the best courses and the best HEIs in the country. Before 1986, the now-defunct Playboy Magazine ranking was the only one on the educational scene in the first half of the 1980s, and started coexisting with the ratings produced by GE after the second half and throughout the 1990s.

Until then, these were the two references on the quality of higher education produced in the publishing market, to which were added, from 1996 onwards, the Exame Nacional de Cursos (ENC) (National Course Exam), the so-called Provão, the official ranking of the federal government (Cechin, 2011; ^{Calderón, et al., 2014}). Within the scope of the league tables produced by the private sector, the extinction of the Playboy magazine ranking in 2000 made GE classifications the only reference for university ranking throughout the first decade of the 21st century; since 2012, it started to coexist with the RUF (^{Calderón; Lourenço, 2014}) until 2018, the year of its abrupt extinction.

This article focuses on this trajectory of the GE, whose prestige and recognition is registered and publicized on the websites of public and private HEIs (for-profit or non-profit), as well as in newspapers and advertising materials that flood the publishing market. The University of São Paulo (USP), for example, announced on its official page: “USP is the best public university according to Guia do Estudante” (^{USP, 2016}, translated by the authors). The Pontifical Catholic University of Campinas published on its portal: “PUC-Campinas is elected by Guia do Estudante as the best private university in Health” (PUC-Campinas, 2006, translated by the authors). In turn, Grupo ^{Anhanguera Educacional (2013}, translated by the authors), a pioneer for stock trading, announced: “Anhanguera has 93 courses starred in the 2013 issue of Guia do Estudante”.

Although the GE is predominantly a rating, when classifying courses by segments with a certain number of stars, it is worth highlighting that it also offered classifications in the form of rankings, for example, the 'Best Universities' Award, in which it ranked the 20 best private HEIs and the Top 20 public ones. Throughout its editions, the GE proclaimed itself as a ranking and not a rating, because the GE’s target audience was more familiar with the word ranking. It is worth clarifying that the term rating is more linked to the English-speaking world, and its use is not common in Brazil; however, as it is an academic article, its appropriate naming is opportune.

To achieve the objectives of this article, bibliographical and documentary research of an exploratory, qualitative and quantitative nature was carried out, based on four referential dimensions: (i) typology, (ii) functioning, (iii) methodology and (iv) predominant trends. The sources of analysis were twenty-eight editions from the Abril publishing house collection. Studying GE’s trajectory allowed us to identify five defined periods over time, which have specificities in terms of patterns, sequences and regularities, allowing facts, processes and methodologies to be grouped chronologically.

For the purpose of comparative analysis of the ratings and rankings produced by GE over time, the Grid prepared by ^{Calderón, França and Gonçalves (2017)} was taken as a reference, making it possible to identify multiple typifications, as seen in Figure 1, being able to classify them as pure (objective or subjective) and hybrid, with indicators that can be classified as objective or subjective, and focused on inputs or products, with the possibility of combining subjective and objective indicators, as well as the concomitant focus between products and inputs, enabling nine categories that can generate multiple combinations in the construction of academic rankings.

Source: ^{Calderón, França and Gonçalves (2017)}.

Figure 1 Academic Rankings Typology Grid 

Hybrid rankings mix in their composition both objective and subjective indicators, and may have more predominance of one than the other, in addition to the possibility of having a concomitant focus on products and inputs (^{Ganga-Contreras et al., 2020}), thus revealing diversified rankings, given the existence of multiple combinations.

First steps in classifying courses (1986-1990)

It is known that the first publication of GE appeared in 1984 as a supplement to Almanaque Abril (^{Guia do Estudante, 2012}). However, its first league table or academic ranking was prepared in 1986, with information obtained from Playboy Magazine, which at the time published a pioneering national university ranking (^{Guia do Estudante, 2005}). The 1986 and 1987 league tables did not present technical data regarding the number of courses evaluated, methodological procedures and results.

Table 1 shows that it was only in 1988 that GE began to carry out its own evaluations, with only a single type of classification focused on undergraduate courses, through performance standards that translated into a quality scale divided into five bands that in turn translated into stars: one star (poor), two stars (regular), three stars (good), four stars (very good) and five stars (excellent).

In the first evaluation, the concepts weak and regular prevailed in 76.4% of the courses evaluated. According to Guia do Estudante (1988, p. 8), the results gave “a dark color to the horizon of Brazilian students”, as only 17.9% of the courses were evaluated as good, very good and excellent. Over the period, there was a slight drop in courses assessed as weak, however, when looking at the set of courses below average, the numbers continued to exceed 75%.

Regarding the evaluation strategies adopted by Guia do Estudante (1988) since its first editions, on-site interviews were carried out in the evaluation process, at which time a GE reporter went to the HEIs. The first evaluation was carried out through the efforts of a team of ten people, supported by consultants specialized in different areas of education. In this first edition (1988), 2,000 interviews were carried out with teachers, employers, trained professionals and colleges, as well as available information about courses and colleges. The criteria were: teacher qualifications and hiring regime; quality of teaching facilities and resources; extracurricular activities; scholarships and internships; restaurants; accommodation; student satisfaction; employability of undergraduates; and quality graduate courses.

The team was supported by consultants chosen from the academic, scientific and business community (Guia do Estudante, 1989). During the on-site visit, reporters were “instructed to go around the schools incognito and look at all open doors” (^{Guia do Estudante, 1990}, p. 6). They talked to professors, managers and students, investigating the facilities and infrastructure up close, “rotating”.

Classification of courses and HEIs (1991-2001)

Table 2 shows that, from 1991 to 2001, different types of classifications were carried out, with two rankings being prominent and persistent throughout that decade. Compared to the previous period, the ranking of ‘Best Universities’ was the biggest innovation, as, until then, the evaluation was of isolated courses, “regardless of the institution” (^{Guia do Estudante, 1990}, p. 6, translated by the authors). Another innovation was the creation of the ranking of ‘Best Colleges and Isolated Schools, with four and five-star courses’, a reflection of the so-called institutionalization of the higher education market (^{Calderón, 2000}; ^{Polidori, 2011}), a process that involved the opening of numerous HEIs private and higher education courses in Brazil, marking a substantial boom that occurred in the second half of the 1990s.

The ‘Best Universities’ ranking, launched in 1991, announced by GE as an “exclusive assessment”, was published as “the ranking of champion universities”, giving visibility to fourteen HEIs that stood out for their concentration of the highest number of stars (^{Guia do Estudante, 1991}, p. 14, translated by the authors).

Chart 2 shows that, in 1992, the GE started to have eight league tables, maintained the ‘Best Courses’ one, being publicized in the media as the ranking of the 317 best academic courses and maintained the ‘Best Universities’ one, now named ‘Champions of Excellence’, including highlighting the 10 HEIs with the most four and five star courses. In 1993 there was no evaluation. In 1994, there was a reduction in the number of classifications adopted. There is an expansion in the number of ‘Best Courses’ and the resumption of the nomenclature ‘Best Universities’, leaving aside the name Champions of Excellence. The total number of league tables was reduced from eight in 1992 to four in 1994, with the addition of a new classification: ‘courses starred by an administrative nature’, with the ranking of the ‘Best Colleges and Isolated Schools with four and five star courses’.

In 1992, it was the last year in which courses that obtained “weak” and “regular” concepts were exhibited. In 1994, only courses and institutions with “excellent” and “very good” concepts were mentioned. There are no records of the reasons for changing the scale, but, as seen in Table 1, the high number of courses classified as “regular” and “weak” highlighted a reality that overshadowed the objective of publicizing excellent courses.

In the years 1995, 1996 and 1997 there were no evaluations, in the same way as in 1993, and there is no information on the reasons that led to the suspension of evaluations during this period. However, the shutdowns coincided with the emergence of Provão, an evaluation system by the Brazilian Ministry of Education (MEC) created in 1996 to monitor the quality of higher education in the country.

The league tables promoted by GE were carried out again in 1998, presenting an important change in the criteria that defined the concepts of quality: courses and HEIs began to be evaluated in three quality ranges (Guia do Estudante, 1998): three stars (good), four stars (very good) and five stars (excellent). It should be noted that these three star bands remained until the abrupt closure of GE in 2018.

In 1998, the Best Colleges and Isolated Schools classification was resumed after being suspended in 1992, highlighting only four and five star courses, and three rankings were maintained: 1) most starred courses by nature/administrative dependence, 2) the best courses and 3) the Best Universities, highlighting the eleven best universities.

In 1999, the publication reduced the previous year’s four league tables to just two: Best Universities, rewarding the ten best by proportion of starred courses and Best Courses by area of knowledge, rewarding 982 courses in the country. In this edition, there was a 12% increase in the number of courses compared to the previous edition (^{Guia do Estudante, 1999}).

During this period, the number of courses evaluated became increasingly significant and set the pace for the growth of the publication. In 2000, 1,012 courses were awarded five, four or three stars (^{Guia do Estudante, 2000}). The Best Universities ranking was maintained, but published in the headlines under the name ‘Excellence in Universities’, and the classification of starred courses by area of knowledge remained. The novelty was the return of the Best Colleges and Isolated Schools ranking, under the name: Excellence in Higher Education Schools.

In 2001, another change in classifications. Two rankings were also maintained in the 2000 publication, Best Universities and Best Colleges, together with The Best by Region in the Whole Country (Chart 2).

Regarding the strategies and instruments adopted, Chart 3 reveals that in the 1991 edition, an on-site evaluation was carried out for the last time. This change is justified due to the growth in the number of courses and the increasing difficulty of evaluating them, making the practice unfeasible with the emergence of postal questionnaires to HEIs and course coordinators.

In 1992, 6,000 questionnaires were sent to HEIs, the year in which the role of consultants as responsible for completing the evaluation emerged, as they received the data already cross-referenced by a team of journalists and researchers for checking. In this process, “professionals from the most different sectors, professors, scientists, representatives of professional associations and governmental and non-governmental organizations linked to Education” were part of the final stage of the evaluation (^{Guia do Estudante, 1992}, p. 11, translated by the authors).

In the 1994 edition, evaluation through consultants was maintained, as well as the practice of questionnaires sent to HEIs and course coordinators, presenting, as an innovation, consultation with professional entities as an evaluation technique, briefly presented as the hearing of “representatives of class entities and governmental and non-governmental organizations from different regions of the country” (^{Guia do Estudante, 1994}, p. 15, translated by the authors). The consultation with professional entities used in 1994 was only resumed in 2011.

If, in 1998, the means of collecting data, sending questionnaires and using consultants were maintained, in 1999, as an innovative evaluation criterion emerged: the crossing of the performance of data collected from courses and HEIs with the results of 1996 and 1997 from Provão (ENC) (^{Guia do Estudante, 1999}).

The strategy of using consultants and questionnaires sent to HEIs and course coordinators was consolidated in 2000, but the crossing of data with performance in the ENC was suspended and resumed in 2001.

It is worth highlighting that the performance criterion in previous publications has been present since 1991, remaining throughout the entire period analyzed.

Diversification and instability of multiple ratings (2002-2008)

Information about this period was systematized and highlights the types of classification tables adopted. Table 4 refers to the types of classifications that existed from 2002 to 2008. In 2002, the GE had four classifications, without making distinctions based on administrative nature. Over the period, the number of classifications increased, indicating a tendency towards instability and diversity in rankings.

Analyzing the period 2002-2004, it was possible to observe four ways of classifying the best HEIs, as seen in Chart 3. In 2003, the distinction due to administrative nature returned, remaining until the last edition. In contrast, the ranking of best HEIs with the best academic courses by area of knowledge appears only in 2004, an edition that saw an increase in the number of classifications in relation to 2002 and 2003. The last ranking of HEIs with the highest percentage of starred courses by region took place in 2004.

In 2006 there were two editions: the publication of the 2005 and 2006 assessments. It was during the presentation of the 2005 data that, for the first time, the ‘Best Universities’ Award took place, valuing 36 courses of excellence, and rewarding HEIs according to their administrative nature. The award ceremony became a gala event in which certificates with stars were awarded to the best courses and HEIs (^{Portal Aprendiz, 2006}). In the 2005 edition (Chart 4), five classifications adopted in 2004 were maintained and, as new features: the ‘Best Universities’ Award, the best HEIs by number of starred courses by state and the best HEIs by region.

The second edition of 2006 again featured the ‘Best Universities’ Award, with six classifications. In the 2005, 2006, 2007 and 2008 editions, in addition to the ‘Best Universities’ Award, several league tables were published, some presented as innovative, such as the ranking of HEIs with the highest percentage of five-star courses, published only in 2006 and 2007. There was also the classification referring to the average of stars per HEI, published in 2006, 2007 and 2008, with the sum of all stars of the HEI divided by the number of courses evaluated, resulting in the average. In 2006, the ranking of the best university centers, colleges, schools and higher institutes was created, published until 2008, which replaced the so-called Best colleges and isolated schools ranking, in force from 2002 to 2005.

In 2007, the ‘Best Universities’ Award began to highlight the best HEIs, including, for the first time, three categories: in addition to regional prominence, the best universities were awarded in the employability/market insertion and innovation/sustainability categories (^{Guia do Estudante, 2007a}). The difference, also observed in the 2005 and 2006 editions, was the creation of multiple classifications applied to both private and public segments, involving not only universities, but also university centers, isolated colleges and higher institutes.

In 2008, the number of rankings was reduced, indicating the path taken in the following period towards stability and a reduction in the number of classifications. This year, the ranking of HEIs with the highest percentage of five-star courses was abolished.

In terms of strategies and instruments, Chart 5 shows that, in 2002 and 2004, a methodological standard was maintained. In 2002, there were around a thousand starred courses, the contingent of consultants totaled 500 individuals, the printed questionnaires sent reached 9,000, the highest number observed until then. Furthermore, Provão grades were used as an evaluation criterion.

In the 2005 edition there were methodological changes. For the first time, technical assistance from Ibope Opinião was used to develop the research methodology and data processing. Likewise, all data collection began online, using specific forms. However, there were problems, as the online forms sent to the participating courses were not answered by the majority of coordinators, in addition to the presence of a large amount of incorrect information. Faced with the difficulty, the team began to obtain data via telephone, based on the knowledge of professors and coordinators of courses starred in previous editions and professional entities, which became basic sources of evaluation. From this edition onwards, the Provão grade was disregarded, as it ceased to exist with the creation of the National Higher Education Assessment System (SINAES).

In 2006, 2,280 courses were completed and, in this edition, two changes emerged: the first, the return of sending questionnaires to HEIs via the internet; the second, the feedback from the online consultants’ assessment. Some coordinators, however, asked for the return of on-site visits, a practice applied in the past, but considered “impractical, given the more than seven thousand courses to be evaluated in the five regions of the country” (^{Guia do Estudante, 2006}, translated by the authors).

In 2007, of the 8,223 courses evaluated, 2,873 were starred, a significant increase compared to previous editions (^{Guia do Estudante, 2007a}). The questionnaires continued to be sent via the internet, as well as the maintenance of consultants, assistance from Ibope Inteligência and, for the first time, technical consultancy from Price WhaterHouseCooper (PWHC). In this edition, a bonus was applied relating to the stars received by the course in the last four editions of the GE, this was the way found to avoid abrupt variations in the performance of universities and courses from one year to the next. It is noted that a similar bonus, in terms of points, had already been practiced in the period 1991-2001. There was also an update to the database of reviewers, “prioritizing consultants who had a CV registered on the CNPq Lattes Platform and were included in the Bank of Evaluators of the Higher Education Evaluator System (BASIs)” (^{Guia do Estudante, 2007b}, p. 21, translated by the authors).

The evaluation was carried out by consultants with “the distribution of the courses to the referees in each area” electronically and randomly, “the majority of the evaluation was carried out via the internet”, with only a few interviews carried out over the telephone. The next stage was carried out by Ibope Inteligência, responsible for technical consultancy in the collection, systematization and analysis of data. Each course evaluated received a grade, however, only those that received three, four or five stars were published. The assessment result was verified by PWHC (^{Guia do Estudante, 2007b}, p. 21, translated by the authors).

In 2008, of the 9,053 courses evaluated, 3,204 were starred (GE, 2008). In the 2007 and 2008 editions, we can see the strengthening of methodological explanations relating to the evaluation process, opening space for the next period of strengthening the evaluation criteria and stability of the rankings. Editorial changes and greater rigor encouraged the growth of information aimed at the reading public. An editorial change is noted as information is presented about partners, such as Ibope and PWHC, experts in evaluation and education and other rankings, such as US News & World Report Magazine and the newspapers El Mundo and The Times (^{Guia do Estudante, 2008}, p. 14).

Methodological improvement and reduction of rating types (2009-2015)

Chart 6 allows us to verify the continuity and stability of the ‘Best Universities’ Award, present since 2006, with a clear division of the public and private segments. The HEIs were classified into two categories: University of the Year and Best Universities by area of knowledge, in the areas of Administration and Business; Environment and Agricultural Sciences; Health; Social and Human Sciences; Communication and Information; Arts and Design; Exact Sciences and IT; and Engineering and Production.

In 2009, the ‘Best Universities’ Award reached its fifth edition, highlighting the university of the year and the best HEIs by area of knowledge. When announcing the Award, 107 trophies were awarded to 40 HEIs in 15 categories (Guia do Estudante, 2009, p. 16).

In 2010, there was no GE assessment. Therefore, the 20th edition of the GE Assessment did not take place, nor did the 6th ‘Best Universities’ Award, which were postponed for the following year. In the 2011, 2013 and 2014 editions, the classification model established in 2009 was maintained, with no other league table being created, beginning the period of stability and maturity of the rankings promoted by the GE. The exception is 2012 in which three more rankings were published: (1) Best University Centers; (2) Best Colleges, Schools and Higher Institutes; and (3) Best HEIs by region. Even though this exception exists, in 2014, the stability standard was returned, with the sole and exclusive use of the two award categories as a way of classifying HEIs: University of the Year and Best university institutions by area of knowledge.

Chart 7 reveals the stability and standardization in the evaluation strategies and instruments adopted by GE between 2009 and 2012, and it is certain that the formula used to calculate the winners received an adjustment for the 2013 and 2014 editions. The consultants carried out the evaluations mainly online, and the questionnaires sent to course coordinators were completed in the same way, speeding up information processing. After the ENC, government evaluations were no longer used as a performance criterion, as well as the use of the regularity bonus and the presence of Ibope Inteligência and PWHC consultancies. The only variation occurred in the consultation carried out with professional entities in 2011.

Basically, from 2009 to 2012, there were two models to establish and identify the winning HEIs in the two award categories. Regarding the University of the Year category, the courses evaluated were first divided into two groups: public and private. Then, in each group, the winning HEI and the others were identified using a formula, based on “quality (A) and quantity (B) indicators of starred courses”, with A “being the average number of stars of the courses at a given school, that is, it indicates the average quality of the courses” and B corresponds to “how close a given school is to the most starred HEI in the category - which has the largest number of starred courses” (^{Guia do Estudante, 1999}, p. 18, translated by the authors). As for the Best by Area of Knowledge category, it was equally divided into two groups: public and private. Afterwards, the courses were evaluated separately in the areas of knowledge and disregarded the HEIs that did not have at least one course with three, four or five stars. Finally, the same formula as in the previous category was applied.

As the editions went by, the award participation criteria underwent small changes, for example: in 2013 and 2014, in the University of the Year category, schools with at least five evaluated courses participated, a criterion established to avoid distortions in relation to smaller HEIs, with fewer courses. In the Best by Area of Knowledge category, those who had at least two courses with at least three, four or five stars in the area in question participated.

An upward linearity is observed in the evaluations corresponding to the years 2009, 2011, 2012, 2013 and 2014. This is because the number of courses evaluated gradually increased during this period. In 2014, for example, there were almost 12,000 courses in a single edition of the GE. Likewise, the number of starred courses grew year after year, as did the total number of consultants. Data that exposes the dimension GE has reached.

Emphasis on methodology transparency and abrupt closure (2016-2018).

In 2016, GE inaugurated a new phase through three changes to its methodology, aiming to increase its rigor in evaluation and give greater value to the stars awarded as a seal of quality. The first change in Course Assessment occurred in the number of grades given by consultants, which, until 2015, was just one grade per course and, from 2016, it became three grades, one for each of the three indicators that started to compose the evaluated quality criteria: teaching staff, didactic-pedagogical project and infrastructure. The final numerical evaluation of each consultant was based on the average of these three scores.

The second change was the exchange of the bonus that occurred due to the accumulated results of recent years, to the consideration of a weighted average of the average grades of the last three years, which was multiplied by three in the most recent year, by two in the previous year and, finally, by one in the second to last year.

The third change occurred in the grade ranges for defining stars, which, until 2015, were fixed and, from 2016, became mobile, defined based on the standard deviation of the grades awarded for the different courses. These changes reduced the absolute number of starred courses, which fell from 8,737 in 2015 to 6,895 in 2016. In relative terms, the drop was greater, as the total number of courses evaluated grew, thus the percentage of starred courses in relation to the total it was 67.8% in 2015, reducing to 51.3% in 2016 (Table 3).

Another change was in the formula for the ‘Best Universities’ Award, which began to consider, in the calculation basis, the number of stars received by the highest rated HEI in the courses, and no longer a fixed base of stars, which occurred until 2015, which made it possible greater variability in the result each year.

In 2017, an additional 4,100 undergraduate degrees from across the country were included in the evaluation scope, reaching a record of 16,797 courses. One of the effects of this modification was the increase in starred courses, although the relative increase was smaller, six percentage points above 2016, but ten points below 2015. In 2018, maintaining stability in the methodology for data collection, 17,075 courses were evaluated, of which 10,476 received quality stars. These fluctuations in numbers have placed GE assessments in a new phase, pointing towards a regular trajectory with a stabilization of the applied methodology. However, it was precisely in 2018 that GE abruptly ended its activities. In August 2018, Grupo Abril filed for liquidation, putting an end to GE, a publication with a 32-year history of external evaluation of Brazilian courses and HEIs.

By way of conclusion

Regarding the GE's trajectory, the analytical dimensions highlighted allowed us to verify that its structure was adapting and improving over time. The study design, based on its historical process, points to six trends: