INTRODUCTION

The Progress Test (PT) is a type of knowledge assessment applied institutionally to all classes of a course, at the same time. It consists of contextualized objective questions, with content and level of complexity that are adjusted to the student soon to be graduated^1)-(5.

The PT is an important tool for evaluating and controlling the quality of the curriculum teaching-learning activities, allowing the verification of the students’ level of knowledge in each class, especially those who are going to be graduated^1)-(5. It also allows checking, through its successive applications, the progress in the acquisition of knowledge, at the individual level and in the different cohorts^1)-(5. This information helps students and curriculum managers to identify learning strengths and weaknesses in the cognitive domain and also in clinical reasoning, decision-making, diagnostic and therapeutic management, health promotion, protection and rehabilitation domains. The results of the sequential tests help students to perform a self-assessment and monitor the evolution of their performance, contributing to the self-regulation of learning and to the formative purpose of the assessment^1)-(5.

Recently, there has been a growing interest from medical schools in implementing student programmatic assessment (PA) in their courses. PA is an innovative way of practicing student assessment, aiming to better meet its purposes, but strongly favoring the formative scope^6)-(8. PA is carried out using different instruments, following the recommendations of good practices, under centralized coordination and well-defined institutional responsibility⁸.

Many of the pioneering schools in the implementation of the PA were also pioneers in the introduction of the PT^1,3 and use it as an important source to obtain information on student performance in the cognitive domain. The Ribeirão Preto Medical School at the Universidade de São Paulo (FMRP-USP) has invested heavily over the last decade in improving educational assessment in its seven undergraduate courses. This investment took the form of the foundation, in 2014, of its undergraduate education assessment center, which institutionally directs the evaluation of the quality of the disciplines and internships in the courses and student assessment. In 2016, the faculty development center for teaching was founded, which, among its different activities, has those that fall into a specific line of work, aimed at training teachers with a view to the future implementation of PA. In 2018, the goal of introducing PA in undergraduate courses was included in the institution academic project within the scope of the multi-year strategic planning.

The institutional movement aimed at restructuring the curriculum of the FMRP-USP undergraduate medical course, which started in 2020, provided a unique opportunity for changes in student assessment within the principles of PA. In the discussions on this topic, it was proposed to include the PT among the main instruments to be used in obtaining information about the student’s performance. The acceptance of this proposal originated the demand for reflection on advantages, limitations and challenges related to the role of PT in PA.

This article aims to disseminate these reflections, which originated from the discussions involving several teachers, constituting a representative group of the entire institution, with the advice of teachers from a pioneering school both in the introduction of PT¹ and PA⁶ and with the support of international literature.

Characteristics and operation of the Programmatic Assessment

In PA, the articulated and organized use of several instruments, under centralized supervision, predominantly meets the formative purpose of student assessment, but also provides data on their performance in the various domains of learning and in relation to the different skills that the curricular objectives determine^6)-(8. These data can be “triangulated”, providing robust information that support decision-making of summative nature^6),(7. These should privilege remediation and not simply the approval-failure binomial. These decisions are the responsibility of the institutional instance and not only of the curricular units (disciplines, modules and internships)^6)-(8.

The practice of PA contributes to changes in paradigms that rule student assessment. Figure 1 shows the main points of this change, going from the model of assessment of learning towards the assessment for learning, one of the goals of PA^6)-(8.

Source: the authors.

Figure 1 Comparison of the main characteristics of the traditional student assessment (current) and the proposed programmatic approach (future). 

In the more traditional assessment of medical students, exams and tests are more focused on the cognitive domain within the scope of the curricular units, usually at the end of their activities, with the sole purpose of making decisions about passing or failing. Some of these events have a strong impact on the student, being called “high stakes”¹⁴. In PA, on the contrary, more comprehensive assessments should predominate, with greater frequency and continuity, focusing not only on knowledge, but, preferably, on clinical and procedural skills and competences in the affective-behavioral domain^6)-(8. The formative purpose of the assessments should also predominate in PA, aiming to contribute to the student’s learning and development, through good quality feedback, which should involve and engage the student^6)-(8. The information obtained in these assessments also contribute to decisions of summative nature, but with reduced weight and less impact on the student’s life than those of the “high stakes” type, being therefore called “low stakes”¹⁴.

Students actively participate in the PA, being the target of the several feedbacks they receive, as well as in the management of some evaluative events, especially in clinical activities, and, mainly, making records of their activities and exercising reflection and self-assessment in their individual portfolio, supported by their mentors^6)-(8.

DISCUSSION

The reflections presented herein are the result of ongoing work, whose progress may reveal other aspects to be considered. The success in the introduction of the PT as an important element of PA will depend on overcoming the expected challenges and others that may arise. It is also worth considering the existence of facilitating factors for the intended changes. Among them are the good reputation that the PT have in the institution, especially among those involved in curriculum management and the student body, which recognizes its contribution to the promotion of learning. Another fact to be considered is that the institution has already been running supra-disciplinary exams to assess knowledge and skills at specific moments of the course, with formative, summative and informative purposes, depending on the exam and the time of application. For this purpose, there has been a large investment in the institution’s faculty development center, which has qualified personnel and established experience in preparing teachers for student assessment and, specifically, for the task of preparing objective questions of good quality^18),(19.

In developing the proposal for the institutional student assessment program, we tried to characterize it as a collective construction, as recommended for the construction of the political-pedagogical project of a course²⁰ and, especially, for the introduction of programmatic assessment²¹. It is uncertain whether this intention became true but, in its search, an attempt was also made to invest in the educational and evaluation culture of the institution, especially that of the faculty, which may reduce future resistance.

Propositions for the new assessment program were based on the basic principles and concepts of the PA^6)-(8, on general international recommendations for student assessment^22)-(25 and on those for the use of specific methods^14),(23. The experience of foreign consultants and that of one of the authors of this article, lived in another institution²⁶ additionally contributed to the design of the current propositions and reflections.

Although the concepts and practice of PA are relatively recent^6)-(8, an integrative review of 27 quantitative and qualitative studies in courses and programs of the health area in which this modality of student assessment was implemented showed its effectiveness in promoting student learning and development, for whom decision-making on more solid bases plays an important role²⁷. However, many of the studies covered in this review point out that the introduction of the PA can generate tensions within the local educational community²⁷.

The inclusion of the PT in the evaluation program is justified by its indisputable advantages, which were recently reaffirmed in important literature reviews published in Brazil⁵ and abroad²⁸, which are consistent with the institution’s clearly positive experience, arising from its active participation, from approximately 15 years ago, in the first Brazilian consortium to apply the PT.

In particular, the systematic use of PT in medical schools can bring clear benefits to students, who can learn more and better with the feedback and become informed about their degree of cognitive mastery of the topics addressed in each exam, in comparison with both their previous results and those of their peers, in serial examinations⁵. The course management, in turn, can, based on student performance, obtain inferences about the effectiveness of the curriculum in promoting learning, thus detecting its strengths and vulnerabilities²⁸.

In the context of wide-ranging changes in student assessment practices, which are intended with the introduction of a PA-based program in which PT plays a relevant role, it is important to highlight that, in addition to providing relevant information for curriculum management and for the student, who can identify areas in which self-directed investment in study and learning is greater (formative assessment), regular participation in successive PT applications can have other educational impacts. Among them is the possibility of helping students abandon the habit of only studying for tests and then forgetting what they learned (“cramming-and-dumping”)^29),(30, as well as facing the assessment of knowledge with a lower level of stress³⁰.

Another likely contribution of the inclusion of PT in the institutional student assessment program is the encouragement of deeper and more meaningful learning, resulting from the use of contextualized questions in concrete situations and problems, which discourages the study focused on memorization and superficial learning^5),(28. In this assessment program, students should be constantly encouraged to reflect on their performance and, together with their mentor, establish maintenance or improvement strategies.

As for the management, it is important to consider that records of PT participation and performance have shown to be good indicators of student performance in future exams of the “high stakes” type^5),(28),(31.

The exclusively summative use of PT has limitations previously mentioned in the literature⁵ and, additionally, it seems to be less effective for this purpose, resulting in lower scores and lower student engagement, as shown by a study specifically designed to evaluate the inclusion of PT in an assessment program organized along the lines of the PA³². In this study, two medical courses were compared, with the exclusively summative use of the test in one of them and with the inclusion of PT in the PA, together with other measures, similar to what is proposed herein, in the other course³². Thus, even with the summative use of the test, it will be important to take measures that encourage students to reflect on their results and engage in the improvement of their performance. Coupling the assessment program with a mentoring schedule and encouraging students to use a reflective portfolio can contribute to overcoming this challenge.

CONCLUSION

The proposal to include the PT among the formal procedures of a student assessment program in the new curriculum of the medical course, based on the concepts and practices of PA, considers that the characteristics of this test are suitable for institutional assessment systems and programs and can facilitate its implementation. However, challenges, barriers and limitations of a practical nature are expected, especially those linked to the more traditional “assessment culture” prevalent among teachers and students, who tend to consider as relevant only summative assessments within the scope of curricular units (disciplines, modules and internships). These barriers will be overcome due to the facilitating factors present in the institution, which has been discussing the educational evaluation topic for several years, reorganizing its management instances to introduce the PA and the already consolidated experiences of participation in the PT, of the administration of institutional exams and faculty development for student assessment. Anyway, the effectiveness of the PT within the institutional PA system will depend on measures that encourage the active participation of the students, which will require recording their results and being able to reflect on their performance in the PT in order to engage in actions that encourage learning.