An Updated Primer on Entrustable Professional Activities (EPAs)

Cate, Olle ten; Cate, Olle ten

doi:10.1590/1981-5271v43suplemento1-20190238.ing

Serviços Personalizados

Journal

Artigo

Mais
Mais

Permalink

Revista Brasileira de Educação Médica

versão impressa ISSN 0100-5502versão On-line ISSN 1981-5271

Rev. Bras. Educ. Med. vol.43 no.1 supl.1 Rio de Janeiro 2019

https://doi.org/10.1590/1981-5271v43suplemento1-20190238.ing

Revisão

An Updated Primer on Entrustable Professional Activities (EPAs)

Olle ten Cate^I
http://orcid.org/0000-0002-6379-8780

^{^I}Utrecht University, Utrecht, Utrecht, The Netherlands.

ABSTRACT

Since the turn of the millennium, competency-based medical education (CBME) has become a new standard for medical training in many countries. CBME has been operationalized in detailed frameworks of competencies that every physician should demonstrate at graduation, and similar frameworks have been created for specialties. However, the competencies, describing qualities that physicians should possess, do not directly translate into everyday activities of physicians. For that reason, the Entrustable Professional Activities (EPAs) were introduced. EPAs are units of professional practice that may be entrusted to undergraduate students, once they show the competencies needed to perform them without supervision. EPAs have become a popular topic within CBME programs in many countries and hundreds of publications within only a few years. This paper was written to introduce the strengths and weaknesses of EPAs. After a brief historical overview, the reason why EPAs are a bridge between a competency framework and daily clinical practice is explained. While competencies are qualities of individuals, EPAs are units of work. The two can be seen as two dimensions of a matrix, showing that almost all activities in health care are based on multiple competencies, such as communication skill, collaboration, professional behavior, content knowledge. Next, entrustment decision-making as a form of assessment is created and a framework of levels of supervision is presented. Entrustment decisions focus on the level of supervision a student requires for a specific activity, divided into five levels (1: allowed to observe; 2; allowed to perform under direct supervision; 3: allowed to perform under indirect supervision; 4: allowed to perform the activity unsupervised; 5: allowed to supervise the activity performed by more junior learners). For readers interested in applying the concept to practice, a stepwise approach to the curriculum development is proposed. The paper concludes with an overview of the state-of-the-art of working with EPAs across disciplines, professions and countries.

Key words: Professional practice; Competency; Curriculum; Entrustable Professional Activities; Medical education

RESUMO

Desde a virada do milênio, a Educação Médica Baseada em Competências (EMBC) tornou-se um novo padrão para a formação médica em muitos países. A EMBC foi operacionalizada em estruturas detalhadas de competências que todo médico deve demonstrar na graduação, e estruturas semelhantes foram criadas para especialidades. No entanto, as competências, descrevendo as qualidades que os médicos devem possuir, não se traduzem diretamente nas atividades cotidianas dos médicos. Por essa razão, as Atividades Profissionais Confiáveis (APCs) foram introduzidas. As APCs são unidades de prática profissional que podem ser confiadas aos estudantes, uma vez que demonstram as competências necessárias para executá-las sem supervisão. As APCs se tornaram um tema popular nos programas de EMBC em diferentes países com centenas de publicações em poucos anos. Este trabalho foi escrito para apresentar em língua portuguesa as fortalezas e fragilidades das APCs. Após uma sucinta revisão histórica, expõe-se a razão de as APCs serem uma ponte entre o marco das competências e a prática clínica diária. Enquanto as competências são qualidades dos indivíduos, as APCs são unidades de trabalho. As duas podem ser vistas como duas dimensões de uma matriz, mostrando que quase todas as atividades na área da saúde se baseiam em múltiplas competências, como capacidade de comunicação, colaboração, comportamento profissional e conhecimento de conteúdo. Em continuidade, apresentam-se o modo de elaborar uma tomada de decisão de atribuição como forma de avaliação e os referenciais para os níveis de supervisão. As decisões de atribuição se concentram nos cinco níveis de supervisão que o estudante demanda para realizar uma atividade específica: 1. ao aprendiz é permitido observar; 2. é permitido executar a APC sob supervisão; 3. é permitido realizar a APC com supervisão indireta; 4. é permitido executar a atividade sem supervisão; 5: é permitido supervisionar aprendizes iniciantes. Para os leitores interessados em aplicar esse conceito na prática, é proposto um processo com o passo a passo dentro do desenvolvimento curricular. O artigo conclui com uma revisão do estado da arte do trabalho com as APCs em diferentes disciplinas, profissões e países.

Palavras-Chave: Prática profissional; Competência clínica; Currículo; Atividades Profissionais Confiáveis; Educação médica

INTRODUCTION

Entrustable Professional Activities (EPAs) are a new concept in medical education that has aroused much interest among medical educators. Introduced in 2005, nearly 15 years later it has been the subject of hundreds of publications and, currently, several medical education programs in different countries have incorporated the use of EPAs. However, few studies have been published in the literature in Portuguese. The aim of this paper is to introduce and explain this concept starting with a brief historical review.

Brief historical review

Medicine is one of the oldest and most respected professions in the world. It has been a highly selective career option and very attractive to young people, and those who carry the title of doctor have often been viewed as respectable members of society.^¹ In many societies, the right to see and treat patients has been limited to well-educated and trained professionals, either in a university community or in a professional association with its own norms and rules.

In the twentieth century, many Western countries started to regulate the medical profession at the national level, assuming responsibilities for the health of the population and the protection of citizens against incompetent medical professional practice. This led to the concept of competency and the identification of components of the medical curriculum. It was only after the establishment of a massive increase in postgraduate training after World War II, and the need to define educational goals in higher education,^² that outcome- and competency-based education became a guiding principle for medical educators.^{³ - ⁵} Competency-based medical education was adopted in different countries in the 1990s and early 21^st century and its structure has been used to describe the full amplitude of the medical profession in terms of competencies.^{⁶ - ¹⁰}

Particularly in postgraduate training, professional associations such as the Royal Colleges were uncomfortable with the model according to which, within a certain period of time, license to practice would be automatically achieved without any rigorous assessment of training competency. The need to move from the time-based to the competency-based medical education model was felt.^{¹¹ , ¹²} The prestigious 2010 Carnegie Foundation’s report on the reform for medical schools and residencies also proposed a model with fixed standards and flexible paths.^¹³

While national regulators of medical training and professional records reinforced requests for medical schools and students to meet the new standards, not everyone agreed. Some clinical educators^{¹⁴ , ¹⁵} and students^{¹⁶ - ¹⁸} expressed their criticism in the bibliography. For them there was a need to establish a better connection between competencies and daily practice in health care.

What are EPAs and why was the concept created?

The desire to bridge the gap between well-designed competency structure and clinical practice with the patient led to the creation of EPAs.^¹⁹ The EPAs are units of professional practice that constitute the daily practice of clinicians.^²⁰ They can be conceived as responsibilities or tasks that must be performed in patient care. These tasks can be simple or complex. An attending physician or resident in a clinical ward, called at night by the nursing staff, should be able to assess the patient’s clinical worsening and implement actions to stabilize them. This could be an EPA for an intensive care resident. A trainee resident in Obstetrics, if there are no signs of complications, can be trusted to attend to a birth. A sixth-year undergraduate student may be called upon to perform the examination and evaluate a patient with a known chronic condition, if necessary, to perform diagnostic tests, to prepare drug treatment and to do all necessary work which should only be completed and supervised by a member of the hospital’s clinical staff. This can be an EPA. Even a younger student can contribute to health care with simple but meaningful tasks that need not be checked if the student is well trained to do them and report their results.^²¹ EPAs are professional activities that typically have a beginning and an end and are only entrusted to trained personnel. A great part of medical practice can be described as an activity that the professional must be able to perform. What is critical in medical education at the end of the training is that this activity must be performed with safety. This means that student assessment should be focused on the student’s ability to do so and to deal with any unexpected events that might occur during the activity. General competencies, such as adequate communication skills, professionalism and collaborative skills (teamwork) are of critical importance and should be evaluated, but they especially useful to inform the key objectives of training: professional activities.

HOW CAN WE ASSOCIATE EPAS WITH COMPETENCIES?

EPAs comprise the description of a task and do not depend on people. They define a profession in an operational manner.^{¹⁹ , ²²} They are a list of tasks that each clinical department, clinical ward or health worker must do on a day, week, or any other period of time. Job descriptions may list general EPAs and task lists apply to those specific things that must occur within a planned time frame.

Competencies describe people. Students who become competent professionals must acquire competencies that include knowledge, skills and attitudes. Professionals have competencies; but they never have EPAs. EPAs and competencies (or competency domains) can be represented using a matrix model ( Figure 1 ).

FIGURE 1 Competency Matrix of Entrusted Professional Activities (EPAs)

	EPA 1	EPA 2	EPA 3		EPA 4	EPA 5	EPA 6
Specialist	xx	xx	xx		xx		x
Communicator	xx	xx	xx		x	xx	xx
Collaborator		xx	xx		xx	xx
Student			x		xx
Leader					xx		xx
Lawyer			x		xx	xx	xx
Professional	x					xx	xx
APC 1: perform a venipuncture APC 2: perform an appendicectomy APC 3: perform a change-of-shift APC 4: develop and implement a care plan APC 5: lead a multidisciplinary meeting APC 6: request an organ donation				x: the competency is useful for this EPA xx: the competency is necessary for this EPA

Figure 1 indicates EPAs at different levels of training and associates them to the most important competency domains (according to the Canadian CanMEDS model), which should be present when performing EPAs.^²³ This easily proves that an EPA requires multiple competencies from the student, which should be applied in an integrated manner.

Even a clear task, such as obtaining a medical history, combines different competency domains. Professionalism and communication skills are clearly needed, but medical experience is also indispensable for obtaining a guided, efficient and productive history. ( Figure 1 )

ENTRUSTMENT DECISIONS

EPAs can be delegated to students when they are considered ready to perform them. The decision to transfer responsibility to the student has been referred to as the entrustable decision.^²⁴ Such decisions occur every day in the real-life clinical setting. An essential component of training is the regular transfer of responsibilities as the situation permits it. The supervising physician may do so when they realize that the student’s skills at that time correspond to the patient’s complexity and the risks of doing so are acceptable. These are referred to as ‘ad-hoc entrustment decisions’.^²⁵ In contrast, the ‘summative entrustment decision’ has a certification nature. These are decisions made at the moment and focused on the students’ future responsibilities. In services where the curriculum is constructed with EPAs, summative entrustment decisions constitute permission to exercise an EPA when there is a sufficient level of confidence among the faculty about the student’s capacity to carry out this responsibility.^²⁶ The student is assessed regarding their skill and receives rights and duties when engaged in the clinical care of patients.^²⁷ While in regular programs, this right is granted at the end of training, by issuing a diploma or license, in the EPA-based curriculum, it is given to separate EPAs at several times throughout the program, i.e. as soon as it is justified, based on the certainty that the student achieved the program objectives for that EPA. Therefore, an EPA-based curriculum can truly be considered a competency-based curriculum and not just a time-based one. In practice, a reasonable expectation would be that most students would be able to perform all EPAs during undergraduate time, but brilliant students could achieve them earlier, while slower students would do it a little later and those who combined clinical training with research or family assignments, or other legitimate activities, could take longer. Studies have shown, however, that programs that invest heavily in training, simulation, supervision, and frequent feedback can shorten the training process.^{¹¹ , ²⁸}

The ‘ ad-hoc entrustment decisions’, even if they happen often, are determined by more factors than one might think at first sight. Clinical supervisors may not always be aware of these factors but may unconsciously measure them. They are grouped into five factors, which include: a) the perception of students’ characteristics, b) the supervisor’s propensity to delegate responsibility, c) the complexity of the EPA, d) the context, and e) the nature of the teacher-student relationship.^{²⁴ , ²⁹ - ³¹} Each of these can be further detailed:

The students’ characteristics include specific skills and experience with EPA, but together with them, three other characteristics of professionalism can be perceived ^{²⁷ , ³² - ³⁴} integrity (veracity and benevolence in reporting to the clinical supervisor), reliability (stable, predictable behavior, and conscientious working attitude) and humility (the ability to observe one’s own limitations and willingness to ask for help when needed). Additionally, the student needs to know how to handle unknown situations.^³⁵
Clinical supervisors may differ greatly in how easily they delegate responsibilities to students ^²⁴ ; this is reported as depending on the specialty, experience and personality.^³⁰
Logically, a more complex EPA will not be entrusted so early to a trainee but an easier and less risky one. This also goes for unusual activities.
The context includes patient complexity, available facilities, time of day or night, staffing requirements, but also the rules of supervision, restrictions of the working hours, and perception of student mentality.^³⁶
The relationship between the student and the clinical supervisor must evolve beyond the initial impressions to allow for criticism in entrustment decisions^{³⁷ , ³⁸} and this relationship in postgraduate studies has been described as a dance that takes place between the experienced resident and the clinical supervisor, when residents reach the end of their training.^³⁹

As the ‘summative-entrustment decision’ is of a certification nature, sometimes referred to as the ‘Statement of Awarded Responsibility’ (STAR)^¹⁹ , it must be based on sufficiently substantiated trust,^²⁶ which, to avoid risks of subjectivity, would include an agreement between multiple observers and multiple observations so that entrustment can be justified.

Recommended sources of information include several direct, longitudinal observations, case-based discussions, and the products of evaluations. Additionally, knowledge and skill assessment scores may be considered.^⁴⁰

It is important to note that an entrustment decision implies acceptance of the risk that the student will encounter at the meeting and which cannot be fully predicted. In particular, the case-based discussion^⁴¹ , a short, focused conversation, usually after an EPA, should thoroughly investigate the student’s understanding about what was done and should include ‘what-if’ questions to understand whether the student would know what to do when facing unexpected or unfamiliar situations.

SUPERVISION LEVELS AS WORKPLACE ASSESSMENT SCALE

Thus far, we have discussed entrustment decisions as dichotomous decisions: to entrust or not to entrust. However, it is very useful to associate entrustment decisions to decreasing levels of supervision. Five main levels are described: Level 1: The trainee is allowed to be present and observe but not perform the EPA; Level 2: The trainee is allowed to perform EPA under direct and proactive supervision present in the room; Level 3: The trainee is allowed to perform EPA without a supervisor in the room, but readily accessible if necessary, e.g., with indirect and reactive supervision; Level 4: The trainee is allowed to work without supervision, Level 5: The trainee is allowed to supervise novice learners. This structure has been widely recommended.^{¹⁹ , ²⁵ , ⁴⁰} Although the idea of using EPAs has emerged for postgraduate training programs, recently undergraduate programs have started to use them. This has led to detailed levels of supervision.^{⁴² , ⁴³}Figure 2 provides a more detailed picture.^{⁴⁴ , ⁴⁵} ( Figure 2 )

FIGURE 2 Assignment and supervision scale: original and expanded form

Assignment and supervision scale with five levels	Expanded entrustment and supervision scale for undergraduate and postgraduate medical education
1. It is not allowed to practice EPA.	1. It is not allowed to practice EPA: Inadequate knowledge / skills; not allowed to observe (e.g., lack of biosafety knowledge), Adequate knowledge, some skills; allowed to observe.
2. It is allowed to practice EPA only under fully proactive supervision.	2. It is allowed to practice EPA only under fully proactive supervision: As a co-activity with the supervisor, With the supervisor in the room willing to intervene, if necessary.
3. It is allowed to practice EPA only under reactive supervision or supervisory request.	3. It is allowed to practice EPA only under reactive supervision or supervisory request: With an immediately accessible supervisor, and all findings and decisions have been double checked, With an immediately accessible supervisor, and key findings and decisions have been double-checked, With a supervisor at distance (e.g., phone), and findings and decisions have been promptly reviewed.
4. It is allowed to practice EPA without supervision.	4. It is allowed to practice EPA without supervision. With at distance monitoring (e.g., check student’s questions on the next day), Without monitoring.
5. It is allowed to supervise others in EPA practice.	5. It is allowed to supervise others in EPA practice.

The Entrustment-supervision (ES)^⁴⁶ scales can have other formats. In the training of anesthesiologists, such scales have been translated into a nine-point mini-clinical evaluation exercise (mini-CEX) scale, which has been shown to increase the reliability of the evaluation.^⁴⁶ This is due to the close association of evaluation practices with the clinic.^⁴⁷ Recently, Weller et al.^⁴⁸ described a nine-point reliability scale that uses colloquial language for anesthesiology supervisors^⁴⁸ : “I am not comfortable leaving the operating room.” “I can take a brief coffee break.” “I can take a break for lunch.” “I have to check in regularly.” “I can start it and leave the room and wait until you ask me for help.” “I trust the student can initially deal with the whole situation.” “I can just review everything with the student beforehand.” “I can leave the room.” “The student can work as a consultant.”

Although it is not the main ES-scale, it is an excellent illustration of the central idea of entrustability scales: aligning assessment with clinical practice.^⁴⁹

BUILDING A CURRICULUM WITH EPA IN PRACTICE

Applying EPA to the medical curriculum can be considered a process that requires several steps.^⁴⁰

EPA identification

Describing a health service curriculum in terms of EPA requires a reflective process. EPAs as units of practice should not be so small or so large. They must be associated with the work in the clinical setting. Several processes have been described to arrive at adequate EPAs, most of which involve local and international expert groups.^{²³ , ⁴³ , ⁵⁰ - ⁵²} For a complete program, the adequate number of EPAs is considered to range from 20 to 34

Creating full descriptions of EPAs

A complete description of an EPA includes eight sections, and most can be restricted to one or two pages. These sections are:

Title. An adequate title of the activity, which allows its immediate recognition by students and clinicians. It can be a name or include a verb but should not contain an adjective. It should not be written as an educational objective – since it is an activity.
Specifications and limitations. The activity must be specified in detail and must contain no more nor less than what the EPA means. The specifications can be a list of task components. The limitations pertain to the liability restrictions of entrustment decisions such as ‘only concerning hemodynamically stable patients.’
Potential risks in case of failure. Entrustment decisions are risky, and this section is used to specify and understand adverse events that can occur when the task is not performed well.
Association with a prevalent competency scale. The matrix in figure 1 shows the competencies required for a given EPA. This can be mentioned in the third section and can guide the task performance assessment.
Knowledge, skills and attitudes and experience that may be considered before making a summative entrustment decision.
Sources of information to support entrustment decisions. These may be the specification of a satisfactory number of direct observations, longitudinal observations, case-based discussions, and assessment products that support a summative entrustment decision.
Expected level of supervision for this EPA in this educational program. For most EPAs in undergraduate medical school, this level can be level 3 and for postgraduate programs, level 4.
Optionally, an expiration date (or period) may be added. This means that after a summative entrustment decision, if the student has not practiced the EPA for a certain period of time, the decision may expire and the individual (undergraduate or postgraduate student) must be supervised again.

Determining forms of assessment and their rules

In an educational program, rules and regulations must be transparent. These should include the times or periods when students or residents are expected to meet the criteria of all nuclear EPAs. This may include the possibility of adding elective EPAs for advanced and brilliant students.

Establishing individualized itineraries through portfolios

Medical training based on competency should allow for some flexibility and individualization in the learning itineraries, due to differences in students and the contexts of practice settings. In a program for assistant physicians, which has been based on EPAs since 2010, it was found that in this relatively short, 2.5-year, highly individualized program, students starting with a target of 7 EPAs (average 6.8) ended with slightly fewer completed EPAs (6.6 on average) and 1.5 altered EPAs, as calculated among 101 undergraduate students.^⁵³ A widely used approach to student achievement follow-up for the recording of the evaluations of entrustment decisions and feedback is the electronic portfolio, and some models are now commercially offered.

Allowing for flexibility in the amplitude or throughout training

A burning question that often arises is how to accommodate flexibility in the practical training programs, particularly in postgraduate training, where students are planned to provide health care in the service and to do so faster or more slowly can seriously interfere with the established rotation and schedule. The organization of flexible and individualized programs is the consequence of true competency-based training, but it must be borne in mind that it may not be easy.

Basically, there are two approaches. One is to update by adapting the schedule. In countries with access to highly regulated postgraduate training (e.g., the United States of America and Canada annually on July 1^st) this seems impossible, but it would be possible in other countries where undergraduate students leave medical school at various times throughout the year and the start of the residency program is equally flexible. Additionally, several programs have the capacity to accommodate pregnancy and maternity leave, and some may combine medical training programs and doctoral programs or residency and doctoral programs. These will not have much difficulty in incorporating the flexibility of the competency-based curriculum.

The other approach is not to adapt time but to distinguish nuclear from elective EPAs and vary the EPAs required for graduation in the portfolio. The Dutch radiology program, which was redesigned based on EPAs, offers that. Residents are expected to add a domain of special interest to their set of nuclear EPAs, but it may also be two or, in rare cases, none. An Academic Medicine supplement of March 2018 was fully devoted to all the details of time-varying competency-based education.^⁵⁴

FINAL CONCLUSIONS

The EPAs first appeared in 2005 and have now become popular in postgraduate programs and, more recently, in undergraduate programs.

Specialization programs in psychiatry, pediatrics, internal medicine, family medicine, anesthesiology, emergency medicine and other areas have documented EPA initiatives.^{²² , ⁵⁰ , ⁵⁵ - ⁵⁹} EPAs in medical undergraduate programs ^⁴² have become a reality in countries such as the United States of America,^⁵² Canada^⁶⁰ and the Netherlands.^⁶¹ In Alternative Medicine, Nursing, Medical Assistant training, Veterinary Medicine, Dentistry and Obstetrics, the EPAs have been actively developed and have attracted interest even in teacher training programs.^⁶²

A specific topic, such as assessment based on the entrustment decision-making is likely to become a research area of interest and, in North America, the concept of Milestones^⁶³ will be related to EPAs.

In brief, EPAs are a new and exciting area of study that has not yet fully developed, but that will likely affect the future of competency-based medical education in many countries.^⁶⁴

Given the relevance of the topic, part of the content of this study was published by the author in Spanish and Korean.

REFERENCES

1 Lyons AS, Petrucelli RJ. Medicine: an illustrated history. New York: Harry N. Abrams; 1987. [ Links ]

2 Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomy of educational objectives: the classification of educational goals. New York: Longmans, Green and Co; 1956. [ Links ]

3 McGaghie WC, Miller GE, Sajid AW, Telder TW. Competency-Based Curriculum development in medical education: an introduction. Geneva: World Health Organization; 1978. [ Links ]

4 Harden RM, Crosby JR, Davis MH. AMEE Guide No. 14: outcome-based education: Part 1: an introduction to outcome-based education. Med Teach. 1999;21(1):7-14. doi: 10.1046/j.1467-2494.1999.181696.x. [ Links ]

5 ten Cate O. Competency-Based Medical Education. In: Cockerham WC, Dingwall R, Quah S, editors. The Wiley-Blackwell Encyclopedia of Health, Illness, Behavior, and Society. Hoboken: John Wiley & Sons; 2014. p. 1329-35. [ Links ]

6. CanMEDS 2000: extract from the CanMEDS 2000 Project Societal Needs Working Group Report. Med Teach. 2000;22(6):549-54. doi: 10.1080/01421590050175505. [ Links ]

7 Swing SR. The ACGME outcome project: retrospective and prospective. Med Teach. 2007 Sep;29(7):648-54. doi: 10.1080/01421590701392903. [ Links ]

8 General Medical Council. Tomorrow ’s doctors: outcomes and standards for undergraduate medical education. London; 2009. [ Links ]

9 Pangaro L, ten Cate O. Frameworks for learner assessment in medicine: AMEE Guide No. 78. Med Teach. 2013 Jun;35(6):e1197-210. doi: 10.3109/0142159X.2013.788789. [ Links ]

10 Bleker OP, ten Cate TJ, Holdrinet RSG. General competencies for the medical specialist in the future [Dutch]. Dutch J Med Educ. 2004;23(1):4-14. [ Links ]

11 Long DM. Competency-based residency training: the next advance in graduate medical education. Acad Med. 2000 Dec;75(12):1178-83. [ Links ]

12 Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med. 2002 May;77(5):361-7. [ Links ]

13 Cooke M, Irby DM, O’Brien BC. Educating physicians: a call for reform of medical school and residency. Hoboken: Jossey-Bass; 2010. [ Links ]

14 Brooks MA. Medical education and the tyranny of competency. Perspect Biol Med. 2009 Winter;52(1):90-102. doi: 10.1353/pbm.0.0068. [ Links ]

15 Glass JM. Competency based training is a framework for incompetency. BMJ. 2014 Apr 25;348:g2909. doi: 10.1136/bmj.g2909. [ Links ]

16 Grant J. The incapacitating effects of competency: a critique. Adv Health Sci Educ Theory Pract. 1999;4(3):271-7. doi: 10.1023/A:1009845202352. [ Links ]

17 Hodges BD. A tea-steeping or i-Doc model for medical education? Acad Med. 2010 Sep;85(9 Suppl):S34-44. doi: 10.1097/ACM.0b013e3181f12f32. [ Links ]

18 Whitehead CR, Kuper A. Faith-based medical education. Adv Health Sci Educ Theory Pract. 2017 Mar;22(1):1-3. doi: 10.1007/s10459-016-9748-8. [ Links ]

19 ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007 Jun;82(6):542-7. doi: 10.1097/ACM.0b013e31805559c7. [ Links ]

20 ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005 Dec;39(12):1176-7. doi: 10.1111/j.1365-2929.2005.02341.x [ Links ]

21 Chen HC, Sheu L, O'Sullivan P, ten Cate O, Teherani A. Legitimate workplace roles and activities for early learners. Med Educ. 2014 Feb;48(2):136-45. doi: 10.1111/medu.12316. [ Links ]

22 Schultz K, Griffiths J, Lacasse M. The application of entrustable professional activities to inform competency decisions in a Family Medicine Residency Program. Acad Med. 2015 Jul;90(7):888-97. doi: 10.1097/ACM.0000000000000671. [ Links ]

23 ten Cate O, Snell L, Carraccio C. Medical competency: the interplay between individual ability and the health care environment. Med Teach. 2010;32(8):669-75. doi: 10.3109/0142159X.2010.500897. [ Links ]

24 Sterkenburg A, Barach P, Kalkman C, Gielen M, ten Cate O. When do supervising physicians decide to entrust residents with unsupervised tasks? Acad Med. 2010 Sep;85(9):1408-17. doi: 10.1097/ACM.0b013e3181eab0ec. [ Links ]

25 ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013 Mar;5(1):157-8. doi: 10.4300/JGME-D-12-00380.1. [ Links ]

26 ten Cate O, Hart D, Ankel F, Busari J, Englander R, Glasgow N, et al. Entrustment decision making in clinical training. Acad Med. 2016 Feb;91(2):191-8. doi: 10.1097/ACM.0000000000001044. [ Links ]

27 ten Cate O. Entrustment as assessment: recognizing the ability, the right, and the duty to act. J Grad Med Educ. 2016 May;8(2):261-2. doi: 10.4300/JGME-D-16-00097.1. [ Links ]

28 Nousiainen MT, McQueen SA, Ferguson P, Alman B, Kraemer W, Safir O, et al. Simulation for teaching orthopaedic residents in a competency-based curriculum: do the benefits justify the increased costs? Clin Orthop Relat Res. 2016 Apr;474(4):935-44. doi: 10.1007/s11999-015-4512-6. [ Links ]

29 Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst W, O'Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014 Aug;19(3):435-56. doi: 10.1007/s10459-013-9474-4. [ Links ]

30 Choo KJ, Arora VM, Barach P, Johnson JK, Farnan JM. How do supervising physicians decide to entrust residents with unsupervised tasks?: a qualitative analysis. J Hosp Med. 2014 Mar;9(3):169-75. doi: 10.1002/jhm.2150. [ Links ]

31 Wijnen-Meijer M, van der Schaaf M, Nillesen K, Harendza S, ten Cate O. Essential facets of competency that enable trust in medical graduates: a ranking study among physician educators in two countries. Perspect Med Educ. 2013 Nov;2(5-6):290-7. doi: 10.1007/s40037-013-0090-z. [ Links ]

32 Kennedy TJ, Regehr G, Baker GR, Lingard L. Point-of-care assessment of medical trainee competency for independent clinical work. Acad Med. 2008 Oct;83(10 Suppl):S89-92. doi: 10.1097/ACM.0b013e318183c8b7. [ Links ]

33 Mayer RC, Davis JH, Schoorman FD. An integrative model of organizational trust. Acad Manag Rev. 1995;20(3):709-34. doi: 10.5465/amr.1995.9508080335. [ Links ]

34 Holzhausen Y, Maaz A, Cianciolo AT, ten Cate O, Peters H. Applying occupational and organizational psychology theory to entrustment decision-making about trainees in health care: a conceptual model. Perspect Med Educ. 2017Apr;6(2):119-26. doi: 10.1007/s40037-017-0336-2. [ Links ]

35 Wijnen-Meijer M, Van der Schaaf M, Booij E, Harendza S, Boscardin C, Van Wijngaarden J, et al. An argument-based approach to the validation of UHTRUST: can we measure how recent graduates can be trusted with unfamiliar tasks?. Adv Health Sci Educ Theory Pract. 2013 Dec;18(5):1009-27. doi: 10.1007/s10459-013-9444-x. [ Links ]

36 Teman NR, Gauger PG, Mullan PB, Tarpley JL, Minter RM. Entrustment of general surgery residents in the operating room: factors contributing to provision of resident autonomy. J Am Coll Surg. 2014 Oct;219(4):778-87. doi: 10.1016/j.jamcollsurg.2014.04.019. [ Links ]

37 Hirsh DA, Holmboe ES, ten Cate O. Time to trust: longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014 Feb;89(2):201-4. doi: 10.1097/ACM.0000000000000111. [ Links ]

38 Sagasser MH, Fluit CRMG, van Weel C, van der Vleuten CPM, Kramer AWM. How entrustment is informed by holistic judgments across time in a Family Medicine Residency Program: an ethnographic nonparticipant observational study. Acad Med. 2017 Jun;92(6):792-9. doi: 10.1097/ACM.0000000000001464. [ Links ]

39 Balmer DF, Giardino AP, Richards BF. The dance between attending physicians and senior residents as teachers and supervisors. Pediatrics. 2012 May;129(5):910-5. doi: 10.1542/peds.2011-2674. [ Links ]

40 ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37(11):983-1002. doi: 10.3109/0142159X.2015.1060308. [ Links ]

41 ten Cate O, Hoff RG. From case-based to entrustment-based discussions. Clin Teach. 2017 Dec;14(6):385-9. doi: 10.1111/tct.12710. [ Links ]

42 Chen HC, van den Broek WE, ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015 Apr;90(4):431-6. doi: 10.1097/ACM.0000000000000586. [ Links ]

43 Wisman-Zwarter N, van der Schaaf M, ten Cate O, Jonker G, van Klei WA, Hoff RG. Transforming the learning outcomes of anaesthesiology training into entrustable professional activities: a Delphi study. Eur J Anaesthesiol. 2016 Aug;33(8):559-67. doi: 10.1097/EJA.0000000000000474. [ Links ]

44 Peters H, Holzhausen Y, Boscardin C, ten Cate O, Chen HC. Twelve tips for the implementation of EPAs for assessment and entrustment decisions. Med Teach. 2017 Aug;39(8):802-7. doi: 10.1080/0142159X.2017.1331031. [ Links ]

45 Chen HC, ten Cate O. Assessment through Entrustable Professional Activities. In: Delany C, Molloy E, editors. Learning and teaching in clinical contexts: a practical guide. 2nd ed. Chatswood: Elsevier; 2018. p. 286-304. [ Links ]

46 Weller JM, Misur M, Nicolson S, Morris J, Ure S, Crossley J, et al. Can I leave the theatre?: a key to more reliable workplace-based assessment. Br J Anaesth. 2014 Jun;112(6):1083-91. doi: 10.1093/bja/aeu052. [ Links ]

47 Crossley J, Johnson G, Booth J, Wade W. Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ. 2011 Jun;45(6):560-9. doi: 10.1111/j.1365-2923.2010.03913.x. [ Links ]

48 Weller JM, Castanelli DJ, Chen Y, Jolly B. Making robust assessments of specialist trainees’ workplace performance. Br J Anaesth. 2017 Feb;118(2):207-14. doi: 10.1093/bja/aew412. [ Links ]

49 Kogan JR, Conforti LN, Iobst WF, Holmboe ES. Reconceptualizing variable rater assessments as both an educational and clinical care problem. Acad Med. 2014 May;89(5):721-7. doi: 10.1097/ACM.0000000000000221. [ Links ]

50 Hauer KE, Kohlwes J, Cornett P, Hollander H, ten Cate O, Ranji SR, et al. Identifying entrustable professional activities in internal medicine training. J Grad Med Educ. 2013 Mar;5(1):54-9. doi: 10.4300/JGME-D-12-00060.1. [ Links ]

51 Fessler HE, Addrizzo-Harris D, Beck JM, Buckley JD, Pastores SM, Piquette CA, et al. Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: report of a multisociety working group. Chest. 2014 Sep;146(3):813-34. doi: 10.1378/chest.14-0710. [ Links ]

52 Englander R, Flynn T, Call S, Carraccio C, Cleary L, Fulton TB, et al. Toward defining the foundation of the MD degree: core Entrustable Professional Activities for entering residency. Acad Med. 2016 Oct;91(10):1352-8. doi: 10.1097/ACM.0000000000001204. [ Links ]

53 Wiersma F, Berkvens J, ten Cate O. Flexibility in individualized, competency-based workplace curricula with EPAs: analyzing four cohorts of physician assistants in training. Med Teach. 2017 May;39(5):535-9. doi: 10.1080/0142159X.2017.1297526. [ Links ]

54 Lucey CR, Thibault GE, ten Cate O. Competency-based, time-variable education in the health professions: crossroads. Acad Med. 2018 Mar;93(3S):S1-S5. doi: 10.1097/ACM.0000000000002080.5. [ Links ]

55 Boyce P, Spratt C, Davies M, McEvoy P. Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011 Nov 23;11:96. doi: 10.1186/1472-6920-11-96. [ Links ]

56 Englander R, Carraccio C. From theory to practice: making entrustable professional activities come to life in the context of milestones. Acad Med. 2014 Oct;89(10):1321-3. doi: 10.1097/ACM.0000000000000324. [ Links ]

57 Jonker G, Hoff RG, ten Cate OT. A case for competency-based anaesthesiology training with entrustable professional activities: an agenda for development and research. Eur J Anaesthesiol. 2015 Feb;32(2):71-6. doi: 10.1097/EJA.0000000000000109. [ Links ]

58 Beeson MS, Warrington S, Bradford-Saffles A, Hart D. Entrustable professional activities: making sense of the emergency medicine milestones. J Emerg Med. 2014 Oct;47(4):441-52. doi: 10.1016/j.jemermed.2014.06.014. [ Links ]

59 Kwan J, Crampton R, Mogensen LL, Weaver R, van der Vleuten CP, Hu WC. Bridging the gap: a five stage approach for developing specialty-specific entrustable professional activities. BMC Med Educ. 2016 Apr 20;16:117. doi: 10.1186/s12909-016-0637-x. [ Links ]

60 Touchie C, Boucher A, editors. Entrustable Professional Activities for the transition from medical school to residency. Ottawa: Association of Faculties of Medicine of Canada; 2016. [ Links ]

61 ten Cate O, Graafmans L, Posthumus I, Welink L, van Dijk M. The EPA-based Utrecht undergraduate clinical curriculum: development and implementation. Med Teach. 2018 May;40(5):506-13. doi: 10.1080/0142159X.2018.1435856. [ Links ]

62 Leijen Ä li, Slof B, Malva L, Hunt P, van Tartwijk J, van der Schaaf M. Performance-Based Competency requirements for student teachers and how to assess them. Int J Inf Educ Technol. 2017 Mar;7(3):190-4. doi: 10.18178/ijiet.2017.7.3.864. [ Links ]

63 Nasca T, Philbert I, Brigham T, Flynn TC. The next GME accreditation system - rationale and benefits. N Engl J Med. 2012 Mar;366(11):1051-6.doi: 10.1056/NEJMsr.1200117. [ Links ]

64 Touchie C, ten Cate O. The promise, perils, problems and progress of competency-based medical education. Med Educ. 2016 Jan;50(1):93-100. doi: 10.1111/medu.12839. [ Links ]

FUNDINGIeda Francischetti: grant 2018/26011-2, São Paulo Research Foundation (FAPESP).

TRANSLATION TO PORTUGUESE

Prof Dra. Ieda Francischetti. Docent in the Faculdade de Medicina de Marília, Maríla, Brazil.

Post-doctorate in the Dieter Scheffner Center for Medical Education and Educational Research, Charité Univertätsmedizin Berlin, Berlin, Germany E-mail: iedafster@googlemail.com

Received: August 22, 2019; Accepted: September 17, 2019

ADDRESS FOR CORRESPONDENCE Prof. Dr Th. J. ten Cate. Center for Research and Development of Education. University Medical Center Utrecht. P.O. Box # 85500, 3508 GA Utrecht, The Netherlands. E-mail: t.j.tencate@umcutrecht.nl

CONFLICTS OF INTEREST

The author declares that he has no competing interests.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.