INTRODUCTION

Anesthesiology is a medical specialty recognized for its associated risks and for its constant concern with improvements in the field of patient safety (PS). Anesthesiologists are involved in the care of a large number of patients, whose work takes place in complex systems involving procedures with several critical steps, professionals with different training, use of equipment, drugs, techniques and technologies, which requires a high level of vigilance. These factors, added to the constant increase in the complexity of surgical procedures, the growing pressure for productivity and results, high workload, lack of control over the schedule and issues related to the high risk of mental exhaustion, represent a greater risk for failures and potential injuries to patients undergoing anesthesia.

A robust education on PS, especially during the training phase of new professionals, is a demand of the National Patient Safety Program (PNSP, Programa Nacional de Segurança do Paciente)¹ and the Declaration of Helsinki² on patient safety in anesthesiology, which highlights that “Education has a fundamental role in improving safety, and we fully support the development, dissemination and training in patient safety.”

The competency matrix of the Medical Residency Programs in Anesthesiology in Brazil also has objectives related to training in PS³. The matrix is the result of the change from a traditional curriculum based on training time to a competency-based curriculum. Mandatory since March 1, 2020, the competency matrix has as its specific objective “To perform anesthesia safely in all its stages” and other objectives related to the stratification and reduction of anesthetic-surgical risk³.

Despite this, however, there is no clear definition of what the curriculum on patient safety in anesthesiology should be like, and it is up to the residency program to structure its method of teaching on the subject using the available resources⁴. Studies that evaluated the performance of anesthesiology residents on the subject showed low performance, which was attributed to the lack of more in-depth and effective knowledge, appreciation of the subject and ineffective teaching⁵. Additionally, physicians in specialization constitute a population of health professionals who are vulnerable in terms of patient safety due to their lack of experience and knowledge on the subject. This, combined with their workload and competing learning priorities, potentially places these physicians and their patients at increased risk of involvement in adverse events⁶.

Considering this scenario, this study aims to train Anesthesiology residents on safe practices that are of great relevance to minimize incidents related to the care of patients undergoing anesthesia. For this purpose, the objective was to develop and implement an educational module on patient safety in anesthesiology for residents in the specialty and to evaluate the module effectiveness and the residents’ degree of satisfaction with it.

METHOD

This is a study with a quasi-experimental design and interrupted time series design with a group⁷, carried out within the scope of the professional Master’s Degree in health education, at Universidade Federal do Rio Grande do Norte - UFRN, with 10 residents attending from the first to the third year of the Medical Residency Program in Anesthesiology at the Onofre Lopes University Hospital, which belongs to the Brazilian Company of Hospital Services (Ebserh - Empresa Brasileira de Serviços Hospitalares) network. The study was developed in two stages, the first being the creation of the educational module entitled “Patient Safety in the Anesthesiologist’s Practice”, in the e-book format with eight units, as shown in Chart 1:

During the second stage, the educational module was applied and residents were assessed, organized in four face-to-face meetings, with the participation of 10 residents (three, four and three residents attending the first, second and third years of residency, respectively). In the first meeting, the participants answered a pre-test questionnaire to assess prior knowledge, which was available online, consisting of eight multiple-choice questions, with each question corresponding to one Unit of the e-book.

Also in the first meeting, the educational module was made available to participants in PDF e-book format and an introductory lecture was held on the general aspects of the topic of PS and safety culture, corresponding to Units 1 and 2 of the e-book. It was established that at each subsequent meeting (with one-week intervals), after the participants had previously read the content individually, two Units of the educational module would be presented in person at the hospital where the residency was held, through the discussion of published clinical cases, resolution and discussion of questions prepared by the Teaching and Training Committee of the Brazilian Society of Anesthesiology (SBA, Sociedade Brasileira de Anestesiologia) between 2015 and 2022, and the sharing of the residents’ experiences. Thus, this pedagogical strategy was maintained in the following meetings.

At the end of the fourth and final meeting, in addition to the presentation and discussion of the module’s content, two types of questionnaires were applied: the first to assess the knowledge acquired through a post-test instrument similar to the pre-test one and the second related to the satisfaction assessment questionnaire.

The satisfaction questionnaire was made available online and included 18 questions, organized into two parts: in the first, the participants answered six questions about their satisfaction with the course structure (content, language, format, organization and acquisition of knowledge) and the following open-ended question: “Would you recommend other topics that you consider important for the discussion on patient safety in the practice of anesthesiology and that were not covered by the teaching material?”.

In the second, the participants self-assessed their perception of the impact that the knowledge acquired through the educational intervention could have on their clinical practice. The assessed items are related to the objectives of each discussed topic, such as: greater interest in contributing to the safety culture; adherence to the World Health Organization (WHO) Safe Surgery Checklist; patient identification; adequate verification of equipment before anesthetic induction; hand hygiene; administration of antibiotic prophylaxis at the appropriate time; adequate drug identification; performing an airway assessment of all patients undergoing anesthesia; planning the airway approach; greater attention to patient positioning care.

In addition, at the end of this section there was a space for criticism, suggestions or praise of the teaching material presented, which was not mandatory.

A five-point Likert scale was used, which included I Fully Disagree = 1; I Disagree = 2; Neutral = 3; I Agree = 4 and I Fully Agree = 5, for the closed questions in both parts of the questionnaire. To estimate the satisfaction questionnaire reliability, the Cronbach’s alpha coefficient value was calculated (analysis performed using the R programming language - R CORE TEAM, 2023)²⁵, with the 16 questions of the questionnaire in which the Likert Scale was used.

Regarding data analysis, the categorical data were represented in tables using frequencies and their respective percentages. Measures such as minimum value, median, mean, standard deviation and maximum value were used to represent the quantitative data. To analyze the pre- and post-test questionnaires, the students’ final grades in the pre- and post-test were compared, assessing whether the grades showed a significant difference, using the t-test and Wilcoxon test. The Shapiro-Wilk test was also used to test the normality of the data. The level of significance adopted was α = 5%. All analyses were performed using the R²⁵ programming language (R CORE TEAM, 2023). Finally, the responses to the Satisfaction Questionnaire were evaluated and summarized in a table with the distribution of the general data and by year of residency, as well as their respective percentages.

The study was approved by the Research Ethics Committee (REC) of the institution under CAAE number 60774022.2.0000.5292 and all residents involved in the study signed the Free and Informed Consent Form.

RESULTS

2-Satisfaction assessment

Nine participants completed the satisfaction questionnaire. The value calculated for Cronbach’s alpha coefficient was 0.793, with the consistency of this questionnaire being considered as “Acceptable”. The coefficient was calculated using the R programming language (R CORE TEAM, 2023)²⁵.

Overall, the questionnaire responses were satisfactory for each of the items, with all of them ranging from “I Agree” to “I Fully Agree”, as shown in Graph 1.

Source: prepared by the authors.

Graph 1 Assessment of student satisfaction with the teaching material “Patient Safety in the Anesthesiologist’s Practice”. 

When asked if they would recommend other topics that they considered important for learning about PS in the anesthesiologist’s practice, and that were not covered by the teaching material, the participants answered “No” in their entirety, showing that they felt covered by the topics developed in the course. In the other open question, aimed at criticism, suggestions or praise for the teaching material presented, of the nine questionnaires answered, 4 participants answered:

“Excellent! Adequate time, well-selected questions. The question-based approach followed by theory is good for increasing engagement in the subject.”

“Approaches with questions lead to greater learning”

“It was great. Didactic and objective.”

“Congratulations, great material.”

Regarding the residents’ self-perception of the impact that the knowledge acquired through the teaching material could generate in relation to their practice, all answered favorably (Graph 2) with answers such as “I agree” and “I fully agree”.

Source: prepared by the authors.

Graph 2 Perception of the impact that the knowledge acquired through the teaching material may generate in practice. 

DISCUSSION

The educational module “Patient safety in the anesthesiologist’s practice” was positively evaluated by 1^st to 3^rd-year anesthesiology residents, who considered its content relevant, with a potential positive impact on the residents’ training in the specialty on the specific topic, with high levels of satisfaction.

The methodology of assessing prior knowledge through pre-testing is recommended in the WHO Patient Safety Curriculum Guide²⁶, which highlights the following benefits: establishing standards, the possibility of assessing the mastery of skills, prior knowledge and progress over time, establishing student classification, diagnosing the points of greatest difficulty, assessing the effectiveness of a course, and the possibility of providing feedback and motivation to students. Multiple-choice questions (MCQ) were considered by the Guide as appropriate formats for assessing student knowledge, but with the disadvantage that this assessment does not measure the applicability of knowledge in practice. Another point to be considered about MCQ is the testing effect. This effect refers to the idea that the test would not only have the function of assessing knowledge, but also improving the learning process²⁷. Thus, the test effect also allows the use of MCQ as a teaching tool, with the ability to promote knowledge acquisition and retention for a longer period than other approaches, in a more adaptable manner²⁸.

Although studies indicate low performance of residents regarding patient safety aspects due to the lack of more in-depth and effective knowledge, little appreciation of the topic and ineffective teaching⁵, the participants’ average pre-test scores (between 6.25 and 8.75, with an average of 7.5) was above 7.0, considered a positive finding. After the educational intervention, there was an increase in the average scores (9.03 - scores ranging from 7.5 to 10), with a significant difference to the values of the median grades, which demonstrates the effectiveness of the worked educational module on its objective of training anesthesiology residents on PS care.

Studies on strategies for the development and implementation of an educational module on PS aimed at anesthesiology residents and anesthesiologists are scarce⁴. Even with the understanding that education is a crucial factor for the continuous promotion of a safe environment, and that training in PS is the most important and most urgent task for a change in the safety culture,²⁹ a curriculum on the subject, its approach and educational methods are not well established in the specialty. The available literature is mainly aimed at students in health areas or residents of various medical specialties.

In a study conducted with 120 students at the Johns Hopkins School of Medicine, a PS curriculum was developed and implemented and its impact on the medical students’ safety knowledge, self-efficacy and systemic thinking was evaluated³⁰. Through the pre- and post-assessment of the students’ safety knowledge, self-efficacy in safety skills and systems-based thinking, in addition to the collection of satisfaction data, a significant improvement was observed in knowledge and systemic thinking scores, as well as a statistically significant increase in communication and safety skills taught and high satisfaction. The study highlights the need for further in-depth analysis of the long-term impact of this education on the students’ knowledge, skills, attitudes and behaviors³⁰.

A randomized clinical trial⁶ evaluated the effectiveness of two types of online learning methodologies to improve PS behaviors required by the Joint Commission National Patient Safety Goals (NPSG) by 371 residents from different specialties. The study aimed to observe the impact on behavior, knowledge retention, and user satisfaction. Improvement in NPSG knowledge, NPSG-compatible behaviors in a simulation scenario, self-reported confidence in safety and quality, acceptability, and relevance of the program were evaluated. A simulation of central catheter insertion was included, and the goals defined for evaluation of the station were: improve communication effectiveness; improve patient identification accuracy; reduce the risk of healthcare-associated infection; comply with hand hygiene guidelines; perform pre-procedure verification process; and improve medication use safety.

The residents of that study who received the educational intervention completed an online satisfaction survey to assess their confidence and acceptability of the interventions on a Likert scale of 1 to 5, ranging from “I fully agree” to “I fully disagree”. The online learning programs improved knowledge retention and self-reported confidence. The study found a relatively poor performance for all trainees on the central line insertion simulation station and concludes that: there may be significant gaps in medical education in the PS area and that it is important to invest in educational programs in the area for young physicians. However, it considers the existence of multiple barriers to sustainable course integration, which included students, faculty and institutional factors⁶.

A study that systematically reviewed the quality improvement and patient safety curricula published for medical students and/or residents³¹ had the following objectives: to determine the applied educational content and teaching methodology; to evaluate the obtained learning outcomes; and to identify obstacles to curricular implementation. The content of the 41 evaluated curricula most frequently included topics such as continuous quality improvement, root cause analysis and systemic thinking. Regarding the methodology, the most commonly used design was the simple before-after comparison and courses with a workload that involved around 10 hours of contact and one to five meetings. Self-reporting was used as the method in the five studies that reported behavioral changes³¹. The curricula were generally well accepted, with significant improvements in knowledge acquisition in all studies. High satisfaction rates were obtained, usually assessed using a Likert scale, and one-third of the studies implemented changes in the provision of care³¹. A subsequent systematic review³² aimed to update this study and showed a similar conclusion.

The present study is compatible with data available in the literature on the implementation of a course on PS conducted with medical students and residents from different medical specialties, such as those presented previously. There are similarities between all these studies regarding several aspects. First, in relation to the choice of topics considered relevant, such as: general aspects of PS (key terminology and history), root cause/systems-based analysis, and the international goals established by the WHO and the Joint Commission International.

Second, regarding the adopted methodology, the design of the studies was the “before and after” type (pre- and post-intervention assessment of knowledge); assessment of satisfaction through post-intervention questionnaires on a Likert scale; assessment of perceptions of behavioral change through self-reporting and using personalized questionnaires with items corresponding to the learning objectives of the course.

Also, regarding the results obtained, there was good acceptance of the presented curricula, statistically significant improvements in the acquisition of knowledge and high satisfaction rates. Finally, some limitations were observed related to methodological issues, such as implementation in single centers and small samples; need to measure the long-term impact of interventions on students’ knowledge, skills, attitudes and behaviors; lack of evidence of the impact of educational interventions on patient outcomes.

In this sense, it was observed that the combination of the adopted pedagogical strategy associated with the protected time and space for the participation of residents in the present study were essential factors for the application of the educational module and for the positive results obtained. These findings highlight the importance of formally including the topic of PS in the curricular structure of residency courses, aiming to promote the acquisition of knowledge, attitudes and skills on the subject during the training process, in line with the recommendations of the WHO²⁶ and the National Patient Safety Program¹ of Brazil.

The available literature on the topic of education of anesthesiology residents in PS mainly deals with the importance of developing non-technical skills such as communication, leadership and teamwork, and the role of practical training in simulation as a fundamental tool³³. However, most published studies make little reference to the theoretical basis applied, making it difficult to generalize results across different contexts³⁴. According to expert consensus, the recommendation is that “investigators describe the theoretical basis for the chosen intervention components or provide an explicit logic model of why this patient safety practice should work” ⁽³⁴. Thus it is possible to better develop and select different approaches to PS education, evaluate their implementation, and allow generalizations.

Additional studies on patient safety education for anesthesiology residents are needed, especially regarding the ideal curriculum and its educational approach. Moreover, it is necessary to evaluate the extent to which educational interventions influence change in behavior or skill development in residents, whether these changes are sustained, and whether they are capable of bringing practical results to patients. However, such research is inherently challenging, since, due to the size of residency programs, sample sizes are often small and student randomization may be unethical or impractical³⁵. Furthermore, studies using methodologies that allow the assessment of parameters at higher levels of the Miller pyramid, such as the application of knowledge acquired by residents, problem-solving, and positive impact on patients, may be difficult and require institutional support.

The practice of anesthesiology involves manual skills and management in a complex system. Thus, the development and implementation of educational approaches in PS for this group requires evidence-based practices to improve the most diverse skills. The teaching and assessment of these skills require active learning methodologies, such as through simulation and structured clinical examinations, as well as resident participation in the analysis of adverse events and involvement in improvement projects related to patient safety⁴.

CONCLUSION

The educational strategy developed and applied in this study proved to be effective for the training of anesthesiology resident physicians on the topic of patient safety, as the acquisition of knowledge and the high levels of acceptance and satisfaction were increased. In this sense, a difference was observed when protected time and space were offered for the acquisition of knowledge, attitudes and skills on the topic during the training process, inferring that the formal inclusion of the topic of patient safety in the curricular structure of residency courses is a path with great potential for success.

However, further studies are needed to measure the long-term impact of this strategy, its influence on the acquisition of skills and whether it results in better outcomes for patients.