INTRODUCTION

The modernization of Medicine over the centuries has changed humankind’s way of life, the evolution of the health context, and innovations in recent decades have also reached medical education^1),(2. The advancement of knowledge and technologies for teaching and learning, thanks to the currently easy access to technical information, generating a revolution in the educational process, and new challenges^1),(2. These changes have required rapid adaptations in our society and the active methodologies with student-centered learning, by stimulating their autonomy and generating growing motivation, have shown to be of a great value in several Universities¹.

For centuries, the basis of medical education in universities followed the traditional model, with great lectures and the teacher as the center of all information³. New interactive active methodologies have been proposed and implemented, and knowledge has become centered on the student’s learning experience^4)-(19. These methodologies became even more evident during the pandemic caused by Coronavirus Disease 2019 (Covid-19), as interpersonal relationships were drastically affected in this scenario and classroom teaching became remote, based on technology and distance learning (DL)²⁰.

Active methodologies are defined as instructional methods to engage students in the learning process through participatory pedagogical tools^4),(5. These models seek to encourage students to learn based on problem situations, with a focus centered on themselves and autonomy²¹. This is important, considering the concepts of cognitive levels or categories described in Bloom’s taxonomy, for the development of skills and competencies²¹. Among the different methods that can be employed are case-based teaching, gamification, the use of virtual voting platforms, “just-in-time” teaching, among others²¹.

The case study is a teaching approach based on actual context situations and on the discussion and resolution of a clinical problem²¹. In this methodology, the teacher assumes the role of learning facilitator and the students the central role of the process²¹. Another methodology that can be used is gamification, which is defined by the use of game mechanics, its elements, strategies and logic to motivate actions and solve complex real-life problems²². The use of questions and votes, such as puzzles in a classroom, helps to structure discussion sessions, and guide the students to focus on difficult points²².

Socrative® and Kahoot!® are digital plataforms wich are founded free online and support student-centered teaching activities¹². Among the benefits achieved through them, it can be observed that they allow educators to mobilize knowledge through questions during classes and receive immediate answers from the participants, which can even be anonymous¹². Through this active participation, there is real-time interaction between the instructor and the student, which allows instant feedback¹².

This is important since, in traditional lecture classes, especially in classrooms with a large number of students, interaction with the teacher becomes difficult and students may show apprehension or anxiety to actively participate or resolve their doubts for fear of public errors, embarrassment or disapproval from colleagues^6),(7),(23. Therefore, educational systems are being challenged by the low involvement of students in teaching activities²⁴, creating a growing demand for the use of active methodologies that, in addition to the benefits described, with a low cost of implementation²⁵.

Thus, through the introduction of new active teaching methods, the students are able to realise their achievements during learning, as well as their weaknesses²⁶, directing their focus on what should be revised and/or deepened¹. Therefore, the aim of this study was to investigate the performance and perception of students using the Kahoot!^® and Socrative^® tools associated with case-based teaching and the “just-in-time teaching” method.

METHODS

The comparative analysis was carried out during the clinical case discussion sessions in the discipline of Urology in a medical course. The sample consisted of 193 medical students attending the third year of the course, from August 2019 to July 2020. The project was approved by the Research Ethics Committee (REC) of the university Pontifical Catholic University of Paraná - (CAAE number 20653519.2.0000.0020) -, and the Free and Informed Consent Form (FICF) was voluntarily signed by the participants.

The application of the tools was carried out using the “pre-test” format. In each session, two objective questions containing four response options were applied. In each of the six weeks, a topic of the discipline was worked on, which was previously communicated to the students. Regarding the test scores, each question was worth one point, with the student’s final score varying between zero and two, which was used as part of the students’ official score, with a weight of 10% of the final grade.

Kahoot!® is a free online learning platform that allows you to create multiple choice questions, discussions, surveys, or a combination of these styles. The participants use their personal devices to answer questions projected on the screen, playing individually or as a team (“gamification”). The interaction occurs through music, colorful shapes and time was measured, aiming to keep participants excited and involved in the activity. The faster the student correctly answers the question, the higher their score will be. At the end, a high quality and high resolution scoreboard is displayed, showing how many students chose each of the answer options and their general classification. Socrative® is a free online response system used to create assessments and view responses in real time, getting instant feedback. It allows creating multiple-choice questions, true or false, open and closed questions, in addition to allowing sharing them.

When starting the clinical case discussion session, the students connected to the internet, which was provided by the school, and accessed the application’s website through their smartphone device or notebook, entered the virtual room previously created by the teacher and answered questions regarding the class topic. Each class was divided into six groups (A to F), with students from groups A to C carrying out the case discussion groups in the first period and groups from D to F in the second period. The tools were rotated between the periods, that is, when students from the first period used Socrative®, those from the second one used Kahoot!®. In the following week, the order was reversed. At the end of the rotation, all 193 students used each tool three times. Aiming to avoid fraud, the two questions in the first period were different from the two questions in the second one. In order to pair the groups regarding the level of difficulty of the questions, in the next class (first semester of 2020), there was an inversion of the questions asked between the tools, and the same questions asked in Socrative® in the first class started to be asked in Kahoot!^® for the second class and vice versa.

Initially, in the second class (first semester of 2020) the methodology for distributing participants into six groups and for scoring the questions occurred in the same way, in addition to addressing the same six theoretical topics. However, due to the Covid-19 pandemic, there was the need to interrupt in-person classes and start the remote virtual classes. Therefore, from the third tutorial session onwards, case discussions were carried out through the online platform Blackboard Collaborate Ultra® (Blackboard Inc., California, USA) and their discussions were recorded, so that all students could review them later. Nevertheless, the same methodology was maintained in the performance of the pre-tests, since at the beginning of the tutorial session, the undergraduate students continued to access the application website and answer the established questions. To reduce the risk of fraud, students agreed to keep their cameras on.

At the end of the semester, the students answered a questionnaire to analyze the perception of each of the tools, anonymously, and the responses were categorized into five options: strongly disagree, partially disagree, neither agree nor disagree, partially agree and strongly agree (Likert scale, from 1 to 5). This questionnaire was based on and adapted from the Brazilian study “Kahoot^® use as an evaluation and teaching-learning tool in the discipline of industrial microbiology”²⁷.

In order to facilitate the description and analysis of the results, we grouped numbers 1 and 2 as a discordant answer, number 3 as a neutral answer and numbers 4 and 5 were grouped as a concordant answer. The results with the option “I neither agree nor disagree” were considered neutral and, therefore, contributed little to the purpose of the research. Questions 5 and 8 had a negative character and, for the adequate analysis of the answers, the options were inverted, as the discordant answers indicated a positive assessment of the technology. Score 1 on the scale became score 5, score 2 became score 4, and vice versa.

The data were analyzed using SPSS Statistics v.20.0 software (Armonk, NY: IBM Corp.) and non-parametric Wilcoxon (questionnaire score) and Chi-square tests were used. Values of p<0.05 indicated statistical significance.

RESULTS

The scores of the 193 students were evaluated and the final perception questionnaire was answered voluntarily by 74.09% (143/193) of the sample.

A total of 174 students participated in the tutorial on urinary tract infection, 86 undergraduate students answered the Kahoot!® and 88 students answered the Socrative® tools. Considering the scores zero, one and two correct answers on Kahoot!® we obtained 22.1% (19/86), 45.3% (39/86) and 32.6% (28/86), respectively. With Socrative®, there were 9.1% (8/88), 40.9% (36/88) and 50% (44/88) for zero, one and two correct answers, respectively. When comparing the applications, there was a higher rate of correct answers with Socrative® (p=0.017).

In the urinary lithiasis topic, of the 158 students, 88 used the Kahoot!® and 70 the Socrative®. Regarding the first one, 8% (7/88) of the sample did not get the correct answers, 45.5% (40/88) got only one and 46.6% (41/88) got two. In the Socrative®, no student got zero (0/70), 48.6% (34/70) got one correct answer and 51.4% (36/70) got two correct answers (p=0.054).

In the tutorials with the topic of outpatient surgery in urology, which included 161 individuals, the Kahoot!® was used by 68 and showed 7.4% (5/68) with zero correct answers, 48.5% (33/68) with one and 44.1% (30/68) with two correct answers. The Socrative®, performed by 93 undergraduate students, showed 7.5% (7/93) with zero correct answers, 35.5% (33/93) with one and 57% (53/93) with two correct answers (p = 0.235).

Among the 145 students present in the urinary incontinence classes, 87 used Kahoot!® and 58 students used Socrative®. A total of 20.7% (18/87) of the participants who used the Kahoot!® got zero correct answers, 41.4% (36/87) got one and 37.9% (33/87) got two correct questions. In relation to Socrative®, 12.1% (7/58), 31% (18/58) and 56.9% (33/58) got zero, one and two correct answers, respectively (p=0.072).

Regarding the topic of benign prostatic hyperplasia, with 170 undergraduate students, 72 on Kahoot!® and 98 on Socrative®, 9.7% (7/72) of the sample that used the first tool did not get any correct answers, 40.3% (29/72) ) got one correct question and 50% (36/72) two correct answers. In the latter, 15.3% (15/98) of the students did not get any correct answers, 39.8% (39/98) got one and 44.9% (44/98) got two correct answers (p=0.540).

Finally, in testicular cancer tutorials, with 139 students, the Kahoot!® application, used by 58 students, showed 10.3% (6/58) with zero correct questions, 22.4% (13/58) with one correct and 67.2% (39/58) with two correct answers. In the Socrative® tool, used by 81 students, 1.2% (1/81) of the sample got zero correct answers, 19.8% (16/81) got one and 79% (64/81) got two correct questions. There was a higher rate of correct answers with the Socrative® tool (p=0.042).

The comparison of the sum of correct questions for each topic performed in the tutorials is shown in Graph 1.

Graph 1 Comparison of the percentage of correct answers between the Socrative® and Kahoot® tools according to the addressed topic. 

The questions asked in the student perception questionnaire are described in Figure 1. Regarding the answers found, questions 1 to 7 showed statistical significance in the comparisons.

Figure 1 Student perception assessment questionnaire applied at the end of the semester. 

Table 1 describes the results of the same answers for the two applications. For instance, in question 1, 65.7% of the students agree that both technologies allow maintaining the degrees of difficulty. In question 8, 63.6% of the students disagreed that both technologies are more tiring than an ordinary written test.

Aiming to assess the superiority between the teaching tools in relation to the written test, we compared the opposite answers. The most relevant data found were that 35% of students agree that only Socrative® allows assessing questions in the same way as a written test (question 2; p<0.001) and 32.2% that only Socrative® allowing a fair grade using the correct answer classification (question 3; p<0.001). In both questions, only 0.7% of the students attributed the characteristics to Kahoot!®. The highest percentages obtained are described in the last line of the table: the “Socrative® is more similar to the written test than the Kahoot!®” (Table 2).

Regarding the global satisfaction index of each tool, the average score of Socrative® was 1.8 points higher than that of Kahoot!® (p<0.001) (Table 3).

DISCUSSION

Based on the result presentation, it was observed that the students obtained better correct answer rates when using the Socrative® application in two topics of clinical discussion session, and in two other subjects there was a trend towards more correct answers with the Socrative® tool (p =0.054 and p=0.072). As the entire sample used both tools for the same number of times, we believe there was no influence on the result regarding the greater or lesser use of a specific tool. The comparison was made through the scores obtained in the pre-tests carried out before the formal beginning of the tutorial session, aiming at promoting the undergraduates’ study beforehand, since these assessments occurred before the students participated in the class about the topic. Therefore, the students had greater practical participation in the tutorial sessions because they already understood part of the content, which possibly resulted in greater understanding and learning, in addition to the possibility of solving any doubts they had in the prior study. This technique comprises the so-called “flipped classroom”, where students complete the pre-readings and pre-consultation, and then coming prepared with basic concepts to develop the understanding during the session¹³.

A possible explanation for such results are the dynamic differences shown by the applications. Socrative® implements more objective and traditional multiple-choice questions, with alternatives from A to E, with simple handling²³. The Kahoot!®, on the other hand, has a gamification dynamic, which encompasses game practices and features to engage and motivate student learning in real time and in a more playful way¹. Among such features, a song is played while the participants carry out the question, the alternatives to the questions are colored pictures, instead of letters, and the time taken to complete the test is shown on the screen, since, at the end of the evaluation, the app projects a ranking of the students who obtained the highest number of correct answers in less time. All of this takes place with the aim of exciting and engaging the participants through active learning in a fun and competitive environment among peers¹. However, these resources may have triggered anxiety in the students, hindering their performance when answering the questions, being a possible hypothesis for the better results observed with Socrative®. In the literature, there were no articles that compared the grades obtained by students when using the Socrative® and Kahoot!® applications in assessments during classes.

In the study on which the questionnaire was based, “Kahoot® use as an evaluation and teaching-learning tool in the discipline of industrial microbiology”, the conclusion was that Kahoot!® generates an additional stimulus for students, making the evaluation process more attractive and the learning longer lasting²⁷. One of the advantages of using mobile tools pointed out in the study “Tips for using mobile audience response systems in medical education” is their autonomy, since they can be used at any time and place where the students are carrying an electronic device with access to the internet¹³.

Previous studies in the areas of pediatrics²⁶, microbiology⁸, gynecology and obstetrics²⁹, physiology³⁰ and radiology³¹ showed several benefits after the introduction of virtual tools that support active methodologies. As for the specific use of Socrative®, students indicated that they appreciated the use of technology, as it improved their understanding and promoted their participation in class^{6),(11),(14),(23),(27),(30}. These data were also verified with the participants of this study, which showed that, after the statistical analysis of the questionnaires, there was significance regarding the comparison between the Kahoot!® and Socrative® tools in all the questions in the questionnaire (except number 8). Through this analysis, it was concluded that Socrative® was the application the students preferred and that it achieved better percentages of acceptance and higher average grades when evaluated between 0 and 10. No previous study promoted the comparison of the students’ perception between both tools.

Among the limitations found in the study, one can observe the disparity of prior knowledge and access to different sources through which students sought knowledge, the possibility of fraud when using electronic devices as a means of communication during the time of activity, absence of a control group performing traditional written assessment and the absence of a comparative group by implementing post-tests. The necessary migration to the virtual environment in the final phase of the study may have influenced the results, as a result of the change in the students’ learning process. New studies using the Kahoot!® and Socrative® tools can be carried out aiming at comparing the virtual and the in-person environments.

Finally, the observations presented herein show us that this type of study opens the door to new studies on the subject, aiming to stimulate the literature on a subject that is currently so important and relevant. Due to the originality of the study, we expect more research in medical education comparing these and other applications, so that we can understand and define the best scenario to be used in the teaching practice. It is hoped that other disciplines and courses can officially implement new assessment techniques as an effective method to improve student learning performance and motivate students and teachers towards the new reality.

FINAL CONSIDERATIONS

Active methodologies are important allies in medical education, as they bring the student to the center of learning. Using tools such as online voting methods helps teachers and students to make the learning environment more active and engaged. In this study, the Socrative® tool showed better results when compared to Kahoot!® when analyzing the scores obtained by the students. Moreover, this same tool was also shown to be more satisfactory when the students’ perception was analyzed.

This study is a pioneer in the comparison of grades obtained in the Socrative® and Kahoot!® apps regarding a discipline in a Brazilian medical education and is expected to encourage new future studies and changes in learning practices.