1 Introduction

A new electronic context is gradually changing the teaching and learning processes and, consequently, changing the teachers’ working environment which determines that those rethink the methodological and evaluative methods focusing on the development of digital competences since, accordingly to the Common National Curriculum Base (^{BRASIL, 2018}, p. 61), “[...] young people have been engaging themselves as protagonists of the digital culture, directly involving in new multimedia and multimodal interactions forms and social networking, which are becoming faster”.

School is a historically located institution and, thus, is subject to transformations and influences that affect society as a whole in order not to stagnate nor ignore the advancements that highlight the contemporary world (^{ALMEIDA, 2014}). On the other hand, it is also one of the roles of the school to prepare active citizens who influence and contribute to transform society, considering the same advancements. One of those is digital, which must be used for education exploring all its possibilities and advantages.

Technology alone will never change the learning in an educational institution or a classroom, once it cannot assume that its simple use will modify the entire learning process. The teacher shall direct its implementation, use, and, most importantly, its evaluation, because “[...] this favorable attitude only makes sense, however, if every teacher has the knowledge concerning what could be done with the available technologies to, later, combine them with the curriculum goals” (^{COSTA et al., 2012}, p. 24).

Thus, it is important to integrate the digital in the pedagogical context as a tool for learning services and, consequently, the evaluative process, since it helps in the development of the “[...] capacity to select and use the necessary information according to certain goals and, inherently, the capacity to judge the effective value of the available information based on these criteria” (^{COSTA, 2019}, p. 19).

Through the digital, teaching and learning processes become more efficient, because there is a possibility to implement:

[...] the working strategies in which the teachers take the role of helping students to think, learn how to identify and solve problems, establish and define their own goals, regulate their learning, evaluate results, and establish new goals according to this evaluation, finally, help students to learn. (^{COSTA, 2019}, p. 23).

The integration of the digital on pedagogical enables educators to potentially transform classes into significant learning experiences for students, making knowledge live and more meaningful, in other words, it helps to create authentic learning contexts subsidized by technology because:

[...] it is essential to have an education that offers learning conditions in contexts of uncertainty, development of multiple literacies, questioning of information, autonomy for complex problems resolutions, living with diversity, group work, active participation on social media, and tasks sharing. (^{BACICH; MORAN, 2017}, p. 16).

Thus, it is necessary to promote the development of digital competence, which happens from the relation among knowledge, abilities, attitudes, and safe and objective use of digital tools. Those are the guiding principles for the construction of the European Framework for the Digital Competence of Educators (DigCompEdu) (LUCAS; MOREIRA, 2018). This reference shows a selection of digital competences for educators, aiming to support its development to use the digital in an effective, efficient, critical, and creative way for their students. Besides that, the Key Competences for Lifelong Learning European Framework, from the Council of the European Union, states that:

Digital competence involves the confident, critical and responsible use of, and engagement with, digital technologies for learning, at work, and for participation in society. It includes information and data literacy, communication and collaboration, media literacy, digital content creation (including programming), safety (including digital well-being and competences related to cybersecurity), intellectual property-related questions, problem-solving, and critical thinking.

While digital literacy might be understood as a necessary ability to reach digital competence, in other words, it involves developing the competences directed for building knowledge using the digital, according to Messias, Loureiro e Barbas (2012, p. 2):

[…] being digital literate does not refer only to the capability to use a computer or an email, but to the capability to gather, understand, interpret and share information available in all digital media. Being digital literate gives us the ability to communicate and work more efficiently, because it involves understanding how all digital devices work and how they can be used to interact with society.

The reference which supported this investigation, DigCompEdu, according to ^{Lucas e Moreira (2018}, p. 8), aims to: “Answer to the growing awareness among many European states members that educators need a group of specific digital competences for their jobs in order to be able to take advantage of the digital technologies to improve and innovate education”.

Thus, the goal is to describe the specific digital competences that educators should develop for the effective integration of the digital in the pedagogical context. It proposes 22 elementary competences, organized in six areas, focused on different aspects of the educators’ professional activities: professional engagement, digital resources, teaching and learning, assessment, empowering learners, and facilitating learners’ digital competence, as shown in Figure 1.

Source: ^{Lucas and Moreira (2018)}.

Figure 1 Areas and competences of DigCompEdu 

The competences defined on DigCompEdu are explained according to a model of progress in six levels of proficiency: newcomer (A1), explorer (A2), integrator (B1), expert (B2), leader (C1), and pioneer (C2). These levels of proficiency indicate to educators their stage of development in each elementary competence and, therefore, in each area.

Along with DigCompEdu, European Commission's Joint Research Centre (JRC) developed and validated a self-evaluation tool for Digital Competences, Check-In. This questionnaire allows educators of all education levels to self-perceive their digital competence, in order to know which level they globally are per area and by each elementary competence, also receiving suggestions of actions for, from the level they are, progress to higher levels.

Check-In is a specific questionnaire for educators of different levels, from preschool to higher education, and it was used in this study as a diagnostic tool to identify the digital proficiency of Portuguese and English teachers from the final years of Elementary School and High School in public state schools in Santo Antônio da Platina, in Parana, Brasil.

2 Methods

This study shows the discussion of the results of an investigation of pre-experiment, pretest, and post-test design applied in one group of participants (^{CRESWELL, 2003}) through a quantitative methodological approach with data gathering and analysis through Check-In. Its main goal is to verify the self-perceived level of digital competences of the teachers who participated in the research aiming to: I) determine the importance of teacher formation designed specifically on the area of assessment with the digital on the development of digital competences in this area; II) understand if the teachers’ identification of digital competences through Check-In is useful for the formation design and programming; and III) if the intensive formation designed accordingly with the teachers’ current digital competences promotes their development to higher levels.

The investigation was between November 2020 and May 2021 with 15 Portuguese and English teachers of Elementary and High public schools in the city of Santo Antônio da Platina, in Paraná, Brasil. Concerning the sample, since the teachers are from the same local and of only two different subjects, Portuguese and English, it is considered a non-probabilistic and convenience sample.

The participant group had three male (20%) and 12 female teachers (80%). Concerning the ages, one teacher was between 25 and 29 years old (7%), five were between 30 and 39 (33%), seven were between 40 and 49 (47%), and two were between 50 and 59 (13%). Concerning the years of service, one teacher has been teaching between one and three years (7%), one between four and five years (7%), one between 6 and 9 years (7%), three between 10 and 14 years (20%), six between 15 and 20 years (40%), and three for more than 20 years (20%).

The investigation consisted of the application of an initial questionnaire with the teachers aiming to know their global and specific digital competences in different areas, through the Check-In diagnostic tool.

After detecting that most of the weaknesses were on the assessment of learning through digital (Area 4 of DigCompEdu), it was possible to consecutively boost a course using digital tools on the formative assessment, planned accordingly to the results obtained by Check-In. The training course included the assessment types (diagnostic, formative, and summative), showing the concepts, characteristics, and examples of them, the integration of digital in assessment processes, being tested through the elaboration of formative assessments using Google Forms. The goal was to enable Portuguese and English teachers on the assessment through digital in order to subsidize actions in the school environment.

Finally, the same initial questionnaire was repeated to check if, after the course, there were positive changes in the teachers’ digital competences, especially in the learning assessment area.

The Check-In questionnaire is a self-reflection tool about digital competences developed by the European Commission's JRC. It is a free access questionnaire, in which authorization is implicit, and a properly validated tool used in different studies about teachers’ digital competences. The application was from December 2020 to May 2021 and online through Google Forms, allowing participants to answer from any device with internet access, thus providing comfort and speed on the participation. The hyperlinks for the questionnaires were sent to the participants via e-mail and instant messages applications. Data collection was through questionnaires imported to Statistical Package for the Social Sciences (SPSS Statistics) program for statistical analysis.

Check-In is divided into six areas, with 22 questions total, focusing on supporting and encouraging the use of digital tools to improve and innovate education. The areas are organized as:

Lastly, it included 15 closed-ended questions with a five-point likert scale, varying from “Strongly disagree” to “Strongly agree”, of personal nature to better understand the respondents’ profiles.

The application of the questionnaires was completely anonymous, using the same tool before and after a training course planned accordingly with the data obtained after the application of the first questionnaire.

With the teachers’ answers from the Check-In questionnaire in hand, after analyzing the results, it was possible to observe weaknesses in the assessment area of digital competence, thus planning the training course.

The course was through Google Meet, with eight hours load, named “Digital technologies on the formative learning assessment”. The classes were given on a single eight-hour session on different days with different teachers, therefore, the same course was offered on different dates, since the teachers were not able to participate on the same date. Thus, three teachers did the course on April 13, seven on April 15, two on May 3, and three on May 13, totalizing 15.

Considering the results of the questionnaire, the course included the following themes: general concepts of the assessment area, assessment modalities, integration of digital in assessment processes, and elaboration of online formative assessments using the Google Form tool.

The goal of the training course was to provide moments that allowed participants to know or recognize different assessment forms, as well as to understand which moments are more favorable to apply each assessment type or procedure, focusing on the formative assessment, showing that digital technologies are ways to evaluate teaching and learning besides enabling teachers to elaborate different assessments using technology.

Aiming to show a digital formative assessment in practice, besides the concepts, the teachers received two assessments, one of English and the other of Portuguese languages, with automatic formative feedback through Google Forms so it would be possible to visualize the viability of the application of the digital formative assessment and understand how data are presented for both students and teachers, reflect about the results, and have a position about them to promote interventions according to the needs shown.

The participants also saw, in practice, how to elaborate a formative assessment on Google Forms, considering that this tool is highly used by teachers and also its possibilities of use.

The training course started with a dynamic activity using the Google Jamboard tool to collect the teachers’ information concerning an associative image of the word “assessment” to generate an initial debate and introduce the theme, which enabled interaction among the participants. After that, we exposed a brief history of assessment, showing how it has been done through the centuries, as well as a brief legislative history of assessment in Brazil to understand and show that the current law guarantees formative assessments.

Next, we showed the differences between the concepts of assessment and exam, showing the characteristics that differ and presenting the concepts of assessment according to different authors, as well as the concepts and differences between diagnostic, formative, and summative assessments aiming to let teachers perceive the differences between each type. During the formative assessment, the emphasis was on feedback and its importance both for teacher and student.

We continued to explain the need, importance, and benefits of digital integration in the assessments processes, and the teachers had the opportunity to make a practical activity of being in students’ position and make a digital formative assessment in their area, with automatic feedback through Google Forms for later discussion about the experience. Later, we exposed some points that deserve attention during the writing of assessment instruments showing examples of assessments that were not correctly elaborated and led students to unsatisfactory results.

The closing of the training course had a step-by-step explanation on how to elaborate online formative assessments using Google Forms, in other words, how to elaborate different questions, insert multimedia elements, include feedback as well as elements of video, text, and links with the final reflection of provoking teachers to elaborate an online formative assessment using the shown tool to be applied in a classroom.

Lastly, the teachers were given a period of a month to apply their digital formative assessments for their students to, then, repeat the application with the teachers of the initial sample to collect data aiming to compare the results of the initial and final questionnaires and analyze if the training course had positive changes on the teachers’ conceptions and digital competences, especially on the learning assessment since the goal was to make teachers achieve a higher level of digital competence.

The complete process of data collection and analysis followed legal precepts in accordance with the Ethics Charter of Universidade de Lisboa, General Rule of Data Protection of Portugal, besides the assent of the Ethics Commission of the Institute of Education of Universidade de Lisboa.

3 Results and discussion

The study included 15 teachers who answered the first questionnaire, did the proposed training course and, finally, answered the questionnaire again after the course.

As previously referred, DigCompEdu describes six different levels in which digital competence usually develops from A1 to C2. It is important to highlight that, in all competences, the progression of proficiency levels is cumulative, thus the descriptor of each level includes the descriptors of the level below it.

The analysis of the results of the application of this questionnaire allowed us to identify, by teacher and by each of the competences, the areas of higher or lower weaknesses as well as the areas that need more courses and studies to follow for the next proficiency level.

The general analysis of the responses collected allows a reflection about the results, identifying the most developed and the weakest areas of competence. Table 1 shows the global results obtained on the initial questionnaire (before the training course) and on the final questionnaire (after the training course) by DigCompEdu area and by competence level.

Concerning Area 4 - Assessment, focus of the training course, on the first application of the questionnaire, before the training course, out of the 15 teachers who answered the questionnaire, 68,8% were on the level A2, the “Explorers”, who are professionals that:

[...] are aware of the potential of digital technologies and are interested in exploring them to enhance pedagogical and professional practice. They have started using digital technologies in some areas of digital competence, without, however, following a comprehensive or consistent approach. (LUCAS; MOREIRA, 2018, p. 30).

Still concerning the levels of the teachers, 18,8% were on level A1, the “Newcomers”, who are aware of the potential of digital technologies but have had very little contact with them; and 12,5% are on the level B1, the “Integrators”, who integrate technologies in most of their activities. None of the teachers are on levels B2 (Expert), C1 (Leader), and C2 (Pioneer), the highest levels concerning the use of digital technologies.

After the application of the first questionnaire, it was possible to evince the low digital competence of the teachers, especially in the assessment area, which might explain the lack of use of the digital in the pedagogical environment, supporting Vargas-D'Uniam et al. (²⁰¹⁴, p. 374):

[...] there is a barely significant correlation between the digital skills of teachers and the educational uses they make in the classroom, denoting a low level of skills and, therefore, a basic use of technological resources by them¹.

After the training course, among the 15 teachers who answered the questionnaire, 12,5% were on level A1, 56,3% were on level A2, 25% were on level B1, and 6% were on level B2. None of the teachers had C1 or C2 levels, the highest levels concerning the use of digital technologies.

After the training course, the participant teachers showed an increase in their proficiency levels on Assessment - Area 4, as shown in Figure 2. Most of the teachers evolved from basic assessment use to efficient and strategic assessment use, consequently being able to explore strategies of digital assessments and improve the traditional assessment approaches.

Source: The authors (2021).

Figure 2 Percentage of teachers by level of digital competence in assessment before and after the training course 

On the first application of the questionnaire, 68,8% were on competence level A2, 18,8% on A1, and 12,5% on B1. On the second application 56,3% were on A2, 25% on B1, 12,5% on A1, and 6% on B2. Therefore, it was possible to verify a decrease in the number of teachers with A1 and A2 competence levels and an increase of the amount of those with B1 and B2 levels in the assessment area. None of the teachers reached levels C1 or C2 in both applications.

Results showed that, concerning the teacher formation through evidence of what those professionals need, the results are related to the transformation of the pedagogic practice is visible, because:

In most of the formative actions, it predominates a formation based on the transmission of knowledge, methods, and techniques through theories which are transmitted without context, that does not consider the teachers’ real needs and problematic situations, which hinders them from being subjects in their own formative process [...]. (^{SOARES, 2020}, p. 153).

It is important to highlight the existence of many studies about teachers’ digital competences, in Portuguese context, about higher education professors, for example:

Espinosa and Gutiérrez (2013) made a self-evaluation based on the model developed by Gutiérrez and Espinosa (2013), in Spanish higher education professors, identifying that: [...] d) 54% had an average-low knowledge concerning implementing and evaluate educational actions with TIC [...]’. (^{SANTOS; PEDRO; MATTAR, 2021}, p. 9).

It is shown that both basic and higher education educators need to improve themselves concerning the use of digital technologies, therefore being able to consider this a transversal need for different countries and levels of education.

In this study, on the global level (Figure 3), before the training course: 0% of the teachers were on level A1, 25% on level A2, 37,5% on level B1, 18,8% on level B2, 18,8% on level C1, and 0% on level C2; after the training course, 6% of the teachers were positioned on level A1, 6,3% on level A2, 31,3% on level B1, 31,3% on level B2, 25% on level C1, and 0% no on level C2, which also indicates an improvement of the competence level. After the training course, some teachers became A1, the numbers of A2 and B1 teachers decreased, and more teachers became levels B2 and C1.

Source: The authors (2021).

Figure 3 Percentage of teachers by level of global digital competence before and after the training course 

Those results strengthen the importance of the teachers’ reflections on their own practices and training, which stimulates new ways of developing their work given that:

The basis of the training is the subjects’ reflection on their teaching practices in order to allow them to examine their implicit theories, their working schemes, their attitudes, etc, making a constant self-evaluation process that guides their work. The orientation for this reflection process demands a critical proposal of educational intervention, an analysis of the practices considering the underlying ideological and behavioral assumptions. (^{IMBERNÓN, 2011}, p. 51).

Thus, after the training course, with the data from the first and second questionnaires, it was evident that there were improvements in the teachers’ digital competences in the six areas. It was possible, then, to observe the importance of the training to be built, as said by ^{Imbernón (2011}, p. 94), “[...] from the teachers’ demands or the educational institutions demands aiming to help the problem-solving processes or professional problematic situations that are inherent to them [...]”, because only through identifying the problem-situations on the teachers is possible to promote pedagogical changes.

Thus, to those demands in the digital area, tools like Check-In are of great help in the design of the training course, because, after identifying the teachers’ specific weaknesses in different areas, it allows the conception of a training course directed to the same difficulties. As previously stated in the study by Lucas, Dorotea, and Piedade (2021), the training course contributes to increasing the teachers’ confidence and positive attitudes related to the integration of technology in the education context. To reach this effect, it is fundamental to stimulate teachers to self-evaluate and self-reflect about the level of their difficulties and needs, things for which the teacher training course is essential.

4 Closing remarks

The main goal of education is to help students to have a stimulus “[...] to critical thinking, to development of interaction capacities, information negotiation, and problem-solving; to the development of the capacity of self-regulation process of teaching-learning (TORRES; ALCÂNTARA; IRALA, 2014, p. 61), in other words, to develop their critical thinking and creativity by identifying their individuality and supplant their difficulties to reach their full potential.

In this scenario, the digital creates pedagogical opportunities by adapting to the needs of the education institution, a classroom, and of students with specific needs and individual development levels to expand their self-conscience, self-decision taking, meta-learning, and independent skills. The digital supports the teaching and learning processes because its use surpasses “[....] the educational approach centered on the teacher’s speech, on reading, and on student’s passiveness, who only answers the questions that were asked to them ” (^{BACICH; MORAN, 2017}, p. 15).

Concerning the importance of the digital in the pedagogical context, it is necessary that teachers appropriate technologies and, consequently, develop their digital competences, because “[...] if, in one hand, it might be considered relatively simple to equip schools with this technologies, on the other hand, this demands professionals who know how to efficiently use them in school practices (^{FIORENTINI; LORENZATO, 2009}, p. 46). The teacher needs to use new theoretical approaches, in which students are the center of the teaching and learning processes, being necessary to teach the search for knowledge focused on the appreciation of that knowledge. It should be acknowledged that digital integration in the educational context helps to prepare students for an increasingly digitized society, providing, thus, the mitigation of inequalities in digital competences.

Therefore, the main goal that guided the current investigation was to analyze the self-perceived digital competences of Portuguese and English teachers through the Check-In diagnostic tool. We developed a teacher training course, after detecting the teachers’ biggest weaknesses in the area of assessments through digital, aiming to promote the overcome of this digital competence in order to understand if those competences had improved after the training course.

It is important, then, to remember that the problem that guided this investigation was noticing at which levels of digital competences the teachers who participated in the study were and if the training course designed specifically focused on the assessment area allowed them to develop and advance on the digital competence in this area.

At the beginning of the study, the teachers were mostly on the Explorer level of assessment area, focus of the training course; after the course, most of the teachers were, still, on the Explorer level. However there were more teachers on the Integrators level as well as it was possible to see the appearance of teachers on the Expert level (Figure 2), which shows that the training course allowed the development of digital competence in this area, in other words, the results prove that there were increase and improvement on proficiency level on assessment, but not only on the assessment level, since there were also improvements in the areas of digital resources (Area 2) and teaching and learning (Area 3) (Table 1), for example, once the competences are transversal and the six areas interrelate.

Concerning the global level (Figure 3), there was a change in the teachers’ digital competences after the training course, since most of them are “integrators” and “experts”, besides the fact that more teachers became “leaders”, those who know, through a great variety of digital strategies, select the most suitable for a certain situation and update continuously through interactions with friends, besides being an inspiration to others.

Thus, Check-In is a tool to improve teachers’ digital proficiency once, with the self-knowledge of the level of digital competency, it helps teachers to improve their perceived competence to teach and learn. Its importance is due to having diagnostic information for the planning of the training course since it, due to the high integration of the digital in education, became necessary for educators to have specific knowledge, new abilities, and competences. Therefore, the importance of using tools like Check-In, which provides factual data to base proposals of pedagogical courses for the use of digital technologies in learning environments more suitable to the educators’ needs and difficulties.

Check-In is the beginning of a training course process that helps trainers to meet the educators’ specificities so the course matches the teachers’ needs at work and their levels of digital competences. Therefore, it should “[...] be forecast as an initial and necessary phase of a continuous process of professional development and deepening of the pedagogical practices with the TIC, aiming to improve the students' learning” (^{COSTA, 2008}, p. 114).

Therefore, Check-In was a stimulus tool to build a new profile of a teacher who uses the digital to teach and evaluate students formatively, thus, it helped to improve the teachers’ competences in this area as well as in others.

Among the limitations of this study, it could be mentioned the small amount of participants teachers, which would not allow the generalization of the obtained results besides the fact that the results obtained via the applied questionnaire concern the teachers’ perceptions and opinions only on a specific time, since they, accordingly to the conditions, may alter. It is also recognized that the Check-In questionnaire only shows, through the answers, the teacher’s self-perception, which does not allow an evaluation of their digital competences in practice.

Lastly, it is worthy to highlight the importance of evaluating digital competence to teachers, who must subsidize formation and strategies aiming to improve teaching work.