CASE STUDY: MODEL DEVELOPMENT AND APPLICATION

The model proposed in this research was built based on good practices evidenced by ^{Pereira et al. (2019}). The elaborated structure aimed to contribute to the process of mediating the structuring of academic papers through VLEs. Thus, a case study was carried out in the first quarter of 2018, in the discipline of Scientific Research Methodology, of the Graduate Program in Information and Communication Technologies at the Universidade Federal de Santa Catarina.

The elaborated structure was applied on the Moodle platform, with some actions being carried out based on the good practices shown in Chart 2.

The first good practice used was the advance planning of the discipline, which addressed the organization of content, technologies employed, assessment and teaching and learning method through the PDCA continuous improvement model (^{Lopez and Eldridge, 2010}; ^{Logan, 2012}; ^{Merillat and Scheibmeir, 2016}). In addition to its use for the organization of the discipline, the PDCA² method was applied to structure, in stages, the preparation of the students’ academic work, through the performance of four steps (Plan, Develop, Correct and Act) described in Figure 2.

From this organization, the contents were taught in face-to-face and online meetings through the Moodle platform. Its various digital resources and tools enabled the use of feedback mechanisms, which is considered a good practice by ^{Robb and Fisher (2015}), ^{Tuffley and Antonio (2015}) and ^{Laflen and Smith (2017}). Thus, some tools that make it possible to send feedback online were integrated into the developed model, highlighting the evaluation laboratory resource.

The evaluation laboratory tool consists of a peer evaluation mechanism, enabling, in addition to sending open feedback, through suggestions, the elaboration of parameters for work evaluation (structured feedback). This resource can also contribute to issues related to interpretation skills, critical analysis and the writing process, influencing the student’s interaction with the digital environment in a collaborative way.

In the proposed model, this resource was applied in the research question elaboration phase. This practice aimed to contribute to aspects related to clarity, objectivity and interpretation of the research problem, since these parameters were examined by colleagues. In the evaluation phase, the research question was analyzed, firstly, online and, secondly, in person, to assess the veracity of the initial interpretation carried out through digital writing.

In order to influence learning collaboratively, activities were carried out through the discussion forum (^{Peachey, Jones and Jones, 2006}; ^{Logan, 2012}; ^{Lai and Savage, 2013}; ^{Stockleben et al., 2017}). The proposed dynamics aimed to encourage research in the repository of the Universidade Federal de Santa Catarina, familiarize students with the structure of scientific works and encourage the sharing and combination of such information. In addition to the forum, another resource used collaboratively was the glossary. The practice applied in such technology consisted in the shared elaboration of a “dictionary” for the methodology discipline.

Aiming to innovate in the organization and structuring of the virtual platform, the continuous improvement model SECI elaborated by ^{Nonaka and Takeuchi (1997}) was used, which allowed the redesign of the VLE structure, and was also a good practice identified by ^{Lai and Savage (2013}), ^{Power and Kannara (2016}), ^{Linder, Bruenjes and Smith (2017}). The didactic sequence elaborated based on the researches by ^{Pereira, Spanhol and Lunardi (2018}) aimed at greater interaction between students and the digital platform. To this end, in addition to various resources and tools (e-activities), content in different formats (videos, text files, web pages, images, animations...) were used (^{Pavey and Garland, 2004}; ^{Uys, 2010}; ^{Merillat and Scheibmeir, 2016}; ^{Stockleben et al., 2017}) (Figure 3).

Thus, the use of VLE tools and resources for the development of activities provided the optimization and time management of students and teachers (^{Lai and Savage, 2013}; ^{Robb and Fisher, 2015}; ^{Tuffley and Antonio, 2015}; ^{Richardson and Watts, 2005}; ^{Laflen and Smith, 2017}). In addition to such practices, other actions were implemented, such as the development of a discipline guide (^{Logan, 2012}). This manual was prepared using the “Book” resource, with all the information on the subject being made available: schedule; teaching plan; discipline methodology; assessments; Moodle organization and tools. In addition to these items, tutorials on the use of the platform’s tools were prepared in the same resource.

From the model developed, it can be seen, as illustrated in Figure 4, the interdisciplinary, cyclical and continuous relationship established between the main constructs of the research. The good practices identified in the literature aimed to assist in the process of interaction with the VLE and, consequently, in the mediation of such works. Among the practices, the ways of knowledge creation stand out, which enabled the elaboration of sequences, activity routines and the PDCA model, which contributed to the planning and improvement of the process of developing academic papers.

DATA COLLECTION

For data collection, an online instrument (Google Forms) was used, consisting of 20 statements (in which the participant informs the degree of disagreement or agreement with them) and three (3) objective questions, applied to the actors involved in the period from May 4 to 19, 2018. The statements were separated into six groups (P6) corresponding to the good practices applied in the VLE, being structured based on the Likert scale of five (5) levels:

The purpose of this investigation was to verify whether good practices contributed to mediating academic work in the VLE.

ANALYSIS OF RESULTS

To analyze the collected data, a qualitative approach was adopted. According to ^{Gerhardt and Silveira (2009}, p. 31), this method “is not concerned with numerical representation, but rather with deepening the understanding of a social group, an organization etc.”

In addition to the qualitative research, Cronbach’s Alpha was applied to estimate the reliability of the assertions made, according to ^{Cortina (1993}). This method was developed by Lee J. Cronbach in 1951, and aims to measure the degree of reliability of the responses obtained. Thus, for its application, Equation 1 was used:

Where:

According to ^{Hora, Monteiro and Arica (2010}), Cronbach’s Alpha values should be in the range from 0 to 1, but there is no consensus among several researchers to interpret the reliability of the questionnaires based on the coefficient value. In their research, the authors show the minimum value considered acceptable for the instrument’s reliability of 0.70, and the maximum value cannot exceed 1.00.

CRONBACH’S ALPHA

As already mentioned, Cronbach’s Alpha “aims at greater precision in measurements, consequently, greater reliability in the instrument used for data collection” (^{Hora, Monteiro and Arica, 2010}, p. 3). In this sense, the formula described in the previous section was applied to measure the Cronbach’s alpha coefficient of the statements corresponding to the six good practices (P6) applied in the Moodle platform, obtaining the values presented in Table 1.

The results ranged between α=0.74 and α=0.90, being correlated to the range of values shown by ^{Hora, Monteiro and Arica (2010}), as acceptable for the reliability of the questionnaire developed. Thus, the validity of the responses obtained, explained in the course of this research, is highlighted.

QUALITATIVE ANALYSIS

Good practices are considered, in this research, as the most effective way to carry out a specific activity (^{APO, 2009}; ^{Lopez and Eldridge, 2010}). Thus, with a view to making the use of VLEs resources and tools effective to mediate the structuring of academic papers, some good practices evidenced by ^{Pereira et al. (2019}) on the Moodle educational platform were adopted. To verify the contribution of these best practices in the mediation process of the course completion papers, an electronic data collection instrument was applied to the 29 students of the Scientific Research Methodology discipline (2018/2001 class), obtaining feedback from 23 participants in the period from May 4 to 19, 2018.

The first good practice analyzed encompassed the use of adequate feedback mechanisms. According to ^{Laflen and Smith (2017}), with the integration of ICTs into the academic scenario, online communication in written form has become increasingly constant. In this sense, it is essential to use feedback resources effectively. From this context, peer review dynamics were integrated into the proposed model. This collaborative activity model enables colleagues to provide feedback on the analyzed work, through online writing or in person.

Thus, according to the tabulated data, approximately 78.3% of those involved (43.5% CP and 34.8% CT) reported that the feedback received (in person or through the peer review resource) was relevant and contributed to the structuring process of their research. In this same statement, 4.3% neither agreed nor disagreed (N) and 8.6% (4.3% DT, 4.3% DP) disagreed, stating that the feedback received was irrelevant, not contributing to this stage.

In addition to this assertion, the research aimed to investigate the use of the assessment laboratory tool as a mechanism to enhance feedback. According to the data, 87% of the students (43.5% TC, 43.5% CP) indicated that, based on the proposed activity (peer review of the research question), this resource was used effectively. 4.3% neither agreed nor disagreed, and 8.6% (4.3% DT, 4.3% DP) considered that this resource was not used efficiently in the course (Table 2).

These data demonstrate the effective use of feedback mechanisms in the applied model, emphasizing the use of assessment laboratory technology in the peer review dynamics. According to ^{Tuffley and Antonio (2015}), the use of technologies to send such feedback enables: clarity, simplicity and greater objectivity of the items to be analyzed; legibility of the return; orientation of deficient aspects; and identification of items in which the student performed well and which ones need improvement. In this sense, it is inferred that the actions taken contributed to the technological mediation of the scientific research structuring by academics (specifically in the research question), through feedback mechanisms.

In addition to the effective use and contribution of feedback, the research aimed to identify which types of these mechanisms the students considered of greater importance. The highest percentage was the individual face-to-face feedback (30.4%). In addition to this, there are also feedbacks sent through individual and collaborative tools of the VLE (both with 21.7%). Collaborative face-to-face feedback obtained 17.4%, and only 8.7% reported using the mechanisms through videoconferences (Graph 1).

From these data, it is noteworthy that, despite the ubiquity of ICTs and their integration in the academia, it can be seen that face-to-face feedback is still considered of greater relevance by students. Even with the effective use of digital technologies, the difficulty in obtaining a digital writing process as effective as face-to-face interaction is evident. However, it was found that the use of these Moodle mechanisms either through individual or collaborative tools obtained good student acceptance (21.7% in both). According to ^{Tuffley and Antonio (2015}), the feedback mediated by ICTs has been gradually applied in higher education (face-to-face and mixed) since, in addition to being flexible when it comes to sending and reading it, it can also be stored online for reference or future assessments.

In addition to feedback through digital writing, another good practice applied in the exposed script included the use of collaborative digital tools. ^{Stockleben et al. (2017}) emphasize that collaborative learning ensures quality in teaching and learning processes through shared knowledge construction. For this purpose, it is necessary to further study the digital tools and structures that enable such models (^{Stockleben et al., 2017}).

Thus, the applied practices addressed the use of collaborative tools, discussion forum, assessment laboratory, glossary, among others. According to 78.2% of the students (47.8% TA and 30.4% PA), the activities carried out through such resources encouraged the construction of knowledge in a collaborative way, enabling the discussion of ideas and externalization, exchange of experiences. On the other hand, 13% did not opine (N) and 8.7% of the participants totally disagreed with such information.

Another action analyzed guided the use of the discussion forum tool. According to 60.8% of the academics (39.1% TA and 21.7% PA), this resource made it possible to share information and encourage consultations in the databases of the Universidade Federal de Santa Catarina, contributing to research planning. In this same statement, 17.4% had no opinion (N), and 21.7% (13% PD and 8.7% SD) reported that the proposed practices, through this tool did not help in the research planning process.

Regarding the use of the glossary resource as a collaborative tool, 52.2% of the students (26.1% TA and 26.1% PA) reported that it was used effectively and contributed to the construction of knowledge in a collaborative way. However, 34.8% did not opine (N), and 13% did not agree with the statement (4.3% PD and 8.7% SD). It is worth highlighting the high percentage of academics who were neutral regarding the use of the glossary tool in action P2.3 (Table 3).

From such data, it is possible to evidence the effective use of the VLE collaborative tools. However, a significant percentage of neutral (neither agree nor disagree) and negative (partially disagree, totally disagree) evaluations regarding the use of the forum and the glossary was identified. The need to use new collaborative tools is evident, providing several opportunities for knowledge construction and enabling the involvement of students according to their learning style (^{Pavey and Garland, 2004}).

Another practice considered relevant by ^{Lai and Savage (2013}) refers to the organization of the VLE to encourage interaction between students and the virtual environment. In this sense, according to the quadrants of the SECI model, the resources and tools of the Moodle platform were organized. ^{Pavey and Garland (2004}) argue that, for greater interaction between students and the virtual environment, it is essential to make teaching material available in different media and to use the various VLE tools and resources in teaching practices (e-activities).

Thus, according to 82.6% of the participants (56.5% TA and 26.1% PA), the structure organized in Moodle (didactic content and digital activities) encouraged interaction between the student and the object of study. 13% (N) did not give their opinion, and 4.3% informed that such organization did not encourage greater contact with the digital platform. Regarding the provision of educational content in various formats (digital media), according to 78.3% of those involved (69.6% TA and 8.7% PA), such media contributed to the interaction between the student and the content available in the digital platform. In this same statement, 21.7% of those involved did not have any opinion.

In addition to the strategy for student interaction with the environment and content, some actions aimed at greater contact, exchange of information between student/student through the VLE collaborative tools. According to ^{Pavey and Garland (2004}), collaborative activities encourage creativity and enable different opportunities for knowledge construction. As per the tabulation of the data collected, 69.6% of the students reported that the use of collaborative tools provided greater interaction with colleagues (34.8% TA and 3.8% PA). 13% did not opine, and 17.3% did not agree with the assertion (13% PD and 4.3% SD).

The elaborated practices (proposed activities) also aimed to encourage greater interaction between the advisers. In the process of scientific research, monitoring and constant guidance by advisors is essential. The participants reported (47.8% TA and 30.4% PA) that the actions taken in the discipline contributed to greater contact with the advisor, once that, in order to carry out the activities, it was necessary to consult with the advisors. 13% (N) neither agreed nor disagreed, and 8.7% partially disagreed, stating that such practices did not encourage greater contact with teachers.

Another practice that aimed to encourage greater interaction between the student and the object of study for technological mediation was the preparation and availability of a discipline guide (explanation of activities, use of tools and resources and methodology of the discipline). This manual was developed in order to provide greater autonomy for students in the use of Moodle resources, also encouraging the improvement of digital skills (^{Salmon, 2000}; ^{Pavey and Garland, 2004}). According to the measured data, 69.6% reported that this resource (book) contributed to the use of digital technologies applied in the discipline (52.2% TA and 17.4% PA). 13% did not opine, and 17.3% did not agree with the statement (13% PD and 4.3% SD) (Table 4).

From the illustrated data, it was evident that the organization of the Moodle resources and tools collaborated to mediation and greater interaction between student/student, student/teacher and student/object. The different digital resources and the practices that encouraged greater contact with the advisor (activities aimed at student research) are highlighted as enhancers. However, further study is needed on the use of collaborative tools and the development of digital manuals (guides) to encourage the employment of such digital technologies. These guides can be built and shared by the students, also encouraging the study of the technology used, thus improving their digital skills.

Another item identified in this research refers, in the students’ opinion, to the activities that stimulated greater interaction between the student and the object of study. According to the participants, collaborative activities, through discussion forums, had the highest percentage (30.4%), followed by individual online and face-to-face collaborative activities, both with 21.7% of the students’ preference. In addition to such technologies, 17.4% of the students also indicated peer review activities and 8.7% indicated face-to-face individual tasks (Graph 2). From such data, there is an evident need to integrate peer assessment practices into the culture of educational institutions, as a creative method for knowledge creation (^{Pavey and Garland, 2004}).

In addition to actions to encourage interaction between the student and the virtual educational platform, another good practice, according to ^{Power and Kannara (2016}), covers the constant redesign and innovation of the structure of VLEs to mediate the construction of knowledge. Thus, taking such good practice as a reference, some strategies were carried out to redesign the virtual environment and systematize routines and activities from the SECI quadrants, presenting the Moodle virtual platform as a space for knowledge conversion.

To this end, the digital platform was structured, combining educational content in different formats of digital media, face-to-face meetings (socialization and externalization), collaborative (combination) and individual (internalization) activities in person and online (^{Pereira, Spanhol and Lunardi, 2018}). It is noteworthy that this sequence was applied to each topic covered in the discipline. According to 78.2% of students (56.6% TA and 21.7% PA), such sequences of actions contributed to improving knowledge of the items covered in class. On the other hand, 8.7% did not opine (N), and 13% disagreed with the assertion (4.3% SD and 8.7% PD).

Another procedure performed from such good practice was the systematization of routines and activities for planning the students’ scientific research process. According to 78.2% of academics (47.8% TA and 30.4% PA), this model contributed to deepening scientific knowledge on the subject of study. However, 8.6% did not agree with this statement (4.3% PD and 4.3% SD), and 13% neither agreed nor disagreed (Table 5).

Thus, one can see the importance of integrating different areas to the composite model, helping to innovate and redesign the virtual platform used. In addition to such information and based on the analyzed data, the need for further study and new proposals to systematize routines and activities is highlighted, with new research on the processes of socialization, externalization, combination and internalization in VLEs.

In addition to the redesign and innovation regarding the use of virtual educational platforms, according to ^{Robb and Fisher (2015}), it is essential to master their resources and tools, collaborating in time management. This theme was initially addressed by ^{Chickering and Gamson (1987}), in face-to-face teaching. After the technological advent and its integration into the academic scenario, ^{Robb and Fisher (2015}) address the principle of time management as a good practice in VLEs, aiming at the effective use of such technology to assist in academic performance.

Based on this good practice, the aim was to analyze whether the resources and tools were used effectively and whether they contributed to the students’ scientific research planning process. From the data analysis, positive feedback was identified by 82.6% of the students (52.2% TA and 30.4% PA), confirming the effective use of the Moodle resources and tools during the course, also helping to plan their research. In this same item, 8.7% did not opine and 8.6% did not agree with the statement.

The discussion forum, according to approximately 73.9% of the participants (30.4% TA and 43.5% PA), contributed to information sharing, optimizing time in the proposed activities. They disagreed with this statement and did not opine 13%. The evaluation laboratory and book resources are also noteworthy. According to the academics, the evaluation laboratory tool contributes to the objectivity and clarity of the research question (82.6% agreed with the assertion), with only 8.6% not agreeing with the statement and 8.7% not having any opinion.

According to the data, the resource book was used efficiently to detail the planning and methodology of the discipline (78.2% agreed with the statement). It also contributed to guidance on the methodology of the discipline, assessment, explanations of the Moodle structure, and guidance on the use of VLE resources and tools (73.9% agreed and 26.1% neither agreed nor disagreed with the statement).

In addition to the book resource, the glossary, according to 60.9% of the participants (17.4% TA, 43.5% PA), was also used effectively in its purpose of unifying and encouraging research on items directly related to the scientific methodology. 26.1% neither agreed nor disagreed and 13% disagreed with the statement (Table 6).

Such data prove the effective use of some technologies integrated to the Moodle platform. However, it is also highlighted that the method of use and application thereof can still be improved, especially the use of the glossary resource. It becomes necessary to carry out new strategies for its use, in order to optimize student time and encourage the search for items relevant to the subject of the discipline.

In addition to time management activities, another best practice encompassed actions for advance planning, good organizational skills, greater attention to detail, and better written communication skills. To this end, one of the strategies addressed the use of tools that encourage the writing of the main items of a research project. Such activities were proposed through individual and collaborative tools.

^{Linder, Bruenjes and Smith (2017}) emphasize the importance of planning, organizational skills, and improving communication skills through digital writing. Thus, according to 82.6% of the students (56.5% TA and 26.1% PA), the activities carried out in the VLE contributed to the writing process of the main items of the final work of the course (objectives, research question, thematic and delimitation, justification and interdisciplinarity). Only 4.3% did not agree and 13% neither agreed nor disagreed (N).

In addition to such practices, another action carried out based on the continuous improvement model (PDCA) was the division of the discipline into two (2) stages. The first phase encompassed the dissemination and sharing of content and various activities related to the planning of the students’ research (P). The second stage corresponded to the development (D) of the students’ final work. At this stage, students learn by doing, developing their own work.

After the end of the course, the work prepared by the student was sent to their respective advisor for corrections (C). After revisions, the considerations made by the advisors were made available to their respective mentees for changes and adjustments (A).

In this sense, in the opinion of 87% of those involved (60.9% TA and 26.1% PA), the stage for guidance and preparation of the final work also contributed to the delimitation and scope of the study. Only 4.3% disagreed with the statement and 8.7% neither agree nor disagree (N) (Table 7).

Once again, the integration of other areas of knowledge to innovate in the use of VLEs is noted as a good practice, thus contributing to mediating the structuring of academic papers. According to ^{Merillat and Scheibmeir (2016}), the PDCA method is a potential tool for application in the most diverse scenarios for educational purposes, contributing to the process of knowledge construction.

In addition to the six good practices (P6) and actions carried out, the research aimed to identify, in the students’ opinion, which VLE tools can contribute to improving communication and writing skills. The tools with the highest percentage were the discussion forum (60.90%), online task (56.5%), and the evaluation laboratory (52.2%). In addition to these, the questionnaires (47.8%) stand out with considerable percentage, as well as surveys (43.5%), wiki and glossary (30.4%). Lesson and chat tools only got 17.4% (Graph 3).

This question highlights the importance of the effective use of collaborative tools: discussion forum and evaluation laboratory. According to the participants, such instruments can contribute to the process of communication and digital writing. Individual tools such as the survey, questionnaire, online task, glossary and wiki were also found to be relevant in this process.

The indication of such resources confirms the applicability of some good practices addressed in this study, such as the organization of VLE resources for greater student interaction with the environment and the use of various platform technologies for better time management. Another issue that deserves greater attention is the low percentage of lesson and chat resources, making further studies on these tools necessary to contribute to the digital writing process.

In addition to these tools, another essential item identified that contributed to the positive results of the implemented research is linked to interdisciplinary issues. The prior planning of activities and routines, as well as their systematization (PDCA and SECI), enabled the effective use of the VLE’s digital resources to mediate the structuring of academic papers.

However, the research also identified gaps in some of the best practices addressed. One of them was the divergence between the Learning Analytics results, referring to the number of accesses to the digital content available and the assertions about the contribution of such media for greater interaction with the virtual platform. The logs registered low interaction of students with the teaching material in different formats, a number less than one access per student. However, according to the academics, the availability of the subject’s content in various media encouraged greater interaction with the VLE.

It is worth studying this practice in detail since its application did not promote greater interaction with the platform. This gap may be related to changes in the way knowledge is built, as mentioned above. Another possibility addresses issues related to digital skills. It is inferred that only the availability of content in other media does not meet the new student profile. It is necessary to integrate mechanisms such as blogs, websites, wikis, among other collaborative networks to make such content available.

Another item highlighted as pertinent was the difficulty in the communication process through digital writing. According to the data, academics prefer face-to-face communication. An alternative indicated in the survey, through good practices, is the encouragement of peer review activities through digital technology. Such dynamic enables the improvement of the process of reading, interpreting and digital writing.

Thus, in general terms, one can see the contribution of good practices to mediate the structuring of academic works, through the VLEs and the effective use of the tools applied in the discipline. Collaborative practices are emphasized, through the use of discussion forum and evaluation laboratory resources. Such technologies enabled the peer review process, contributed to mediating the development of work and encouraged the sharing of knowledge, enabling “learning by doing”.

Based on the applied practices that obtained positive results and as a result of this study, good practices are recommended for the effective use of VLE resources and tools to mediate the structuring of Academic Works, as described below.

BEST PRACTICE RECOMMENDATIONS

Based on the results of this research, the importance of the study regarding the method of use and application of ICTs in academia can be seen, since such resources can foster the process of mediation for the construction of knowledge (^{Pavey and Garland, 2004}; ^{Becker et al., 2017}). The effective use of VLEs is still a major obstacle to be faced by educational institutions, given that the digital skills of those involved (professors/instructors/tutors) are often not sufficient for mastery and knowledge of the resources and tools available.

Technological convergence enables different teaching and learning models, but the mediation of more complex processes that demand greater interaction between the actors involved, such as the preparation of academic papers, is an issue that needs innovation and proper monitoring (^{Álvarez, Beltrán and Valdehíta, 2017}). Thus, aiming to contribute to the effective use of ICTs available in the VLEs to mediate the structuring of final works, the research presents the following good practice recommendations (Chart 3).

It emphasizes the importance of combining interdisciplinary methods, such as the systematization of routines and activities, based on the SECI model, from the space for conversion of knowledge in digital environments and the prior planning of the discipline from the PDCA continuous improvement model. It is also noteworthy that such integration, in addition to encouraging students’ interaction with the VLE, ​​enabled innovation in terms of organization and structuring.

Another issue that deserves to be highlighted is the domain of the technologies used, contributing to the effective use and management of academic processes, optimizing the time of the actors involved. To this end, digital teaching competence is eminent for the application of technological resources and guidance to students regarding the use of technology according to the dynamics developed.

In this sense, in addition to planning and technological mastery, best practices address the use of collaborative tools for knowledge construction. Such action is essential, as it encourages the development of the students’ critical view, making them independent in the teaching and learning process, also contributing to communication skills through digital writing. Thus, the importance of planning, innovation, organization, collaboration, optimization and interaction for the effective use of VLEs in the process of mediating the structuring of academic papers is emphasized.