Inglês (pdf)
Artigo em XML
Tradução automática
Enviar este artigo por email
Permalink
INTRODUCTION
The autonomy to develop ideas for an education that is consistent and with the ability to forward transformative action in the students, both with regard to changes in thought as well as action in the world, is a necessary condition for the good teacher. However, this is a complex attribute to be developed and, for this reason, it needs to be a continuous process, especially in teacher training courses. In the period of initial formation, educational disciplines are committed to the development of such a competence. Examples that provide the experience of an autonomous approach are essential, and are offered through exercises that require increasingly higher levels of independence.
The research presented here was conducted in a teaching module, entitled Didatica, presented to the future teachers of different fields of knowledge, in a private University, located in the South of Brazil. It is a compulsory unit in the curriculum and it has been constructed using three elements of pedagogy; learning, teaching, and assessment. In this study the focus was on the evaluation of learning. This is a basic requirement in teaching practice and the proposed activities must generate in undergraduates an appreciation of the concepts and methodologies that help in the exercise of their professional duties, so that they may properly evaluate the work and progress of their future students.
There are many possible ways of developing the content, and one of the approaches would be, for example, the teacher trainer starting with an explanation of different pedagogical models and the associated forms of evaluation. However, with the intention of stimulating the thinking of future teachers, this unit opted for a different form of activity. Starting with the students' own concerns, expressed in the form of questions, and progressing to the theoretical study of evaluation. Such an approach is in accordance with the ideas of Freire (1985, p. 49) who postulates that all questions from students are valid because, in addition to contributing, the student also becomes more familiar with asking questions, the exercise provides that "educating advances, discovering a dynamic relationship, strong, alive between word and action, between word-action-reflection". This exercise is therefore advisable when the students are future teachers.
In the exercise under consideration, students were asked to formulate questions about evaluation, expressing their doubts and aspects that concerned them in relation to the theme, and what they consider essential learning for the operation of good evaluation. These questions constitute the corpus of the research now presented, which had as it’s main objective, to understand what are the questions/concerns of future teachers about the process of evaluation of learning. The analysis resulted in three final categories: Principals and concepts of evaluation; Management of the evaluation process; and Evaluation in particular groups. Also, it was intended to demonstrate the potential of Discursive Textual Analysis as a method of analysis, as suitable for use in research in the field of teacher education.
The article is organized into four sections, in addition to the Introduction, in which are presented the reasons, context and objective of the research. In the second section Discursive Textual Analysis is considered as a method that maintains effective analysis. In the third section elements of the methodology of research are explained, such as the type of study and the participants. In the fourth section are the results of the analytical process, showing the preliminar classification of questions prepared by the students and the considerations of the three final categories, which emerged in the process of analysis. Lastly, in the Final Considerations the main findings of the study are presented, along with the implications of these findings for the field of teacher training.
DISCURSIVE TEXTUAL ANALYSIS
Qualitative research is characterized by the freedom of the researcher to define the best paths to follow in the undertaking of their study. Such freedom, however, is not to be confused with an absence of the criteria that confers validity on the research. For this reason, throughout the study, the design of the research is reported in detail to make explicit that criteria constructed by the researcher (OLABUENAGA, 2007; DENZIN, LINCOLN, 2006; FLICK, 2009). Information on the chosen method of analysis is an essential element in this description since, in the field of qualitative research, several methods may be adopted for the analysis of empirical material. The method ‘Discursive Textual Analysis’ (DTA) has been chosen, a method widely used in Brazil, especially in the area of teacher education and in the training of science teachers, the context in which the creators of DTA work. At present, authors such as Lima, Ramos, Gessinger (2014); Ramos, Ribeiro and Galiazzi (2015), Souza, Galiazzi (2016); ENS, Ribas, Favoreto (2016) use the method and advance its use.
Moraes and Galiazzi (2007), argue that DTA rests between Content Analysis and Discourse Analysis. According to the authors, first, DTA incorporates assumptions that allow the organization to set guidelines to perform the analysis. Secondly, DTA takes the emphasis on interpretation of the phenomenon, from the perspective of Gadamer (1984).
Some conditions are necessary to ensure that Discursive Textual Analysis is applied correctly, the most prominent being: (i) immersion of the researcher in the empirical material and social discourse; (ii) organization of ideas to create categories and (iii) extrapolating beyond the empirical data (LIMA, 2019).
The first condition, immersion of the researcher in the empirical material and social discourse, is the premise for the emergence of new understandings of the phenomenon studied. It is the deconstruction of ideas belonging to the initial text (unitarization) from which emerges other possible views (categorization). The intense and deep involvement of the researcher with the empirical material constitutes an essential condition for this reorganization. Still, the opportunity for the emergence of new understandings is linked to elements of the social context in which the phenomenon occurs and also the theoretical repertoire and worldviews assumed by the researcher regarding the social context in which he or she is inserted.
The second condition, organization of the ideas, is a requirement for the creation of a system of categories. There are many possible combinations of the ideas available in the corpus. New associations between them are generated by the organization of knowledge by the researcher in relation to the object of study. This self-organization is determined by the interpretation of the ideas available in the empirical material. It is the researcher who chooses from amongst the different possibilities of arrangements and, by making this choice, he constructs a system of categories, thus being able to add new understandings to the phenomenon under examination.
The third condition, extrapolation beyond the empirical data, is essential to the advancement of knowledge production. Understanding the elements of the research which helps to expand existing theories, allowing renewed understandings of the phenomenon investigated, is one of the reasons that justifies the research. The researcher must stand back from the situation in which the events have been studied, thus providing the premise for the development of new and original arguments.
With regards to the implementation of the method, the analytical process is organized around three main axes, each presenting further developments. They are: Unitization, Categorization, and Production of the Metatext - the new text produced from the analysis of the categorization of the research material.
In the first instance, Unitization is the deconstruction of texts to identify and isolate important ideas, significant for meeting the goals of the research. The disassembly of the textual material results in fragments called "units of meaning", each featuring information about some aspect of the object of study. According to RAMOS, RIBEIRO, GALIAZZI (2015), after deconstructing the initial texts it is important that the researcher rewrites the units of sense as a strategy to immerse themselves, to deepen their understanding of ideas already present.
Then follows the Categorization. In this, the units of meaning are grouped into initial categories, based on the establishment of relationships between ideas, both in terms of convergence and dissonance, focussing on the topic of research. These categories are referred to as emerging, as they arise from the corpus. However, it is also possible for the researcher to work with a priori categories, from previously established frameworks, which are strong enough to understand the phenomena being studied.
The grouping of the ideas gives rise to the initial categories, and in turn to the intermediate and final categories. It should be noted that in the whole process the authorship is evident, because the groupings of ideas are decisions that the researcher takes, consistent with their concepts and on the basis of the goals formulated for the investigation. This system of categories is expressed in the metatext, the new text, produced by the researcher, bringing together the insights, theoretical contributions and, principally, the interpretations made. This guarantees the authorship of the text.
In other words, the metatext is the presentation of the central and secondary arguments, expressing the analysis performed by the researcher. From a particular set of results, he can generate theoretical propositions applicable to other contexts. Yin (1984, p. 39) calls this "analytical generalization". Therefore, the elaboration of the metatext itself is a dualistic process that involves not only the physical and psychological efforts - in their emotional, behavioural and cognitive dimensions - but also encompasses activities that foster in educational institutions the process of engaging students in the process of teaching and learning (Table 1).
Table 1 RELATIONSHIP BETWEEN CONCEPTS AND STAGES OF DTA
| Fundamental Concepts | Associated Assumptions | Steps in the analysis process |
|---|---|---|
| Immersion | Emergence of new meanings Reorganization of initial ideas | Unitization and Categorization |
| Self-organisation | New combinations of ideas Expansion of knowledge about the object of study | Categorization |
| Extrapolation | Beyond the empirical field Expression of new understandings about the phenomenon | Metatexto |
Font: The autors, 2020.
METHODOLOGY
The research has a qualitative approach (TRIVIÑOS, 1987; DENZIN, LINCOLN, 2006; FLICK, 2014; STAKE, 2010), the case study form (YIN, 2013). The subjects of the research were 47 undergraduates who attended the unit in Didactics. The students belonged to the following disciplines: Physical Education, Portuguese Language, Biology, Geography, History, and Physics.
The data was collected through a specific activity in which the undergraduates presented, in writing, five questions, expressing doubt and/or concerns about the process of the evaluation of learning.
235 collected questions formed the corpus of the study. The analysis of the material was conducted using the method of Discursive Textual Analysis (MATHUR; GALIAZZI, 2007) and each question was considered as a unit of meaning. Afterwards, units of meaning were grouped by similarities, resulting in 8 initial categories. The quantitative distribution of the material is presented in Table 2 and the titles of the original categories are in question form.
Table 2 INITIAL CATEGORIES ORGANIZED FROM THE QUESTIONS OF THE GRADUATES
| INITIAL CATEGORIES | NUMBER OF QUESTIONS |
|---|---|
| I How to define instruments for evaluation? | 152 |
| II How to carry out formative evaluation? | 23 |
| III How to evaluate students with special educational needs? | 21 |
| IV How to evaluate group activities? | 9 |
| V How to evaluate learning when the classes are large? | 9 |
| VI How to assess attitudinal content? | 9 |
| VII How to deal with cheating? | 7 |
| VIII Can self-evaluation be considered a good evaluation tool? | 6 |
| TOTAL ASSESSMENT QUESTIONS | 235 |
Font: The autors, 2020.
In the process of analysis, the 8 initial categories, through a comparison of their constituent elements, allowed for the organization of three final categories, as shown in Table 3.
Table 3 FINAL CATEGORIES
| Categories | Initial categories (subcategories) | Number of questions |
| I - Principles and Concepts of Evaluation | * How to carry out formative evaluation? * Can self-evaluation be considered a good evaluation tool? * How to evaluate attitudinal content? | 23 6 8 |
| II - Management of the Evaluation Process | * How to define instruments for evaluation? * How to deal with the cheating? * How to evaluate group activities? | 152 7 9 |
| III - Assessment in Specific Groups | * How to carry out the evaluation of learning when the classes are large? * How to evaluate students with special educational needs? | 9 21 |
| TOTAL 3 categories | 8 subcategories | 235 |
Font: The autors, 2020.
ANALYSIS AND DISCUSSION
The group of categories that emerged allows us to say that the students' concerns vary from ways of acting in accordance with the principles and concepts of evaluation that they study in theoretical references, and expressed in category I, to questions about how to proceed when the groups have specific needs, as presented in category III. Furthermore, the questions of the students are also related to methodology, as revealed in category II.
It is important to emphasize that in the construction of the texts of analysis, the metatexts, some units of meaning are cited to exemplify and strengthen the arguments developed. Equally important is that questions from all of the students are represented in the analysis.
Category I: Principles and concepts of evaluation
The future teachers, in general, experienced the traditional model of evaluation as school pupils. This model, called by Luckesi (2011) Pedagogy of Assessment, follows a general principle of assigning a supposed mark of achievement, a numerical value, which is the result given from a test which covers the material learned. When confronted, in the study of their discipline, with another possible form of evaluation, which is broader, taking into consideration the gradual development of learning, they were then surprised, sometimes insecure, as can be seen in the following statements: “What types of evaluation, besides the distribution of grades, are applied by the best teachers ?; How to move away from this method of evaluation that values the grade and not the knowledge gained ?; How to make students take seriously an alternative assessment, other than grading?".
As a result of these reflections, the future teachers also ask about how the formative evaluation works in practice, as can be seen in the set of questions below: Is it possible to evaluate each student individually without using a pre-established form of analysis? How to compensate if an evaluation activity ends up having a negative impact? What are the different means of evaluation and how best to use them? Can simply grading work make students more sloppy?
The students' questions show the understanding that the function of evaluation is classificatory, that is, they understand evaluation as a product and not as a process, establishing an average as a parameter to classify students and using the exam as the only evaluation instrument. As a result, it is understandable that they present difficulties in putting into practice a conception of evaluation as a dynamic process, in which different instruments of evaluation are used (JOHNSON, JOHNSON, 1999).
For Lima and Grillo (2010), the process of evaluation and the product of evaluation are different instances, but they coexist. In general, at the end of a period, the teacher must make decisions regarding the promotion of the students, and a classificatory evaluation is necessary. However, this does not invalidate the formative function of the evaluation. On the contrary, the formative assessment will provide data for the teacher to make the decision more securely.
The undergraduates expressed further doubt and uncertainties and demonstrate through their questions an ignorance of the operation and effectiveness of self-assessment: Self-assessment, how does it work? When is the best time to apply a self-assessment?
Providing students with a reflection on self-assessment is important so that they can broaden their understanding of the formative function of evaluation. Authors such as Hadji (2005; 2016) and Tardif (2006) emphasize that self-evaluation promotes the development of metacognition, which enables an improvement in the regulation of learning. According to Grillo and Freitas (2010), self-assessment is a powerful way for the student and teacher to follow the progress made in learning. For the student, the process represents the opportunity for reflection and the organization of what has been learned, bringing to light the defecits or weaknesses in the understanding of the concepts involved. For the teacher, the self-evaluation indicates the points requiring strengthening and extra help, so that the student, through metacognition, becomes aware of his learning and can make adjustments in the process.
In this sense, some of the questions from the undergraduates show an understanding that evaluation provides data about teaching and not only about learning: Do bad grades also reflect my performance as a teacher? At the end of each week, could I make an assessment with the students about how the classes were? The questions also demonstrate the possibility of using the evaluative process to follow the teaching process and make changes to it from reflecting on the diagnosis. Trainee teachers should understand and take on board the double meaning of evaluation, as a way to follow the learning of their students and as a way to collect information for developing their teaching strategies.
Another theme in the questions asked concerns the evaluation of an attitudinal order, that is, how the student participates and her attitude to class or studies. The following questions show this aspect: If the teacher evaluates with attitudinal criteria, how would these criteria be relevant to the final grade? What would be the value to be attributed to the attitudinal content in the student's grade? How do we assess student behavior?
The doubts expressed by the students are quite pertinent, because it is a complex subject that deserves close attention. All student learning can and should be evaluated (FINKEL, 2000). However, evaluating does not mean always assigning a grade. In the case of attitude, they can be evaluated and the teacher can give a written or oral opinion, without necessarily assigning a grade. According to Grillo and Lima (2010), when assigning a grade for student participation, for example, the teacher runs the risk of assessing the evaluation by the impression caused by the student and thereby may distort the true results.
To summaries, the questions asked by the students show some limitations regarding conceptions about evaluation and its guiding principles, probably as a consequence of their school experiences. However, another group of questions shows that some initial conceptions are being reconsidered, giving space for a new understanding about evaluation and generating a series of queries about how to put it into practice.
Category II: Management of the Process of Evaluation
In this category, we bring together the questions of the future teachers that seek to equip them to manage the evaluation of the learning of their students. They refer to the types of tools of evaluation, the possible ways of evaluating work in groups and also the ways of handling cheating. It should be noted that the greatest number of questions (168) appear in this section, with students clearly seeking answers that lead to concrete suggestions for dealing with the situations. The set of questions presented below exemplify the questions of how best to deal with tools of evaluation and differentiated evaluation procedures: Does the teacher need to use exams as a form of evaluation? How to determine whether the exam should be descriptive or objective? How do you know which subjects are most difficult to understand? Is it only through exams? How to evaluate practical activities? What should be taken into account in a practical assessment? To what extent is it feasible to do tests with ‘open book’ or in pairs? When doing work in groups or research, what exactly is evaluated? How to evaluate ‘play’ activities? What is the required number of assessments to form a good understanding of the student in all aspects of learning?
When faced with a new conception of evaluation, it is natural for students to express doubts as to how to put it into practice, especially with regards to the tools for data collection of learning that enable the formative function of evaluation to be exercised. In this sense, it is important that the future teacher has access to different evaluation tools, so that she can judge which is best suited to each situation when working with her students.
In other words, there are different instruments of data collection and the choice is clearly related to the desired purpose. In some situations, the test or exam may be the most appropriate, in others it may be a report, some work or another alternative. In this sense, to problematize the use of the test as the only evaluation instrument is fundamental to trigger new understandings about the different ways of collecting data about student learning.
It should be noted that the students' questions and their concerns regarding the implementation of the formative evaluation, already point to a new understanding about the evaluation process. This is an important development, since the first requirement of educational evaluation is that it be formative and that it helps the student to educate himself (PÉREZ GÓMEZ, 2015).
A substantial number of questions related to group work and the questions presented below express these concerns: What to do when conducting group work but only a few students are working? When group work occurs, should I intervene only if the students ask? How do you have an evaluation of an individual when they are engaged in group work outside the classroom?
The questioning evidences the understanding that the exam need not necessarily be the only evaluation instrument and that group work can also provide data about the students' learning. However, there are many doubts as to how to put this into practice, possibly because in their own school experience they were evaluated primarily by means of exams. Thus, it is important that in teacher training, spaces are created to experience new forms of evaluation and to discuss them, since it is known that new teachers tend to reproduce practices experienced during the training period.
Still within the context of the management of the evaluation processes, the future teachers expressed apprehension about the aspect of "cheating" and the following questions illustrate this concern: What to do if a student is caught? What to do when a student is cheating in a test? If a student denounces another for cheating, and the teacher did not see the act, how are these students to be evaluated?
The questions presented by the students point to the need to promote a reflection with the undergraduates about the relationship between evaluation and the didactic contract - that being the working agreement between the teacher and student in the classroom. According to Freitas and Gessinger (2008), in all teaching scenarios, a contract is presented that permeates the relations that are established. An explanation of this contract is important so that the student finds meaning in what she is studying or in the activity she is doing, otherwise there is a risk that she will study only to attain a good grade; and so the goal is focused on the grade and not on learning. In addition to guaranteeing a good didactic contract, the teacher can also suggest tasks that allow students to construct good arguments and, in this way, open book exams can constitute an instrument for this purpose (BERNARDO, 2007).
Finally, another aspect of the undergraduates' concerns is the search for actions that help them deal with differences in the classroom, because inclusion is a reality in Brazilian schools.
Category III - Evaluation in Discrete Groups
The future teachers expressed particular concern over the assessment of students in specific conditions, notably in the case of excessive numbers of students in the classroom. This is an issue where researchers fail to reach a consensus. For authors such as Hanushek (2002) and, more recently, Camargo and Porto Júnior (2014), studies cannot statistically demonstrate whether the number of students in the classroom affects, or not, the students' learning. Among the questions asked by the undergraduates were: How should we evaluate students in a large class? How to evaluate participation in a large class? How can we establish an evaluation method that at the very least, considers the skills of about 30 people in a group?
From the perspective of this study, the concerns in this area of these future teachers are understandable and appropriate, and, possibly, this was not an aspect that the teacher trainer considered when discussing the traditional classroom. From the questions, however, it was possible to present and discuss various forms which could be employed in large group evaluation.
In the reflection process, it was possible to determine some possible ways to improve the evaluation when there is a high number of students in a class. The first refers to the importance of engaging in many activities in class, because in this way the teacher can follow the development of the task in person and assess the involvement of each student. The second aspect concerns clarity in communication. Large classes require the teacher to reiterate information and provide clear guidelines for activities to ensure that all students understand what they are expected to do. A third point to consider is giving feedback to the student about their learning. Frequent feedback of progress, or not, in learning needs to be given to students in order to ensure formative evaluation. It is from this information that the students guide their studies in relation to the content that is being developed by the teacher.
In another perspective, a great number of questions were raised regarding the evaluation of students with specific needs, such as those with attention deficit or hyperactivity. This reveals the understanding that each special need has a certain form of evaluation, that is, students who have attention deficit should be evaluated in a particular way, those who have dyslexia in another way and so on.
The questions presented by the students also refer to the understanding that the only students who have their differences recognized are those who present some special need or difficulty. Therefore, they need a special evaluation by the teacher, as can be exemplified in the question posed by one of the students: Should I have another (less demanding) attitude towards the assessment of a student with special needs? Such questioning leads us to the ideas of Mittler (2012) who, in approaching school inclusion, states that it is not a question of introducing previously excluded pupils into ordinary schools, but changing schools so that they become responsible for the needs of all children. Therefore, school inclusion concerns all those who are not benefiting from the school and not just those with some kind of disability.
From the perspective of inclusion, it is important that each student be understood as a unique being, regardless of his or her characteristics. In other words, it is important to realize that difference does not refer to certain groups, but it is what constitutes us as human beings. Thus, the assessment assumes another dimension, taking the student as a parameter of himself and assuming that each one has different interests, learning styles, abilities and needs, and this must be taken into account in the evaluation. It is not a question of stipulating that everyone should learn the same content, in the same way and at the same time, but to follow the course of learning of each one, obtaining data to better understand the learning process and improve teaching. This presupposes taking each student as a parameter of him or herself, thus overcoming the idea of stipulating a standard reference.
In order for undergraduates to be able to understand evaluation in an inclusive perspective, it is necessary to deepen the understanding of the differences in the classroom. In this way, based on the understanding that equality and difference are culturally produced, Silva (2009, p. 97) proposes that the understanding of this production should be broadened because it is not only a social question, it must also be a subject of pedagogical and curricular concern. Therefore, spaces for questioning identity and difference, as well as the supposed homogeneity of specific groups, are fundamental in teacher training courses. It is understood that this can help the undergraduates overcome their concerns of evaluation in a class with students with specific needs, which is a recurring concern, expressed in questions such as: When we have a student with a specific requirement, should we assess using the same standard for all? How to evaluate equally the special needs student and the mainstream student?
The students' concerns show that some are still quite tied to the idea of standardization, probably supported by the belief in the homogeneity of groups, which needs to be overcome. Some students, however, seem to be already problematizing such ideas by questioning whether all students should be evaluated in the same way or what evaluative method could be adopted within a classroom that does not see them as a whole, but as individuals with their singularities.
The questions raised by the undergraduates regarding the evaluation in specific groups are quite pertinent and reinforce the idea, common to many teachers, that only the specialists have the necessary knowledge to teach and evaluate children with some type of special need. In contrast, Mittler (2012) states that most teachers have the knowledge to act in an inclusive way, what they lack is confidence in their own competence. This may be an important notion to be developed during the initial training course.
FINAL CONSIDERATIONS
In the present research, the discussion indicates possible avenues for exploration by teacher trainers. The research, which sought to understand the students' concerns about evaluation of learning, explains ideas that fall into two main perspectives. The first one, denoting that future teachers are concerned with issues related to the aspects described in the theoretical frameworks on evaluation. They want to know about the different tools that can be used to assess students, ways of incorporating attitudinal content in assessments, and ways in which results of student assessment can contribute to the review of their own teaching strategies. However, the undergraduates also ask about the purposes and principles of learning assessment.
The second is to advance aspects that may not be included in the areas for discussion as defined by the teacher trainers, but which represent fundamental issues for the undergraduate. When asked about cheating, or how to evaluate pupils with special educational needs, prospective teachers place the study of assessment in more everyday, less academic circumstances. Concerns about old issues, such as cheating and evaluation, are coupled with contemporary questions, such as the presence of students with special educational needs.
It is also worth noting that working with the students' questions, with their real concerns, is to make them participants in the development and unfolding of the research. In this sense, we affirm that this form of work is an exercise of authorship, which is one of the facets by which autonomy is expressed.
As to the implications of the results of the research, it is important to highlight (1) the importance of the teacher trainer being attentive to the emergence of topics of interest to the undergraduates, which will require the restructuring of the teaching strategy used by the teacher trainer; (2) the students' elaboration of questions is a way of exploring students' doubts and concerns about the concepts to be studied and (3) the potential of DTA to help understand the phenomenon investigated. The assumed method allows the researcher to develop statements that represent new ideas about the object of study. The use of Discursive Textual Analysis in the study carried out reinforces the validity of the method for research on teacher education. In this sense, we understand that our option for DTA is more meaningful than the mere exposition of a way of analyzing the data of an investigation, constituting the sharing of a pedagogical experience, characterized by the active participation of the researchers. This approach has allowed us to look closely at the immediate reality, without losing sight of aspects such as the larger national scenario in which the university is inserted.
