SciELO - Scientific Electronic Library Online

 
vol.34DESENVOLVIMENTO PROFISSIONAL EM COMUNIDADES DE APRENDIZAGEM DOCENTEFORMAÇÃO HUMANA, VISÃO DE MUNDO, DIÁLOGO E EDUCAÇÃO: A ATUALIDADE DE PAULO FREIRE E MARTIN BUBER índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Compartilhar


Educação em Revista

versão impressa ISSN 0102-4698versão On-line ISSN 1982-6621

Educ. Rev. vol.34  Belo Horizonte  2018  Epub 10-Jan-2018

https://doi.org/10.1590/0102-4698172839 

Article

INSTITUTIONAL PROGRAM OF TEACHING INITIATION (PIBID): PERFORMANCE OF FELLOWS VERSUS NON-FELLOWS1

Adriana Castro Araújo*

Wagner Bandeira Andriola1  **

Afrânio de Araújo Coelho1  ***

1Federal University of Ceará (UFC), Fortaleza - Ceará, Brazil


Abstract:

We report the results of a study with the objective of comparing the academic performance of fellows from the Institutional Program of Teaching Initiation (PIBID) with non-fellows, considering (a) the concept of the courses in the National Student Performance Examination (ENADE) and (b) the turn of operation of these courses. The studied population consisted of 2,193 graduates of the Federal University of Ceará (UFC), referring to the period 2009.2 to 2015.1, of which 1,924 were not PIBID holders (87.7%) and 209 were PIBID scholarship holders (12.3 %). It was verified that the higher averages of academic performance were associated to the alumni graduates of PIBID, regardless of the concept of the course in the ENADE, as well as the turn of operation of this. The results demonstrated the relevance of the PIBID to the training of university students, since substantial differences were identified in the quality of the alumni’s learning, expressed by the higher performance averages, compared to the graduates who were not PIBID scholars.

Keywords: Educational Assessment; Higher Education; Program Evaluation.

Resumo:

Relatam-se resultados de estudo com o objetivo de comparar o desempenho acadêmico de bolsistas do Programa Institucional de Bolsa de Iniciação à Docência (PIBID) com não bolsistas, considerando-se (a) o conceito dos cursos no Exame Nacional de Desempenho de Estudantes (ENADE) e (b) o turno de funcionamento destes cursos. A população pesquisada foi composta por 2.193 egressos das licenciaturas da Universidade Federal do Ceará (UFC), referente ao período 2009.2 a 2015.1, dos quais 1.924 não foram bolsistas do PIBID (87,7%) e 209 foram bolsistas do PIBID (12,3%). Constatou-se que as maiores médias de rendimento acadêmico estiveram associadas aos licenciados ex-bolsistas do PIBID, independentemente do conceito do curso no ENADE, bem como do turno de funcionamento deste. Os resultados demonstraram a relevância do PIBID para a formação dos universitários, posto que foram identificadas diferenças substantivas na qualidade do aprendizado dos ex-bolsistas, expressas pelas maiores médias de desempenho, em comparação aos licenciados que não foram bolsistas do PIBID.

Palavras-chave: Avaliação Educacional; Ensino Superior; Avaliação de Programas

INTRODUCTION

Consonant Sousa, Andriola and Lima (2016), the evaluation is a relevant theme to higher education, because it is circumscribed in the sphere of public education policies, becoming a strategic activity of public administration in relation to the processes of supervision and promotion of educational quality. In the same line, Andriola (2008) highlights that the specialized literature has focused discussions around iterant aspects about the thematic above-mentioned, among which stand out: the university autonomy, the expansion-evaluation-financing linkage, the universalization of the higher education, the institutional evaluation, the quality of education, the teacher performance evaluation and the student learning evaluation as a support for educational enhancement.

In relation to the learning assessment, it is important to highlight that the cognitive conception asserts that the learning can be characterized as being a qualitative and stable behavioral change, occurred in mental processes of memorization, comprehension, application, analysis, synthesis and judgement, among others (ANDRIOLA, 2012). It is worth highlighting, still, that the cognitive abilities characteristics of learning are associated with the contents or curricular objectives addressed in the training process of student training (PÉREZ GÓMEZ, 2009). Therefore, for example, the teacher of Portuguese Language, by means of any literary text, will be able to measure the degree of achievement of curricular objectives, such as the specified below:

  • To understand a literary text, which requires the cognitive capacity of comprehension;

  • To analyze explicit elements of a message, which it is evaluated the cognitive capacity of analysis;

  • To synthesize an original message into a second and simplified parallel form, which requires the cognitive capacity of synthesis;

  • To judge the literary value of a literary work, based on established criteria, which demands the cognitive capacity of application judgment.

As mentioned in the example, the teacher is evaluating the degree on which his students have achieved four educational or curricular objectives associated with the most diverse cognitive capacities, all components of learning (ANDRIOLA, 2004a).

Under this view, in the Federal University of Ceará (UFC) the assessment of student learning is regulates by General Regiment2, specifically on Chapter VI about the Evaluation of School Performance. In article 109 of the said document it is stablished that the evaluation of the performance will be done by discipline and, when necessary, from the perspective of the whole course, always including assiduity and efficiency, both qualifiers for themselves” (page 28). In the 1st paragraph of the referred article it is considered that the assiduity is understood as being the activities’ frequency corresponding to each discipline”. In terms of efficiency, referred in article 109 on the 2nd paragraph, it is mentioned as the “student’s degree of achievement in the studies developed in each discipline”. Further up, in article 111, it is referred that “the results of performance verifications will be expressed in grades from zero to ten, with, maximum, one decimal place” (page 29).

Resulting from school performance evaluation realized within each taken discipline by the learner, UFC calculates the Academic Performance Index (IRA), which is an institutional indicator synthesizer of this information, since it is about the average of the grades obtained by a student while attending the disciplines of its graduation course3, whose range of values range from zero to ten.

From a broader perspective, the Ministry of Education (MEC) adopts, currently, the National Examination of Student Performance (ENADE), whose main objective is to evaluate the performance of the graduates of the graduation courses, in relation to the programmatic contents, habilities and competence acquired during the academic formation. Posteriorly the results are transformed into a national indicator labeled as Concept ENADE, which expresses the quality of the course from the students’ results in tests that check the level of learning, in terms of technical and proper knowledge of the higher career (specific factor), as well as for the set of information about the general culture and the contemporary society (general factor) 4 .

It is seen, therefore, that both sistematics described above have the evaluation of learning as a relief indicator to express the quality of student formation (in case of the use of IRA by UFC) and the quality of the course in which the student is inserted (in case of the use of ENADE’s concept by MEC). In addition, it should be emphasized that the use of these informations is vital for educational managers, since the purpose of any systematic evaluation is to provide subsidies so that the responsibles for coordinating and planning educational actions can make decisions, aiming at its improvement (ANDRIOLA, 1999; PÉREZ JUSTE; GARCÍA RAMOS, 1989).

Therefore, for the managers Higher Education it is important to know the changes in students’ behavior regarding the acquisition of new learnings, new knowledge useful for professional practice and integration into a democratic society, subjected to the transformations arising from the technological and labor revolution (ANDRIOLA, 2004b; ANDRIOLA, 2014; ANDRIOLA, 2015). These referred ideals are added to the students’ learning through several systems used by Higher Education Institutions (IES), among which can be cited: the teaching, the scientific research, the application of scientific knowledge (extension), the teacher monitoring and the intership aimed to teaching practice. In this context, it is worth to emphasize the purposes of the Institutional Program of Teaching Initiation (PIBID).

THE INSTITUTIONAL PROGRAM OF TEACHING INITIATION (PIBID): HISTORY AND REPERCUSSIONS

PIBID is the fulfillment of a legal requirement, established by the country’s main educational legal order of the country, the Law of Guidelines and Bases of Education (LDB), in its article 62, §5º:

The Union, the Federal District, the States and the Counties will encourage the training of teaching professionals to act in public basic education through an institutional fellowship program for teaching to students to students enrolled in undergraduate, full undergraduate courses in higher education institutions (BRASIL, 1996).

In order to value the teaching profession, encourage the entry of teaching career into public basic education and attend different legal requirements regarding the training of teachers for this level of education, the minister of education, Fernando Haddad, signed the Ordinance nº 38 (BRASIL, 2007), deliberating rules to regulate and institute the objectives and other criteria of the program.

Over the years, PIBID has expanded considerably at national level. The program, which attended around three thousand scholarship holders in the year of 2007, reached almost 30 thousand in 2011 and currently has more than 90 thousand (BRASIL, 2010). According to André (2012), although a more comprehensive assessment has not been made about the impacts of PIBID, nor comparisons between professional graduates, punctual assessments have served to demonstrate good effects both related to the motivation of students to enter the teaching profession as well as the improvement of the continuing training of professionals who act as supervisors of the graduates.

Langer, Ribeiro and Schroeder (2013) related experiences of work carried out in State University of Western Paraná (UNIOESTE) between 2010 and 2012, disclosing situations not only about pedagogical and interdisciplinary practices but also interaction between university professors, school teachers and academic fellows who are part of the PIBID.

The research carried out by Araújo, Andriola, Rodrigues and Coelho (2016) in the Federal University of Ceará (UFC), using a sample of 84 graduates of PIBID during the period of 2009 to 2014, highlighted that PIBID made it possible for a considerable part of the fellows to feel safer to act professionally, to improve academic achievement, overcome certain difficulties and developed a taste of teaching. According to the authors, such indications give a positive and significant boost to the experiences arising from the activities carried out within the scope of PIBID.

We realize from the reports that the cooperation between school and university, propitiated by PIBID, has been contributing to the development of habilities and e to the theoretical and methodological deepening of teaching practices, increasing the actions in the scope of the bachelor’s degrees and collaborating for a new perspective and conception of initial formation focused on the professional practice (CIANI et al, 2013). Besides, there was a change in the conception of teaching by university professors, making them understand that the exercise of teaching requires a lot more than the mastery of content (SILVEIRA, 2013).

These aspects related to the benefits of PIBID for undergraduate students’ formation can also be found in the research done by Darroz and Wannmacher (2015) which aimed to identify the focus of physics graduates teachers’ learning, participants in the program’s subprojects in Brazilian state of Rio Grande do Sul. The referred authors verified, based on the reports of 32 graduates, that the scholars acquire various skills, feel more secure, are led to research and study more in order to solve problems they detect in the classroom during the program activities, as well as developing competences related to the exercise of teaching.

PIBID has also been successful in assisting the continuing education of teachers from the schools participating of subprojects. Through a study made by Fetzner and Souza (2012) about the interculturality and its contemporary challenges and the conservative and emancipatory perspective of school knowledge, it was found that, from an analysis of perception of the basic education professionals of schools, where are developed PIBID subprojects of Federal University of Rio de Janeiro State (UNIRIO), not infrequently schools consider their unique knowledge, thus, excluding itselves from the others. Thus, through the actions carried out by the fellows of teaching initiation and its supervisors, the program has been contributing to intensify cross-cultural perspectives normally overlooked by schools, although very important.

However, although PIBID has been presenting good results, especially regarding to the qualification of teacher training, combating the low professionalization and attractiveness of the teaching career, constituting itself as a great ally of the public school, the existing problems in the country’s higher education hinder the recruitment of more qualified teachers (ARANTES, 2013).

It is noticed that the success of PIBID depends on a set of factors that together influence future teachers, especially in the school context. Furthermore, it is worth highlighting, as relevant, that the way the fellows of PIBID relate to public school, to supervisors, to the pedagogical team and even with other teachers seems to make a lot of difference in the training process, and may influence positively or negatively in the choice of profession and work environment (LANGER; RIBEIRO; SCHROEDER, 2013).

PIBID IN THE FEDERAL UNIVERSITY OF CEARÁ (UFC)

According to Araújo (2015), the first UFC’s project contemplated the undergraduate degrees typified as priorities by the own edital of the Coordination of Improvement of Higher Level Personnel (CAPES), which are the Biological Sciences, Physics, Chemistry and Mathematics; in besides two more that were considered necessary: Philosophy, whose Law nº 11.684 (BRASIL, 2008) turned it into a compulsory discipline in the school’s curriculum, having, therefore, the need to qualify and encourage future teachers to act in these schools; and Portuguese-Letters, which should contribute to other disciplinary areas, improving the capacity of student’s comprehension, reading and writing skills.

It should be emphasized, however, that during the corresponding interim period to the beginning of the Project in UFC until the year of 2014, some impacts were observed and reported on the institutional projects subsequent to the first. Therefore, several significant contributions were considers, for example: the effectiveness of the relationship between theory and initial practice in teacher education, the improvement of the scholars’ academic performance in the various disciplines studied, the development of tools that help in the dynamization of pedagogical practices focused on the exhibition of contents, the improvement of orality of the graduates, the improvement of understanding the contents by high school students, the motivation of these students to participate in the proposed activities and, lastly, a greater interaction between coordinator teachers working in the subprojects of PIBID.

Based on the exposed content above, the study was conducted with the main objective of carrying out an evaluative study to allow comparisons regarding the academic performance of the fellows of PIBID with non-fellows, considering (a) the concept of the courses in the National Examination of Student Performance (ENADE) and (b) the operation shift of these courses. For the present study the measure of academic performance was adopted as the Academic Performance Index (IRA), which, as already mentioned before, is the average of the grades obtained by a student while attending the UFC undergraduate courses, whose scale of values is situated between zero and ten. From this point, the typologies of studies are discussed focused on educational evaluation and public policies.

EVALUATIVE STUDIES: PROPOSAL OF TAXONOMY

In the vast literature about educational evaluation and public policies, there are numerous attempts to classify the evaluative investigations (SHADISH, Jr.; COOK; LEVITON, 1991). According to the opinion of Capucha, Almeida, Pedroso and Vieira da Silva (1996), such investigations can be organized into, at least, four types, namely:

  1. Ex ante evaluation: consists in identifying needs and conducting the realization of feasibility studies about the performance of any activity whatsoever. Such actions should guide the formulation and the development of an activity, a program or a policy. It includes a goal setting, the scope or locus of application, the characterization of beneficiaries and fulfillment of needs (Bourguignon; Ferreira; Leite, 2003).

  2. Formative or procedural evaluation: aims to identify the degree of coherence and perfection of activities (or processes) that are being executes, based on the objectives that were determined a priori (ROSALES, 1984). It is about an activity that will determine the best way forward, what are the best strategies or processes to be implemented, with a view to achieving the objectives and the targets set. The essence of the evaluator’s job is to follow, observe and diagnose the processes inherent in achieving the objectives of the program, with views of ways to improve them, in many cases, concomitantly with its execution. This monitoring includes the diagnosis of possible failures of the instruments, procedures, contents and methods, as well as the adequacy of the activity, program or policy, to the target audience (SCRIVEN, 1994).

  3. Accompaniment and monitoring: in general, seeks to identify the degree of adequacy between the planned and the executed (Blankenberg, 1995). It is, thus, an action which aims to evaluate the efficiency of something, from the objectives and goals established a priori. In this context, the monitoring activities allow to intervene in the process of carrying out the activity, program or policy, correcting the directions each time that deviations are detected. Most of the time, this intervention requires initiative, creativity and alternative solutions to the obstacles encountered during the executive project (Carvalho; White, 1995).

  4. Summative, outcome or ex post evaluation: involves studies focused on the results of an action, activity, program, course, policy, or similar. The object is the product or the tangible consequence of a planned action, as stated by Scriven (1994). In this context, it should be noted that the comparative analysis of PIBID between fellows and non-fellows will make it possible to obtain diagnostic data, whereas, in its turn, will enable the planning of improvement actions and the decision-making by managers and program and public policy makers, with a view to adapting the latter to the public used, according to the ideas of Andriola (1999).

Thereby, and to end this topic, it is possible to resort to Therrien and Sobrinho (1983/1984), for whom the evaluation should enable the efficient planning of actions, with a view to ensuring the credibility (of the activity, program or policy) in the eyes of the users and before the society in which it is inserted. To this end, they stand out:

“The evaluation should show not only the sum of achievements, but, above all, the effect or impact of its presence and performance in the social context that gives it the reason to be. (...) At the same time that it contributes to the effective unification of the parties into a coherent and active whole, the evaluation consolidates the involvement commitments, because it recognizes the principles of democracy and joint responsibility” (Therrien; Sobrinho, 1983/1984: p. 19).

EVALUATION OF PUBLIC POLICIES: BRIEF THEORETICAL REVIEW

Considering the proposal of the present research, it is necessary to briefly discuss the evaluation of public policies, as well as to present what has been done in this sense in the educational area. When emphasizing the cycle of a public policy, Frey (2000) presents it in four phases, which are: (i) the perception and definition of problems, (ii) the elaboration of programs and decision, (iii) the implementation and, finally, (iv) the evaluation for correction and improvement of the action.

Indeed, considering that this is the last stage that interests us, within the scope of this proposal, we must understand what it means to evaluate a public policy. Initially, it should be noted, as appropriate, that, according to Andriola (1999), the evaluation is a scientific that has the pretension to obtain useful information about the analyzed object with dual purpose: allow its valuation and enable its improvement.

In this sense, Faria (2005) corroborates this view, when emphasizing that evaluation represents the stage of the policy cycle which aims to improve actions besides providing subsidies for interventions, accountability and responsibilization of involved agents. Souza (2009) uses this theoretical positioning when highlighting the evaluation as a process of monitoring and analyzing the implementation of policies, which assists all stages of planning and guides the decision making.

It is still at the evaluation stage that is verified which objectives were achieved, in order to promote decisions on the continuity of the political cycle, on the initiation of a new cycle, the elaboration of a new program or until the modification of the previous (FREY, 2000). In this context, Souza (2003) points out that the researches should pay special attention to the variables which impact on the results of public policies, with a view to better understand the results of these.

Said that, the evaluation, which can be made during or after the execution of a program, is part of the public policy cycle and is exonerable to analyze and verify its repercussions, besides contributing to the continuous development of government actions. It should be noted that since beginning of the evaluation of public policies, in the decade of 1960, its function changed its focus.

According to Trevisan and Bellen (2008), initially the evaluation emphasized the provision of information, assuring a feedback; lately, in the decade of 1980, when seeking more efficiency on allocation of resources, the focus was directed to the (re)allocation fundamented on budgetary criteria; already in the decade of 1990, the last verified in literature, its function became of legitimation by questioning the role of the State, attending the requirement to verify the results of public administration investments.

It should be noted that in the last years there is a global movement within the legislative branches in various continents such as Asia, Africa and Latin America about the need to create national evaluation policies, with the purpose of analyzing the relevance, effectiveness, efficiency and fulfillment of public programs. In reality, it is a movement whose origin dates goes back to the ideas of Scriven (1969) about the concepts of meta-evaluation and accountability.

When we focus on an accountability based on a broader perspective, based on not only their forms and processes, but in a vision in which it emerges as an instrument in the service of the democratic ideals of a country, the positions of those who see it as a mere instrument of results control, or those who suppose it as a mere control of processes guaranteeing the provision of efficient public services, can be easily criticized on the grounds of the unknown and the disregard for the complexity of the functions performed by public administrators in contemporary societies, in which more than to govern, public administrators have a duty to seek what Plato, Aristotle and Cicero treated as “the common good” (ROCHA, 2011). Therefore, from our theoretical perspective, the accountability allows to impose transparency and control of public affairs, through evaluation systems that aim to generate relevant information about the adequacy of the public resources used in the execution of the programs and activities that are components of public policies5.

Thus, the present study is justified on the basis of the idea that contemporary society requires accountability of financial institutions (governments, multilateral agencies, research organization, among others) of public policies and their respective social programs or projects, in order to make the costs clear and the impacts involved in the results obtained from these, using, for such, systematic evaluation.

METHODOLOGICAL PROCEDURES

In order to achieve the general objective a Field research of the type ex-post facto was done, of descriptive nature (KERLINGER; LEE, 2000) with the universe of 2.193 students of the graduation courses of UFC during the period of 2009.2 to 2015.1, comparing to the group of alumni of PIBID (n = 209 or 12,3%) with the non-fellows students of the program (n = 1.924 or 87,7%). These students were segmented according to (a) the ENADE concept of the respective course and (b) the shift of the course. Subsequently, there was information obtained from the IRA of these students in order to make comparisons, according to the results explained in the following section.

PRESENTATION AND DISCUSS OF THE RESULTS

The Table 1 contains information about the ENADE concept of the courses, os the size of the samples, of average values and standard deviations of students’ IRA, of the values of T Test of Student 6 , as well as the level of significance (p-value).

Table 1 Results of T Test of Student for the comparison of students’ average IRA (fellows and non-fellows), according to the ENADE concept of the course to which they belong. 

ENADE concept Samples (n) IRA’s average Standard deviation T Test p
Non-fellow Fellow Non-fellow Fellow Non-fellow Fellow
2 65 12 7,243 7,891 1,670 0,8570 -2,009 0,05
3 270 24 7,185 8,070 1,600 0,9141 -4,202 0,00
4 1268 164 6,669 7,100 1,169 0,7179 -6,636 0,00
5 310 56 6,622 7,093 1,099 0,5474 -4,905 0,00

Source: The authors (2016).

According to the data on Table 1, it is observed that the quantitative of the samples of PIBID’s non-fellows are always superior to the sample of PIBID’s alumni. It should be noted that in two specific situations the samples are less than 30 subjects: PIBID’s non-fellows belonging to the courses with concepts 2 and 3 in ENADE. Nevertheless, the T Test of Student is robust enough to be used in those situations where the sample size is less than 30.

Still considering Table 1, it should be pointed out that the observed values for the standard deviations are always lower in the subsamples of PIBID’s fellows, indicating greater homogeneity in the academic performance of these graduates, in contrast to the subsamples of PIBID’s non-fellows. In addition, it is convenient to highlight the fact that the more qualified the course, for example, in the case of courses with concepts 3, 4 and 5, the lower is the standard deviation of IRA’s graduates, irrespective of whether they have been fellows or not of PIBID. In both cases, such aspect indicates that the greater homogeneity in the academic performance of the graduates, more qualified is the course, expressed by ENADE concept attributed by MEC.

In Figure 1, shown below, it can be clearly seen that the average values of IRA’s fellows of PIBID are higher than the average values of non-fellows, regardless of the ENADE concept of the course to which they belong. Therefore, it seems that the learning of PIBID’s fellows is qualitatively superior than the learning of the non-participants, regardless of the quality of the course measured by MEC through ENADE.

Figure 1 Line graphic with average values of IRA, according to the ENADE concept of the course. 

To finish, it should be noted that the observed values of T Test of Student point to the existence of statistically significant differences between the averages of IRA’s fellows of PIBID in comparison to the averages of IRA’s non-fellows of PIBID. In other words, the average values of IRA’s fellows of PIBID are higher than the average values of non-fellows, regardless of the ENADE concept of the course to which they belonged, and these differences are statistically significant by adopting the level of 5% of confidence (according to the observed values of p-value).

As referred before, the IRA is an indicator of importance that expresses the quality of student learning. Therefore, the differences observed in favor of the fellows of PIBID allow us to assert that their learning reveals the development of cognitive capacities, as well as the assimilation of curricular content in a superior way to the development of these same aspects among the graduates who were non-fellows of PIBID. It can be inferred that the difference in the quality of the training undergraduate is significant between the group of alumni of the PIBID and the group of non-fellows graduates, independently of the concept of the course in ENADE.

In Table 2, as presented below, informations are available to ascertain the existence of statistically significant differences between the average of IRA’s fellows of PIBID and PIBID’s non-fellows, in the graduation courses in the different shifts.

Table 2 Results of T Test of Student to do a comparison between the average of IRA’s students (fellows and non-fellows), according to the course’s shift to which they belonged. 

Course’s Shift Samples (n) IRA’s average Standard deviation T Test p
Non-fellow Fellow Non-fellow Fellow Non-fellow Fellow
Daytime 499 93 6,806 7,168 1,1151 0,6674 -4,243 0,00
Mixed 1007 117 6,569 7,140 1,1647 0,6683 -7,939 0,00
Nightime 418 58 7,143 7,548 1,5288 1,0157 -0,709 0,00

Source: The authors (2016).

According to the present data in Table 2, it is observed, initially, that the quantitative of the samples of PIBID’s non-fellows are always higher than the samples of PIBID’s fellows. All samples are greater than 30 individuals.

Continuing the analyses of Table 2, it should be pointed out that the observed values of the standard deviations are always lower in the subsamples of PIBID’s fellows, indicating greater homogeneity in the academic performance of these graduates, in contrast to the subsamples of PIBID’s non-fellows. In addition, it is convenient to emphasize the fact that in the daytime shift courses, the value of standard deviation of IRA’s students was always lower than the graduates of the nighttime shift courses, regardless of whether being a fellow or not of PIBID. In both cases, this aspect indicates greater homogeneity in the academic performance of the graduates during daytime shift courses.

According to Figure 2, shown below, the average values of IRA’s fellows of PIBID are higher than the average values of PIBID’s non-fellows, regardless of the course shift to which they belong.

Figure 2 Average values of IRA, according to the course shift 

It appears from the analysis of Figure 2, that the learning of PIBID’s fellows is qualitatively higher than the non-fellows, regardless of the graduation’s course shift. Therefore, as well as in the previously carried out analysis, the differences observed in favor of PIBID’s fellows allows us to assert that their learnings reveals the development of cognitive abilities, as well as the assimilation of curricular contents far superior than the PIBID’s non-fellows. It can be inferred that the difference in the quality of student’s formation between PIBID’s fellows and PIBID’s non-fellows, in favor of the first, regardless of the course’s shift to which they belong.

Lastly, it should be noted that the observed values of the T Test of Student points to the statistically significant differences between the averages of IRA’s ex-fellows of PIBID, in comparison to the averages of IRA’s non-fellows of PIBID. In other words, the average values of IRA’s ex-fellows of PIBID are higher than the average values of PIBID’s non-fellows, regardless to the course’s shift to which they belong, and these differences are statistically significant by adopting the 1% confidence level (according to the observed values of p-value).

CONCLUSIONS

Starting from the premise that the university formation takes place through teaching activities, through scientific research and effective application of scientific knowledge (extension), the present study sought to identify if there would be significant differences in the academic performance of graduates who were PIBID’s fellows in comparison to other graduates who didn’t have the same experience, carrying out and evaluation of the ex post-fact type. Thereby, a study was outlined considering the insertion of these students in graduate courses (i) with different concepts in the National Examination of Student Performance (ENADE) and (ii) with different shifts.

Based on the obtained data regarding the academic performance of these graduates, it can be verified that the highest averages of academic performance, measured through the Academic Performance Index (IRA) adopted by Federal University of Ceará (UFC), have been associated with PIBID’s graduates. Therefore, the results of the study evaluation of type ex post-fact demonstrated the relevance of PIBID for the training of university students, since there were substantial differences in the quality of the fellows’ learning, when compared to other apprentices who did not have the same fortune to submit themselves to the experiences offered by the said institutional program.

Therefore, it is understood that the learning of PIBID’s fellows is qualitatively higher than the non-fellows, regardless of the ENADE’s concept and the shift of the course. Thus, the differences observed in favor of PIBID’s ex-fellows allows us to assert that their learning reveals the development of cognitive abilities, as well as assimilation of the curricular contents much higher than the other PIBID’s non-fellows graduates.

These findings corroborate with the findings of other researches in the literature which are Langer, Ribeiro e Schroeder (2013), Araújo, Andriola, Rodrigues e Coelho (2016), Darroz e Wannmacher (2015), whose results points to the benefits provided by PIBID on the training of graduates participating in the referred program.

It is important to consider that PIBID has as main objective to attract the best professionals to work in public schools, thus contributing to the improvement of public basic education. Therefore, in addition to improving the quality of the future graduates’ training, it is indispensable to find out if PIBID’s graduates, after completing their graduation, if they are practicing the teaching profession, and, in positive case, where are they acting: in public or private school?

Indeed, it should be also pointed that, actually, society requires that the public policies and the programs that compose them are accountable to society, through systematic evaluation. Somehow, this was the aspect that justified the present research. From this notion one can assert that PIBID proved to be a well-planned program, elaborated and articulated, which adds value to the training of the participating students, empowering and enriching the experiences of future professionals, narrowing the relationship between school and university. However, there is a lack of research involving the monitoring of PIBID’s graduates, besides the lack of comparative data among the graduates that identify the percentages of their entrance in the schools of public basic education.

Furthermore, it is important to note that one of the main criticisms of the federal government regarding PIBID is exactly the low rate of return of what is being invested in the program, due to the low number of PIBID’S professionals who graduated who become teachers in public schools.

In spite of the above considerations, it is worth highlighting, one more time, that the excellent results obtained in this research, in line with other studies, corroborate the idea that PIBID promotes the development of substantive competences in the teaching profession. In other words: PIBID has become a policy that provides opportunities for the enrichment of learning, whose the most visible reflection consubstantiate in the high IRA of fellows.

It is clear that PIBID is a public policy of teacher education, without any pretension of being compensatory, as is the case of the Quota System for Higher Education (Law n° 12.711/2012) and the University For All Program (PROUNI - Law n° 11.096/2005). Therefore, PIBID does not envisage to opportunize working students, women with children and low-income students who, in IES which are devoid of appropriate student assistance actions, may be engaged in other activities during graduation, making it impossible to enter to the referred program. These latter learners should, therefore, be covered by another public policy, of compensatory nature.

One aspect to be mentioned is that, in certain way, imposes limitations on the study, but does not subtract its merit, not its relevance, it refers to the adoption of segmentation variables (IRA and ENADE concept). These variables, as indicators, have limited scope. In the educational field, indicator is an artifice providing relevant and synthetic information about significant aspects of the observed reality, which, usually, results in some kind of quantitative information (ANDRIOLA, ARAÚJO, 2016; JANUZZI, 2003). However, we should not attribute its success or acceptance only to its explanatory capacity, since specialist and social scientists use many other resources to know and interpret the reality that they address. Therefore, indicators should be seen as what they are in reality: limited scope artifacts, that is, of relative efficacy (VAN RAAN, 2005).

Why, then, are the relevance and the sharp diffusion of indicators, currently verified in the educational field? No doubt, the main reason lies in two characteristics intrinsic to the indicators: synthetic character and ability to guide the decision-making. Both indicators allow decisions to be made on the basis of information they generate. IRA allows that UFC students can progress their studies and complete their courses, based on the minimum criterion established to denote quality of learning, which is grade seven (in a scale that can range from zero to ten). In concern to ENADE, this enables the guardian of the quality of the courses and, therefore, of upper-level student formation, the Ministry of Education (MEC), regulate the Higher Education System and its courses, making decisions about maintenance, opening and extinguishing courses, based on the minimum criteria established to characterize the quality of concept three (in a scale that can range from one to five).

To conclude, it should be highlighted that the Higher Education it’s supported by three pilasters whatever the teaching is, the scientific research and extension. However, teaching is one of humanity’s most noble and ancient activities. In this regard, it should be pointed that, in certain occasion, was reported by the greatest of all scientists of XX century, the German physicist Albert Einstein: I don’t teach my students. I create conditions for them to learn. This seems to be the essential principle of PIBID. Therefore: long life to PIBID.

REFERENCES

ANDRÉ, M. Políticas e programas de apoio aos professores iniciantes no Brasil. Cadernos de Pesquisa, São Paulo, v. 42, n. 145, p.112-129, 2012. [ Links ]

ANDRIOLA, W. B. Evaluación: La vía para la calidad educativa. Ensaio. Avaliação e Políticas Públicas em Educação, Rio de Janeiro, v. 7, n.25, p. 355-368, 1999. [ Links ]

ANDRIOLA, W. B. Avaliação da aprendizagem: conceitos, dados, problemas e perspectivas. Psicologia Argumento, Curitiba, v. 22, n. 39, p. 45-55, 2004a. [ Links ]

ANDRIOLA, W. B. Avaliação institucional na Universidade Federal do Ceará (UFC): organização de sistema de dados e indicadores da qualidade institucional. Avaliação: Revista da Avaliação da Educação Superior, Campinas, v. 9, n.4, p. 33-54, 2004b. [ Links ]

ANDRIOLA, W. B. Propostas Estatais Voltadas à Avaliação do Ensino Superior Brasileiro: Breve Retrospectiva Histórica do Período 1983-2008. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en Educación , v. 6, n. 4, p. 127-148, 2008. Texto disponível em <http://www.rinace.net/reice/numeros/arts/vol6num4/art7.pdf>. Acesso em: 08 ago. 2016. [ Links ]

ANDRIOLA, W. B. Avaliação do aprendizado discente: estudo com professores de Escolas Públicas. Educar em Revista, Curitiba, n. 46, p. 141-158, 2012. [ Links ]

ANDRIOLA, W. B. Estudo de egressos de cursos de graduação: subsídios para a autoavaliação e o planejamento institucionais. Educar em Revista, Curitiba, p. 203-220, 2014. [ Links ]

ANDRIOLA, W. B.; ARAÚJO, A. de C. Relevância do uso de indicadores de gestão para a autoavaliação e o planejamento estratégico de Instituições de Ensino Superior (IES). Revista Meta-avaliação, Rio de Janeiro, v. 8, n. 24, p. 515-533, 2016. [ Links ]

ANDRIOLA, W. B.; OLIVEIRA, K. R. B. Autoavaliação institucional na Universidade Federal do Ceará (UFC): meio século de história. Avaliação: Revista da Avaliação da Educação Superior, Campinas, v. 20, p. 489-512, 2015. [ Links ]

ARANTES, M. A. A prática docente e o ensino de sociologia, p. 27-46. In: LANGER, Arleni Elise Sella; RIBEIRO, Dulcyene Maria; SCHROEDER, Tânia Maria Rechia (org.). PIBID: incentivo à formação de professores. Porto Alegre: Evangraf/UNIOESTE, 2013. [ Links ]

ARAÚJO, A. C. Avaliação do Programa Institucional de Bolsa de Iniciação à Docência da Universidade Federal do Ceará na perspectiva dos egressos (98 pág.). Dissertação de Mestradoapresentada ao Mestrado Profissional em Políticas Públicas e Gestão da Educação Superior (POLEDUC). Fortaleza: Universidade Federal do Ceará (UFC), 2015. [ Links ]

ARAÚJO, A. C.; ANDRIOLA, W. B.; RODRIGUES, M. do S. de S.; COELHO, A. de A. Avaliação do Programa Institucional de Bolsa de Iniciação à Docência (PIBID): estudo de caso na Universidade Federal do Ceará (UFC). Revista Foro Educacional, Chile, n. 27, p. 33-56, 2016. [ Links ]

Blankenberg, F. The Role of planning, monitoring and evaluation. The Hague: Novib, 1995. [ Links ]

Bourguignon, F.; FERREIRA, F.; LEITE, P. Conditional Cash Transfers, Schooling, and Child Labor : Micro-Simulating Brazil’s Bolsa Escola Program. The World Bank Economic Review, v. 17, n. 2, p. 229-254, 2003. [ Links ]

BRASIL. Decreto nº 7.219, de 24 de junho de 2010. Dispõe sobre o Programa Institucional de Bolsa de Iniciação à Docência - PIBID e dá outras providências. Brasília, DF, 2010. Disponível em: <Disponível em: http://goo.gl/U3SqzP >. Acesso em: 22 fev. 2015. [ Links ]

BRASIL. Lei nº 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Brasília, DF, 1996. Disponível em: <Disponível em: http://goo.gl/3YQoF >. Acesso em:25 jun. 2015. [ Links ]

BRASIL. Lei nº 11.684, de 11 de junho de 2008. Altera o art. 36 da Lei no 9.394, de 20 de dezembro de 1996, que estabelece as diretrizes e bases da educação nacional, para incluir a Filosofia e a Sociologia como disciplinas obrigatórias nos currículos do ensino médio. Brasília, DF, 2008. [ Links ]

BRASIL. Portaria nº 096, de 18 de julho de 2013. Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES. Brasília, DF, 2013. Disponível em: <Disponível em: http://goo.gl/6YO9vh >. Acesso em:27 jun. 2015. [ Links ]

BRASIL. Portaria nº 38, de 12 de dezembro de 2007. Ministério da Educação. Dispõe sobre o Programa de Bolsa Institucional de Iniciação à Docência - PIBID. Brasília, DF, 2007. Disponível em:<Disponível em:http://goo.gl/PxTFMf >. Acesso em:27 jun. 2015. [ Links ]

BRUNNER, J. J. El proceso de Bolonia en el horizonte latinoamericano: límites y posibilidades. Revista de Educación, número extraordinario, p. 119-145, 2008. [ Links ]

CAPUCHA, L.; ALMEIDA, J. F.; PEDROSO, P.; SILVA, J. V. Metodologias de avaliação: o estado da arte em Portugal. Sociologia: problemas e práticas, Lisboa, v. 22, p. 9-27, 1996. [ Links ]

CARVALHO, G. P.; WHITE, H. Performance indicators to monitor poverty reduction. Washington, DC: World Bank, 1995. [ Links ]

CIANI, A. B.; LANGER, A. E. S.; RIBEIRO, D. M.; ANTUNES, F. C. A.; BASSOI, T. S. A experiência do PIBID na licenciatura de matemática da Unioeste - campus de Cascavel, p. 61-75. In: LANGER, A. E. S.; RIBEIRO, D. M.; SCHROEDER, T. M. R. (org.). PIBID: incentivo à formação de professores. Porto Alegre: Evangraf/UNIOESTE, 2013. [ Links ]

DARROZ, L M.; WANNMACHER, C. M. D. Aprendizagem docente no âmbito do PIBID/física: a visão dos bolsistas de iniciação à docência. Revista Ensaio: Pesquisa em Educação em Ciências, v. 17, p. 727-748, 2015. Disponível em Disponível em http://www.scielo.br/scielo.php?pid=S1983-21172015000300727&script=sci_abstract&tlng=pt . Acesso em: 24 mai. 2017. [ Links ]

FARIA, C. A. P. de. A política da avaliação de políticas públicas. Revista Brasileira de Ciências Sociais, São Paulo, v. 20, n. 59, p. 97-109, 2005. [ Links ]

FETZNER, A. R.; SOUZA, M. E. V. Concepções de conhecimento escolar: potencialidades do Programa Institucional de Bolsa de Iniciação à Docência. Educação e Pesquisa, São Paulo, v. 38, n. 3, p. 683-694, 2012. [ Links ]

FREY, K. Políticas públicas: um debate conceitual e reflexões referentes à prática da análise de políticas públicas no Brasil. Planejamento e Políticas Públicas, Brasília, n. 21, p. 211-259, 2000. [ Links ]

FRIGOTTO, G. Os circuitos da história e o balanço da educação no Brasil na primeira década do século XXI. Revista Brasileira de Educação, Rio de Janeiro, v. 16, n. 46, p. 235-274, 2011. [ Links ]

GENTILI, P.; FRIGOTTO, G.; LEHER, R; STUBRIN, F. (org.). Políticas de privatización, espacio público y educación en América Latina. Rosario: Homo Sapiens/CLACSO, 2009. [ Links ]

JANUZZI, P. de M. Indicadores sociais no Brasil. Conceitos, fontes de dados e implicações. Campinas: Alínea, 2003. [ Links ]

KERLINGER, F. N.; LEE, H. Foundations of behavioral research. Orlando: Harcourt College Publishers, 2000. [ Links ]

LANGER, A. E. S.; RIBEIRO, D. M.; SCHROEDER, T. M. R.(org.). PIBID: incentivo à formação de professores. Porto Alegre: Evangraf/UNIOESTE , 2013. [ Links ]

LEHER. R. Educação no Governo Lula da Silva: a ruptura que não aconteceu (p. 369-412). In: Os anos Lula: contribuições para um balanço crítico 1993-2010. Rio de Janeiro: Garamond, 2010. [ Links ]

LIBÂNEO, J. C.. Adeus Professor, adeus professora? Novas exigências educacionais e profissão docente - Coleção Questões da Nossa Época; v. 67. 7ª ed. - São Paulo: Cortez, 2003. [ Links ]

LIMA, K. R. de S. O Banco Mundial e a educação superior brasileira na primeira década do novo século. Revista Katál, Florianópolis, v. 14, n.1, jan./jun. p. 86-94, 2011. [ Links ]

MAURIEL, A. P O. Fundamentos do combate à pobreza na contemporaneidade: Amartya Sen e a perspectiva do desenvolvimento humano (p. 87-128). In: SERAINE, A. B. dos S.; S JUNIOR, R. B. dos; MIYAMOTO, S. Estado, desenvolvimento e políticas públicas. Ijuí: Unijuí; Teresina: Ed. Universitária da UFPI, 2008. [ Links ]

NÓVOA, A. Para una formación de profesores construida dentro de la profesión. Revista Educación, n. 350, septiembre-diciembre, 2009. [ Links ]

PÉREZ GÓMEZ, A. I. La evaluación como aprendizaje. Madrid: Ediciones AKAL, 2009. [ Links ]

PÉREZ JUSTE, R.; GARCÍA RAMOS, J. M. Diagnóstico, evaluación y tomada de decisiones. Madrid: Ediciones Rialp, 1989. [ Links ]

PERRENOUD, P. A prática reflexiva no ofício do professor: profissionalização e razão pedagógica. Porto Alegre: Artmed Editora, 2002. [ Links ]

PIMENTA, S. G. Formação dos Professores - Saberes da Docência e Identidade do Professor. Nuances- Vol. III- Setembro, 1997. [ Links ]

ROCHA, A. C. Accountability na Administração Pública: modelos teóricos e abordagens. Contabilidade, Gestão e Governança, Brasília, v. 14, n. 2, p. 82-97, 2011. [ Links ]

ROSALES, C. Criterios para una evaluación formativa. Madrid: Narcea, 1984. [ Links ]

SCRIVEN, M. Evaluation as a discipline. Studies in Educational Evaluation, v. 20, p. 147-166, 1994. [ Links ]

SCRIVEN, M. An introduction to meta-evaluation. Educational Products Report, 2, p. 36-38, 1969. [ Links ]

SHADISH, Jr., W.; COOK, T.; LEVITON, L. Foundations of program evaluation: theories of practice. Londres: SAGE, 1991. [ Links ]

SILVA, T. M. Teste t de Student. Teste de igualdade de variâncias. Belém: Universidade Federal do Pará (UFPA), 2014. Disponível em <Disponível em http://www.ufpa.br/heliton/arquivos/aplicada/seminarios/M1_01_Teste_t_Tais.pdf >. Acesso em: 09 jul. 2016. [ Links ]

SILVEIRA, J. A.P. da. Ensaio: confissões de um observador sem método das impressões em segunda mão das escolas de ensino médio do Paraná, p. 97-107. In: LANGER, A. E. S.; RIBEIRO, D. M.; SCHROEDER, T. M. R. (org.). PIBID: incentivo à formação de professores. Porto Alegre: Evangraf/UNIOESTE, 2013. [ Links ]

SOUSA, A. C. G.; ANDRIOLA, W. B.; LIMA, A. S. Expectativas da Avaliação Docente na Educação Superior Brasileira. Um Estudo com os Envolvidos em uma Instituição de Ensino Pública. Revista Iberoamericana de Evaluación Educativa, Santiago do Chile, v. 9, n. 2, p. 81-105, 2016. [ Links ]

SOUZA, C. “Estado do campo” da pesquisa em políticas públicas no Brasil. Revista Brasileira de Ciências Sociais, São Paulo, v. 18, n. 51, p. 15-20, 2003. [ Links ]

THERRIEN, J., H SOBRINHO, J.. Avaliação institucional em Universidades: considerações metodológicas. Educação em Debate, Fortaleza, v. 6/7, n. 2/1, p. 17-27, 1983/1984. [ Links ]

TREVISAN, A. P.; BELLEN, H. M. van. Avaliação de políticas públicas: uma revisão teórica de um campo em construção. Revista de Administração Pública, Rio de Janeiro, v. 42, n. 3, p. 529-550, 2008. [ Links ]

VAN RAAN, A. Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, Budapest, v. 62, n. 1, p. 133-143, 2005. [ Links ]

1 Founded research by the National Council for Scientific and Technological Development (CNPq), through the grant of a scholarship of Productivity in Research (Process number 300.878/2013-3 - PQ 1D) for the second author. Partial results of this study were presented in the XVI International Colloquium of University Management (CIGU), held in November of 2016, in Arequipa, Peru. The authors express their gratitude to the servers of Pro-Rectory of Graduation (PROGRAD) of the Federal University of Ceará (UFC) - Brazil, for the extreme goodwill and availability to provide the data used in the present research.

2Document available in <http://www.ufc.br/images/_files/a_universidade/regimento_geral_ufc/regimento_geral_ufc.pdf>. Access in 08/08/2016.

3Additional information about the Academic Performance Index (IRA) adopted by Federal University of Ceara (UFC) can be consulted in <http://www.prograd.ufc.br/perguntas-frequentes/384-perguntas-frequentes-ira>. Access in 01/07/2016.

4Additional information MEC’s website <http://portal.inep.gov.br/enade>. Access in 01/07/2016.

5In scientific field, especially among human and social sciences, there is no sustained theoretical positioning on the unanimous acceptance of the researchers. Therefore, with due respect to the notion of accountability used here, there are several authors with distinct and/or contrary positions, as the case of Mauriel (2008); Lima (2011); Brunner (2007); Frigotto (2011); Leher, (2010); Gentili, Frigotto, Leher, Stubrin (2009), among others. It is not the purpose of this article to discuss these theoretical differences, of world and state conception. Thus, we leave to the readers interested in the subject, the works mentioned above as a reading suggestion.

6The t statistic was introduced in 1908 by William Sealy Gosset (student was his pseudonym), chemist of the Guinness Brewery in Dublin (Ireland). W. S. Gosset had been hired because of the innovative policy implemented by Claude Guinness to recruit the Best graduates of Oxford and Cambridge for the positions of biochemist and statistician of Guinness Industry. W. S. Gosset developed the T Test as an inexpensive way to monitor the quality of beer type stout. He published the T Test in the academic journal Biometrika in 1908, but was forced to use his pseudonym by his employer, who believed that using statistics was an industrial secret. The T Test of Student is used to test hypotheses, based on statistical concepts to corroborate a null hypotheses when the test (t) statistic follows a t - Student distribution. This assumption is usually used when the test statistic, in fact, follows a normal distribution, but the population variance is unknown. In this case, the sample variance is used, and with this adjustment the test statistic follows the t - Student distribution (SILVA, 2014).

Received: November 30, 2016; Accepted: September 16, 2017

Contact: Mrs. Adriana Castro Araújo, Tenente Amauri Pio Street, 57, Ap. 1100 - Meireles, Fortaleza | Ceará | Brazil, ZIP CODE: 60.160-090

*

Master in Public Policies and Management of Higher Education - Federal University of Ceará (UFC) - Brazil; Doctoral student in Educational Evaluation by the UFC. UFC Technical-Administrative Servicer. Researcher of the Center for Educational Evaluation (NAVE/UFC). E-mail:<driaraujo12@gmail.com>.

**

Doctor in Philosophy and Education Sciences - Universidad Complutense de Madrid (UCM) - Spain. Associate Professor IV of the Federal University of Ceará (UFC) and Researcher 1D of the National Council for Scientific and Technological Development (CNPq). Researcher-Leader of the Educational Evaluation Center (NAVE/UFC). E-mail:<wagner.andriola@pq.cnpq.br>.

***

Doctor in Meteorology - Université de Toulouse III - Paul Sabatier - France. Associate Professor III of the Federal University of Ceará (UFC) and Coordinator of Management of Institutional Processes of PIBID /UFC. E-mail:<afranio@fisica.ufc.br>

Creative Commons License Este é um artigo publicado em acesso aberto sob uma licença Creative Commons