English (pdf)
Article in xml format
Automatic translation
Send this article by e-mail
Permalink
Introduction
Over the past few decades, test-based accountability reforms1 have been widespread in the global education agenda, aimed at holding schools, teachers and students accountable for their results, overcoming bureaucratic inefficiencies or increasing school outcomes (Verger & Parcerisa, 2017). In Brazil, large-scale assessments have been consolidated and rearticulated locally (Costa et al, 2019; Lima & Gandin, 2019) and have been adopted as accountability policy solutions, so as to increase educational quality and effectiveness, improve student performance and overcome state inefficiencies (Brooke, 2008; OECD, 2010). In the state of Minas Gerais, large-scale evaluations have been adopted as accountability tools to measure the quality of teaching, linking teachers’ salaries to school performance through monetary incentives. Studies highlight the changes in teachers’ work (Augusto, 2012; Borges & Sá, 2015) and schools’ priorities and practices as a consequence of these reforms (Almeida, 2020), as well as teachers’ and principals’ resistance at school level (Brooke, 2008). If accountability policies have shifted schools’ internal management and organization, and shaped new professional identities, the role and practices of principals at school level are worthy of consideration.
The existing literature on school leadership from an accountability perspective acknowledges that principals operate in a complex and dynamic environment, experiencing multiple, often conflicting pressures, which derive from different sources and purposes (Shipps & White, 2009; Wang, Hauseman & Pollock, 2022). In addition to the external pressures, principals can also experience internal accountability demands, such as those deriving from professional judgement (i.e., professional accountability) or their moral obligations and sense of duty (i.e., moral responsibility). In terms of the principals’ role in schools, they hold a primary position, directly focusing on students’ learning and indirectly influencing organizational features (Hallinger, 2003). More interestingly, principals play an intermediate role in high-stake contexts, acting as “managers in-between” policy and school levels (Spillane et, 2002). In other words, principals negotiate between external pressures, achievements or demands and the schools’ internal context and resources (Spillane et al., 2002). When school leaders need to align their daily practices to external expectations, they may adopt different leadership practices. However, the activation and effectiveness of such practices depend on the school’s available resources and the principal’s characteristics (Finnigan, 2010; Hallinger, 2003).
In the literature regarding accountability reforms in Brazil, there seems to be a lack of research into the way in which leaders intervene in the reception of accountability reforms in schools. The literature on principals’ leadership mainly focuses on the effectiveness of leadership styles, without analysing the conditions or beliefs according to which those practices are adopted. In this sense, it is relevant to frame the study of leadership in a ‘multiple’ accountability framework (Shipps & White, 2009), acknowledging how and to whom principals are held accountable, understanding the way in which accountability mechanisms operate and the impact that these accountabilities have on principals’ decision making and practices. Normore (2004) argues that educational accountability experienced by school leaders differs from the concept of accountability framed in policy discourse, which generally refers to school, teacher or student accountability. Therefore, we should consider empirical evidence on accountability experienced by school administrators (Normore, 2004).
The theoretical framework is based on a sense-making leadership framework (Ganon-Shilon & Schechter, 2016; Spillane et al., 2002) coupled with a contingent leadership framework (Shipps & White, 2009). Sense-making is “an active process of constructing meaning from present stimuli, mediated by prior knowledge, experiences, beliefs, and values embedded in the social context within which people work” (Ganon-Shilon & Schechter, 2016, p. 684). The main idea is that principals prescribe different meanings to the policy message and that these meanings shape their practices (Spillane et al., 2002). Indeed, sense-making implies three interrelated stages, namely creation, interpretation and enactment. The enactment process means that “people incorporate new information and eventually take action based on the interpretation they have created” (Ganon-Shilon & Schechter, 2016, p. 684). In other words, principals’ sense-making is deeply embedded in policy arrangements and context, which also influence their leadership practices (Spillane et al., 2002). Policies are, therefore, not considered linear processes but mediated by the actor's interpretation and translation of the reform, as well as by the material or professional school contexts (Braun et al., 2011). A contingent leadership framework, on the other hand, relies on the idea that “leaders perceive their responsibilities as varied according to the relationships they have with adults crucial to the organization’s performance, the structure of the task, and the positional power they have to reward or coerce desired behaviour” (Shipps & White, 2009, p. 353). In this sense, leaders’ thinking, behaviour and situation are considered together (Spillane et al, 2004), acknowledging a distributed perspective of leaderships’ practice and tasks and the interactions with other formal and informal leaders in schools.
The aim of the paper is to identify the way in which school leaders perceive and respond to accountability mandates and their role in shaping teachers’ reception of such reforms, while also considering the school context in which they are embedded. The goal is threefold: 1) to identify how accountability policies shape the role, tasks and behaviour of school leaders; 2) to analyse how school leaders understand and translate the accountability policy mandate in their schools and 3) to explore how this behaviour shapes teacher’s perceptions of the reform and the factors which are involved in such a relationship.
Data were collected during three months of fieldwork (October-December 2017) and are based on qualitative research, conducted in four primary and low-secondary schools in Belo Horizonte (a state in Minas Gerais, Brazil). Data are analysed by means of a realist evaluation (cf. Pawson & Tilley, 2004) analysis, which helps understand how mechanisms are activated under particular circumstances.
The paper is structured as follows. Firstly, a literature review of school leadership within accountability regimes is presented. The policy context of Minas Gerais’ accountability reforms follows. Subsequently, the data collection process, research instruments and school sampling strategy will be outlined. Findings will be presented, initially focusing on similar enactment trends found in all schools and then discussing the emerging differences and their mediating factors. Conclusive remarks, discussions and future lines of research are presented in the last section.
School leadership under accountability regimes
Pressure from accountability reforms influences the tasks, roles and challenges faced by principals. Within accountability environments, principals must respond to multiple responsibilities and demands at any one time (Wang et al., 2022), such as providing vision and school goals, ensuring good teaching and learning, monitoring curriculum and student results, and managing and allocating resources (Day et al., 2008). In accountability reform contexts, principals’ priorities have also changed. Principals spend more time on administrative and management tasks, such as budget, school finance or personnel administration than instructional activities (Meyer & Macmillan, 2001). This central involvement in administrative or managerial tasks is required since “while instructional leadership is important […], principals suggest that immediacy of other, time-dependent issues often take precedence over and over-shadow the more complex, yet less immediate, issues associated with instructional leadership” (Meyer & Macmillan, 2001, p. 6).
Accountability policies have also altered the perception of principals’ work. On the one hand, schools become business models and their leaders are “entrepreneurs”, who take a proactive role in advancing initiatives and changes that respond to the needs of their schools (Yemini, Addi-Raccah & Katarivas, 2015). On the other hand, the “new manager” becomes the “new hero of educational reform” (Ball, 2003, p. 2019), who instils attitudes in his/her employees, with a focus on accountability and commitment to the organization. In this sense, school leaders are caught between school and policy demands. They may experience tensions between the need to build trust, commitment and collegiality on the one hand, and the necessity of holding teachers accountable and controlling their instruction on the other (Shirrell, 2016), or there may be conflict regarding the need to align teaching, so as to standardize tests and to adapt these methods to teachers’ professional standards (Knapp & Feldman, 2012). Interestingly, principals are considered "managers in the middle" between school and policy levels, while dependent on both in order to successfully implement reforms (Spillane et al., 2002).
In response to schools’ changing needs in accountability contexts, school principals may adopt different practices, such as direct practices relating to the curriculum and instruction or indirect practices, which pass through organizational features (Hallinger, 2003). Direct mechanisms are associated with improvement strategies, based on data collection and analysis from standardized test results, including identifying problems, setting goals and developing a school curricular programme accordingly (Cosner & Jones, 2016). Direct practices also include using the language of student assessments in schools’ internal activities every day (Knapp & Feldman, 2012) or using data to identify issues that impact school performance and student learning (Cosner & Jones, 2016). These practices resonate with the form of ‘instructional leadership style’, which concerns the adoption of those practices revolving around curriculum and instruction for the purposes of improving teaching and learning in schools (Robinson, Lloyd & Rowe, 2008). This is related to the direct involvement of principals in defining school goals and instructional strategies. However, this leadership style is also associated with top-down approaches, directive behaviours and external rewards to stimulate the achievement of results (Hallinger, 2003).
Indirect leadership practices, conversely, act through organizational features and their primary function is to create a professional learning environment in schools and sustain schools’ capacity to improve. Especially under high-stake regimes, teachers’ morale can be low, and their motivation and capacity play a role in influencing the implementation of accountability policies (Leithwood & Jantzi, 2006). Hence, supporting teachers’ self-efficacy, understanding and professional development is crucial. With this in mind, certain authors point to the positive value of building professional learning communities in schools, communication structures, trust and shared learning (Daly, 2009; Finnigan, 2010; Tschannen-Moran, 2009). A ‘transformational leadership style’ is concerned with creating a shared vision, attention to teachers’ needs and a culture for change (Leithwood & Jantzi, 2006). These practices are effective under accountability regimes because they sustain teachers’ motivation to change (Yu, Leithwood & Jantzi, 2002) by influencing teachers’ motivation, capacity and work settings (Leithwood & Jantzi, 2006). Principals operating in accountability systems are also found to have an impact on improvement in schools, indirectly influencing teachers’ leadership, collaborative culture, learning climate and self-efficacy (Paletta, Basyte Ferrari & Alimehmeti, 2020).
Nonetheless, certain conditions influence the adoption of such leadership practices. For instance, Finnigan (2010) found that positive approaches, such as inclusive leadership, teacher-principal trust and principal support for change were less evident in low-performing schools. As Spillane et al. (2001) argue, transformational and distributed leadership require additional human, social and material resources. On the other hand, directive leadership strategies tend to be adopted in poorer schools, with a clearer academic mission (Hallinger, 2003).
Principals also prescribe different meanings and interpretations to the messages regarding policy reforms (Spillane et al., 2002) and may have different views of accountability, associating it with test scores, academic growth, attendance, transparency, funding, market-based accountability and parental or student development (Ford, 2016). The sense-making that principals Spillane possess regarding accountability reform also shapes their adopted practices (Gawlik, 2015; Diamond et al., 2002). For instance, Gawlik (2015) found that if principals perceive the importance of accountability as indicating the reality of students’ performance, they adopt strategies to respond to student diversity in school. Otherwise, if accountability is embodied in professional development, principals would sustain teachers’ learning opportunities. Ultimately, this means that principals are actively constructing a new accountability environment: “by placing new information into cognitive frameworks, individuals not only develop a sense of what is going on but also develop a sense of how to engage in the situation” (Ganon-Shilon & Schechter, 2016, p. 685).
Brazilian accountability reforms and context
Over the last 15 years, Brazil has become a pioneer in the field of large-scale assessments (Brooke, 2008), implementing an “effective policy of educational evaluation” (Castro, 2009, p. 5), composed of various programmes and sophisticated, designed testing items and instruments. Such policy architecture, influenced by international discourses on large-scale evaluations (Lima & Gandin, 2019), aims to manage the educational system by monitoring and formulating public policies and using education evaluations for accountability2 purposes (Brooke, 2008; Castro, 2009). The system includes a national test (a census-based test, Prova Brasil), which evaluates the abilities of all students nationwide in grades 5 and 9 (ISCED 1 and 2, respectively) in the Portuguese language and mathematics. The test results are publicly available, capturing school specificities and comparing school data longitudinally (Bonamino & Sousa, 2012). In 2007, the test results were further aggregated in a performance indicator (IDEB), synthesizing academic performance and educational flux. The indicator set educational targets and responded to diagnostic purposes, by enabling an intergovernmental policy dialogue between the federal, state and municipal systems (Bonamino & Sousa, 2012; Costa et al., 2019). The results of the index generate rankings which are published in the media (Andrade, 2008; OECD, 2010), an aspect that promotes the mobilization of school staff and the school community (Bonamino & Sousa, 2012) and represents a significant step towards educational accountability (OECD, 2010).
There has also been significant adoption and rearticulation of large-scale evaluations at subnational levels (Costa et al., 2019), gradually spreading to 18 states from 1992 to 2001 (Brazil Ministry of Education, 2015). Financial and economic consequences are attached to their results, leading to a form of high-stake accountability (Bonamino & Sousa, 2012; Costa et al., 2019). In the state of Minas Gerais, the evaluation system (SIMAVE), adopted in 2000, is composed of three tests (Proalfa, Proeb and PAAE). The design of the SIMAVE test is aligned with the Prova Brasil test in terms of the grades and subjects evaluated. The adoption of the SIMAVE evaluation system was part of a managerial restructuring reform (named “choque de gestão”), implemented by the centre-right government to overcome bureaucratic inefficiencies and increase the effectiveness of the system and its levels of productivity (Augusto, 2012). The standardized tests aim to develop management processes, based on a continuous evaluation of public education policies (Minas Gerais, 2000). In the first phase of the reform, low stake consequences were attached to the test results. These included the divulgation of school pedagogical and evaluative bulletins. The second part of the reform was characterized by an outcome-based and results-driven logic (Neto, 2013), in which external test results were used as targets within the accountability agreement established between schools and administrations (Acordo de Resultados, Minas Gerais 2008 law 17.600/2008). During this phase, schools which achieved the established targets were awarded an institutional financial prize (Premio Gestão Escolar) and a financial bonus for teachers (Bônus de Desempenho).
The reforms set new demands and expectations for principals. Principals were incentivized to use test results to elaborate a political and pedagogical plan, monitor the quality of education and implement accountability instruments for continuous professional improvement (Secretaria de Estado de Educação de Minas Gerais, 2013). Principals were also responsible for signing the ‘results agreement’ and achieving the established objectives through a democratic management model, which represents the interests of the entire school community (Mariano et al., 2016). School principals in Brazil are, therefore, generally referred to as managers (gestores) or administrators (administrator) since their roles focus on the management of the school organization (Honorato, 2018). In parallel, principals and pedagogical coordinators are responsible for making teachers aware of last year’s student performances and encouraging further discussion and analysis of the data, so as to formulate strategies to improve student performance (Brooke & Amália, 2007). Schools should readjust their ‘Political Pedagogical Plan’ (PPP) based on the diagnostics received from the external test results. In this sense, SIMAVE results orient school practices through schools’ internal management (Silveira & Almeida, 2019). As a result of such external evaluations and bonus payment schemes, certain authors outline the degradation of teachers’ working conditions (Augusto, 2012) and the alteration of the content and pedagogy of teaching (Almeida, 2020; Borges & Sá, 2015; Passone, 2014). Overall, the reforms have faced opposition and resistance from teachers, principals and labour unions (Andrade, 2008; Brooke, 2008) and have had significant repercussions on school professionals’ practices and identities.
However, school principals in Brazil do not work alone. Schools’ strategic decisions should be taken collectively through the school board (LDB/'96) of which the principal is the president, responsible for the functioning and coordination of its activities. The school principal is also part of the leadership team. This is composed of the vice-principal, who assumes the tasks and role of the principal in his/her absence, the pedagogical coordinators or the educational specialists (especialista em educação básica/coordenador pedagogico), who are responsible for coordinating the school curricular and pedagogical activities and the educational advisor (orientador educacional), who focuses primarily on students’ behaviour.
Data and methods
The study is based on a realist evaluation methodological approach, which highlights the context and institutional design of the policy programmes and the meaning-making processes of the actors who enact these programmes (Pawson & Tilley, 2004). According to this approach, policy programmes are active as they are produced by and require the active engagement of individuals. They are also open systems because several externalities can transform them. These can change the conditions that have rendered the delivery of the programme possible in the first instance. Indeed, the effects of a particular policy programme depend on the activation of specific underlining mechanisms, defined as the “processes by which the actors interpret and act upon a determinate program” (Pawson & Tilley, 2004, p. 6). Pre-existing structures can both ‘enable’ or ‘disable’ the intended mechanism of change, which will render the outcomes mixed and varied (Pawson & Tilley, 2004). At the heart of the approach is the idea that policy programmes do not work for everyone but that one should address the issues of ‘for whom’ and ‘in what circumstances’ (Pawson & Tilley, 2004, p. 7).
Data were collected during three months of fieldwork in four primary and low-secondary schools in Belo Horizonte (Minas Gerais). The focus on the state of Minas Gerais is justified by the presence of research partners (GESTRADO research group from the Universidade Federal de Minas Gerais/UFMG) and the nature of the test-based accountability reforms in this state. In addition, public schools (both municipal and state schools), which offer both ISCED 1 (primary education) and ISCED 2 levels (lower secondary) were considered. External examinations are conducted at these two education levels (grade 5 and 9, respectively) and are compulsory for all public schools. Moreover, Brazil is a large, federal and decentralized country3, thus, it is essential to consider school differences regarding administrative dependencies.
The selection of schools is based on data from the 2015 school census (INEP database) and two main discriminatory variables: student socio-economic status (Indicador de Nível Socioeconômico- INSE) and Prova Brasil test results. By dividing the variables into two categories (above and below average), the final sample generated four ideal school types: poor schools performing well, poor schools performing badly, well-off schools performing well and well-off schools performing badly. Four schools were selected, one per category and the final number of schools included two state and two municipal schools. The characteristics of the participating schools are presented below:
Table 1 Characteristics of participating schools (2017)
| School | School Size | Position in rankings | Student SES level | Administrative dependency |
|---|---|---|---|---|
| School A | 21 classes (approx. 550 students) | Below average | Above average | Municipal school |
| School B | 32 classes (approx. 960 students) | Above average | Above average | State school |
| School C | 18 classes (approx. 360 students) | Below average | Below average | Municipal school |
| School D | 20 classes (approx. 450 students) | Above average | Below average | State school |
Source: own elaboration based on the 2015 school census (INEP database).
Semi-structured interviews and observations are used as research instruments. This choice allows to capture the individual meanings of participants’ discourses and practices and relate them to schools’ internal dynamics and characteristics. Thirty-two semi-structured interviews were conducted in total with all key personnel, including principals, the leadership team (vice-principals and educational coordinators/specialists) and teachers in the four schools. Based on common themes, two interview scripts were used, one for the principals/leadership team and one for the teachers. The topics included the perception of their professional role, use of test data in schools, school climate and leadership style, and perceptions regarding accountability reforms and tests.
Finally, a qualitative realist analysis logic (Pawson & Tilley, 1997, 2004) was used to analyse the data. According to this approach, ‘regularities,’ ‘rates,’ ‘associations,’ ‘outcomes’ and ‘patterns’ are generated by underlining mechanisms, based on the interplay between structure and agency and only used in certain local, historical or institutional contexts (Pawson & Tilley, 1997, p. 71).
Analysis and findings
Similar trends in the four schools
Top-down pressure, change in role and sense of over responsibility
A general tendency has been identified in the four schools, namely the perception of a top-down accountability pressure, transmitted from the external administration to the school principal and then to teachers, to ensure alignment with the external accountability demands.
The principal is subjected to this direct pressure from the secretary of education and transmits it to us, you know, whatever the objective of the secretary, whatever the secretary wants or does not want (School D, Teacher, 2).
The principals in the four schools referred to a perceived pressure, deriving from the administration and a sense of self-responsibility to achieve good school results. Despite recognizing that school results in external tests are also dependent on other factors, principals often perceived that they were the only ones responsible for these results, as is evident from the following quote:
If the results are bad, no-one is responsible but me! They [the administration] do not want to know if it was a supervisor’s fault, a professor’s or a student’s. If the results are poor, I am accountable [...]. And it seems like I am the one who is not teaching students. They think I am not taking responsibility for the results. We can all take responsibility, you know (School D, Principal 1).
School principals talked about school performance and results as “theirs” and referred to their capacities as educational leaders in terms of achieving these outcomes. This aspect emerged in expressions such as: “I have increased the intake of students to a certain level,” “I managed to increase the number of students who enrolled” or “we have reached a target of 5.5.” In Andrade's (2008) words: “the logic of the policy is to tie resources to school results […] this way the educator becomes accountable to society for student learning” (p.451).
Similar to the literature, principals perceived that accountability for the entire school institution had magnified their sense of duty and individual responsibility. Indeed, leaders felt they needed to comply with several different demands and domains, leading to work overload, which influenced their perception of increased responsibility. In line with Duarte, Augusto and Jorge (2016), anxiety related to delivering results and developing actions and projects was apparent.
It is the role with the most significant responsibilities in school because everything that happens, positive or negative, is the principal's responsibility (School B, Principal 1).
Being a teacher is better because you go into your class, do your job with your students and leave. Within the school leadership, many things need to be resolved. All problems regarding professors, students and parents are much more complicated, there is more work, more worry and more responsibilities (School B, Vice-principal 1).
The bureaucratic, high-stakes and performance-based accountability policies have also altered principals’ duties. In line with the literature (e.g., Silva & Alves, 2012), the interviewed principals acknowledged being majorly involved in financial, managerial and administrative tasks, which required more attention, to the detriment of pedagogical issues. However, school leaders also felt they did not have the necessary capabilities to do so and repeatedly expressed that they lacked adequate skills when dealing with managerial and financial aspects. This may be related to a lack of training before becoming a principal4. Consequently, leaders’ involvement in managerial and administrative work translated into less time to support teachers. This aspect repeatedly emerged in interviews with teachers, who felt that they were left to work independently.
The coordinators do not have much time to work; they have a lot of bureaucratic things to do and little time for teachers (School A, Teacher 3).
School coordination is very poor. We need to do everything by ourselves, [...] you cannot even talk to them because they have so many things to do and people to deal with; we are missing time for this (School A, Teacher 5).
Multiple understandings and gaps in shared sense-making
The principals in the four schools had different views regarding accountability and understood test-based accountability differently. Some regarded it as a means of being transparent and accountable to society (i.e., decentralization accountability), as a positive tool to improve a school’s instructional planning through school reviews and testing (associated with a form of management accountability), as a form of systemic alignment (i.e., administrative/bureaucratic accountability), as a means for parents to choose their children's school (i.e., market accountability) or as a form of educational quality assurance (i.e., performance-based accountability). This is not surprising since the concept and type of accountability can be very differently defined (see Leithwood, 2010; Shipps & White, 2009).
In line with the sense-making framework, the interpretation of principals regarding the type of accountability also influenced the perceived accountability pressures and the value given to the external test. More specifically, those principals who understood test-based accountability as a transparent, democratic mechanism vis-a-vis society, as generating strategic planning and improvement processes or as a need to standardize and align education, also viewed the standardized test favourably and did not perceive it as increasing pressure. This is clearly illustrated in the following quote:
I think the external test is used in this sense, in the meaning of transparency. Because in public state policy, we need to show society and be accountable to the community. Prova Brasil is a test that I think is excellent when made public. Everyone can read the school’s results […]. Prova Brasil indicates if all school action is channelled towards student learning (School C, Principal 1).
On the contrary, principals who associated the publication of test results with parental choice, funding or market-based mechanisms also had an unfavourable view of the test. In fact, the market-based mechanism and pressure deriving from parental choice were found to have a detrimental influence on principals, translating into a negative view of the test and its consequences.
The interviews also highlighted a clear gap between leaders’ and teachers’ understanding, knowledge and value attributed to the external test. Teachers were generally more critical of the test for accountability purposes and did not see it as particularly useful for students’ learning. They often felt that it was a limited method of evaluating students, used for political rather than educational purposes.
I am very critical about external evaluations, you know? I think the discourse is that they help achieve quality standards. Still, I do not believe it [...] I think these evaluations are conducted for political reasons rather than actually improving education (School A, Teacher 3).
Other principals and school leaders, on the contrary, generally perceived the test as a valid instrument with which to evaluate school and student learning and regarded its use as a government requirement. This may be related to teachers’ hierarchical position, as they are less exposed to the policy message than principals. This gap in understanding and positional power is highlighted by a participant, who argues that:
It is like the story of the phone without a wire. What comes from up there or what the administration sends here arrives with few explanations, which causes unease among teachers because you do not have adequate information. If they gave more information and details, we could be in agreement. But if you ask people to do something and do not provide an adequate explanation, there will be more uncertainty. If you speak to the leadership team, they will probably have had more contact with those above us, so they will have a better idea about the politics and these evaluations (School A, Teacher 2).
Directive and strategic leadership practices
To align with external accountability demands, school leaders in the four schools adopted different practices and strategies. Contrary to the literature, principals did not adopt direct or indirect mechanisms of change, but a combination of both, regularly prioritizing one over the other.
Often, these practices were strategically aimed at showing a positive image of the school to the external administrators, translating into a need to ' marketize' the school and its performance.
I send e-mails to the education secretary, communicate school results and take part in positive propaganda. I have constant and favourable contact with them, take pictures, and show them that the school is working well (School D, Principal 1).
By feeling pressured to comply with the external accountability mandate, school leaders adopted direct forms of control over teachers to ensure adherence to government objectives, such as transmitting pressure and a sense of responsibility. Adopting this behaviour may be related to the difficulty of aligning teachers’ work to external accountability pressures and supporting teachers’ commitment to change (Leithwood, 2010). In fact, principals often regarded teachers as demotivated and unwilling to change, and some lacked trust in teachers to respond effectively to their instructions.
The problem is that sometimes teachers have received full training, but sometimes it seems they have not been trained well. The pedagogical part, I think, sometimes they miss out on this. They do not have that attitude, that behaviour in class. So, I take a class; with me, they are silent. Then you put the same class with a teacher, and the students are noisy and do not perform well, which happens a lot in schools (School B, Principal 1).
Practices such as instructing teachers to teach to the test, to use test simulations or to train students for the external test have also emerged. At the same time, opportunistic behaviours were incentivized, such as helping students during the external assessment, in the form of “cheating” which was acknowledged by teachers.
That is why I think that such an external assessment gets to a point that it is made-up [...] the principal told me, “Give... give them a little help” ... like make it up, you know? “If you look at the answer and see that the student has made a mistake, look at his mistake and… see if you can help,” which is almost giving him the answer, you know (School D, Teacher 4).
In the four schools, school leaders also instructed teachers on the objective and sense of the reform. In this sense, leaders adopted an indirect mechanism of change that focused on discussing and communicating external test results with teachers and supporting changes in their teaching practices. Leaders said they reviewed teachers’ pedagogical strategies and evaluation methods and supported teachers’ capacity to adopt new evaluation forms, more closely aligned with the external test model (i.e., test-friendly-evaluations). However, this practice may again be associated with the idea that school principals perceived teachers as having difficulty adjusting their teaching methods or being unresponsive/unwilling to change.
There are different ways of evaluating students; sometimes, teachers only give the same type of question, but this is being assessed differently. So, in this case, what we do is orientate teachers, show them how to evaluate the same thing but with different activities, so they do not have difficulty with it […] because students will have to take an external test, you understand? (School A, Pedagogical coordinator 1).
To sum up, the shared idea of a teacher as someone who needs to be motivated, directed and externally controlled seems to be present in all schools. This aspect may have influenced the adoption of controlling, directive or instructional leadership strategies, perceived as more effective by leaders in terms of aligning with the accountability mandates. Such a belief confirms that control mechanisms are in place where there is a lack of trust between stakeholders (Cerna, 2014). It also highlights the tension between the pressures deriving from the accountability mandate and the need to control teachers’ work, which principals generally experience in high-stake accountability regimes (Shirrell, 2016).
Different trends and their mediating factors
The four schools also display different enactment trends shaped by the three main factors, illustrated below:
Leaders’ positional power and gender
Leaders’ positional power in the organisation and gender emerged as the first mediating aspect. According to the literature, female principals are expected to experience less tension between internal and external accountability pressures than their male counterparts, and their moral and professional obligations are better aligned (Shipps & White, 2009). In this study, however, male principals were found to experience less conflict between the requirements of the policy demands, and the practices adopted, and they displayed a more coherent, rational and strategic decision-making approach.
Women in lower leadership positions (such as pedagogical coordinators) had significant contact with teachers and worked with them more, hence they also perceived greater conflict when they needed to exert control and hold teachers accountable for school results. This proximity with teachers is evident in reflexive judgements and in a more understanding and accommodating attitude towards teachers.
We also have this human affective aspect because you would not achieve good results without this combination of factors. We pretend, but at the same time, we are also kind to everyone (School C, Pedagogical coordinator 1).
Leaders in the leadership team defended and considered teachers’ requirements and viewpoints, as well as those of students, actively prioritizing forms of moral accountability (Normore, 2004) over external accountability. They also expressed higher morale, professional reasoning and criticism of the policy mandates. This aspect is even more important in a group setting, since “group members will elect a leader who seems capable of representing the group’s best interests” (Appelbaum et al., 2003, p. 46).
They [administration] ask for quality and look for specific results but do not accept that some students do not learn and have difficulty... so what can you do? I am not immune to this, and teachers are right when they say, “you want me to invent a grade?” teachers are not so wrong (School C, Pedagogical coordinator 1).
In line with the view of Court (1998), a shared and democratic leadership was more evident amongst women than men. Teachers’ perception of the principal’s role was associated with political responsibility, managerial capacities and top-down directive behaviours. Teachers often perceived male principals as unavailable and distant, and their behaviour as managerial or authoritative. On the contrary, teachers expressed a closer relationship with women leaders, who also mainly held lower leadership positions.
I have a closer relationship with the pedagogical supervisor because they (the principal and vice-principal) are occupied with other things that need resolving with the education secretary. We have more access to her (the pedagogical coordinator) and can share and exchange ideas with her. When I have a personal issue that I need to solve, I go to her and talk to her about it, so we have that closeness, you know? (School D, Teacher 2).
Schools’ socio-economic composition
The socio-economic composition of students was a second aspect influencing the way in which leaders perceived and responded to accountability demands. In poor socio-economic contexts, principals adopted an instructional strategy and pedagogy, based on the school’s major problems and priorities. Priorities included focusing on students’ discipline and increasing their educational attendance while at the same time maintaining an organized and disciplined school environment. Not surprisingly, to meet these goals and improve student performance, principals in low SES schools adopted what was perceived to be an "effective traditional and conservative pedagogy" instead of a progressive, child-centred, invisible pedagogy (cf. Bernstein, 2003).
What I think is missing is a strategic vision. Our indicators are improving. Still, it is a constant and slow improvement [...], so I looked into the use of textbooks; we set up a whole policy regarding books and organized the timetable to facilitate students. I took back much of the conventional model of schooling because I do not know in other contexts, but in this school, students need it; it is a socio-cultural problem of the family, and students need to know how to study by themselves. So, we slowly gave them things to do at home, to be responsible at home [...] maybe this was thought to be very conventional and traditional, but I think it worked here and made the school improve (School C, Principal 1).
The previous quote is aligned with Bernstein's (2003) argument, namely that pedagogical forms of traditional, conservative and visible practices, instead of child-centred, progressive or invisible practices are more likely to be adopted in lower-working class communities. Child-centred progressive approaches assume the existence of specific and more advanced forms of organizing the timing, pace and language of teaching and learning. These aspects are “less likely to be met in class or ethnically disadvantaged groups, and as a consequence, the child is likely to misread the cultural and cognitive significance of such classroom practice, and the teacher is likely to misread the cognitive and cultural significance of the child” (Bernstein, 2003, p. 211).
Students' low SES and associated behaviour have also influenced teachers’ sense of work, commitment and (dis)satisfaction, increasing the pressure they experienced to comply with accountability demands. In fact, in schools with a lower socio-economic composition of students, teachers (and all staff members, generally) had difficulty being listened to by students and perceived a greater de-motivation, frustration and pressure as regards increasing students’ results and complying with the accountability demands. Teacher de-motivation also rendered the principals’ objective of aligning the school and teachers’ work to the accountability mandate more difficult.
Many times, teachers are dissimulated, you know? Because as much as you can dedicate yourself to teaching, as much as you are committed, you speak, speak, but you do not manage to help that student understand. At the same time, you must also achieve what has been determined and stipulated (School C, Pedagogical coordinator 1).
This last aspect may also be related to the fact that school staff working in more challenging circumstances did not consider educational content meaningful for the student population. The curricular content was perceived as irrelevant, and the role of education as a means of improving students’ life chances was also deemed limited. This consideration negatively influenced teachers’ motivation to effectively stimulate and enhance students’ learning, let alone achieve positive results in an external test.
It is even more difficult for some teachers, and I hear many of them saying, “I do not think this has any meaning for this student... I know I am doing this for nothing” I have heard this many times (School C, Pedagogical coordinator 1).
It has emerged that the schools’ socio-economic context influences teachers’ expectations of students and their learning (Diamond et al., 2004; van Maele & van Houtte, 2009), which also affects teachers’ perception of their self-efficacy. However, it is essential to acknowledge that in such contexts, good relationships and a favourable climate in school played a decisive role in mitigating the perceived negative effects associated with the socio-economic composition and pressure to achieve.
The leadership manages to create a good environment and climate, which is fundamental in a school, where you do not feel that pressure to be here, where students’ behaviour has a detrimental effect on our teaching (School C, Teacher 4).
Schools' administrative dependency
The final aspect that was found to shape the enactment of accountability reforms in schools was the difference in administration dependency (state and municipal schools).
Firstly, principals’ and teachers’ training varies amongst state and municipal schools, influencing principals’ perceptions of their capacities and roles. Participants (teachers and part of the leadership team) agreed that adequate training was provided, especially in administrative and managerial areas. However, the leaders in municipal schools enjoyed free in-service courses and training to increase administration and school management skills (Secretaria Municipal de Educação de Belo Horizonte, 2017). Those principals who did not have any background in administrative or managerial areas found these tasks more challenging. On the contrary, one principal with a degree in administration and school management studies affirmed that she preferred working in these areas. Therefore, adequate training influences the perception of one’s own self-efficacy, namely, the capacity to meet expectations and demands (Tschannen-Moran & Gareis, 2004).
Secondly, in contrast to state schools, municipal schools had a closer dialogue and received more instruction and support from educational authorities regarding policy reforms5. Principals in municipal schools, in fact, also felt less pressurized to comply with accountability demands; they also better understood and made significant use of the test results. Indeed, those principals working in municipal schools were more informed and knowledgeable about the tests’ composition and consequences and were more conscious of the development of the school’s results, considering the test a useful diagnostic instrument to evaluate and monitor students’ achievement. These principals were of the opinion that a clear and positive understanding is essential, believing that “if the purposes, intentions, roles, and expectations are clearly understood, the chances for successful accountability systems are enhanced” (Normore, 2004, p. 58).
We organized ourselves and engaged in a dialogue with the regional secretary. I never had any problem with them; I told them, “The reading we did in our school is this; what we should improve is this.” We have this dialogue, yes. I do nothing covert here. They listen to me, and we have a good relationship (School C, Principal 1).
Leaders who had a positive understanding of the reform also shared this understanding with teachers, supporting teacher’s acceptance and internalization of test results, as emerges clearly from the words of the interviewed principal reported below:
Teachers did not have a reference and did not understand what these evaluations meant and had more resistance towards them […] They started to see this as a ranking, schools against one other, but in reality, schools should look at other schools […] Now they accept them more; they participate in them. They need to understand why this is the case, that they are not just accountable to the government, but also to society. I always discuss this, so Prova Brasil has always been incorporated into the school by the professors (School C, Principal 1).
Finally, teachers’ working conditions (i.e., contract status) differed between the administrative units. Teachers experienced poorer working conditions in state schools, due to flexible contracts6. Due to a greater teacher rotation and instability, there were fewer opportunities for pedagogical exchanges and collective learning in state schools. This lack of shared learning moments influenced staff cohesion and trust and the principals’ ability to sustain a shared learning community. This aspect also influenced teachers’ perception of “working by themselves” and feeling “abandoned,” undermining their sense of self-efficacy.
In the end, the discourse is always “what did the professor do? She did nothing,” but the professor did, except that she could not do everything by herself (School B, Teacher 1).
Poorer working conditions and material resources in state schools also undermined teachers’ overall satisfaction. Teachers complained repetitively about their working conditions, referring to issues of work and activity overload, poor salaries and a lack of resources7. As a result, teachers perceived their work and the achievement of quality results in state schools to be more difficult. On the contrary, in municipal schools, the presence of collective moments of shared learning, in which teachers had the chance to discuss pedagogical aspects, was found to support teachers’ sense of self-efficacy and job satisfaction, as can be appreciated in the following quote:
When, during break time, we talk amongst ourselves, teachers start understanding more about the students’ learning processes, but we did not have these collective moments, I do not know [...] when we work together it is much more productive you know? (School A, Teacher 3).
Discussion
This study aimed to analyse the mediating role of school leadership in the area of accountability in four different schools within Belo Horizonte (Minas Gerais, Brasil), adopting a combination of a contingent and sense-making framework and using a realist, analytical evaluation approach.
Following other studies, performance-based accountability has intensified the principal’s role in school management and has shaped schools’ pedagogical focus, with school actors’ work being based on performance indicators and ruled by a preoccupation and necessity to comply with targets and results. This aspect translates into a perception of work overload, self-responsibility and less time devoted to adequately instructing and supporting teachers. Therefore, the provision of support for principals’ training and the need to build the necessary leadership capacities may be recommended to assist in the implementation process in schools.
The analysis also reveals that school leaders perceive pressure of accountability from external sources and understand the mandate differently. The sense-making that principals have with regard to the reform appears crucial, as this influences the value and perception of the accountability message. In addition, in this study, leaders adopt direct and indirect practices to align with the policy mandate. However, external pressures to comply with accountability mandates mainly result in principals adopting controlling and directive behaviours towards teachers. Therefore, the extent to which a shared democratic leadership can emerge, in a context where school principals feel the need to control teachers’ work or face more difficulties, is questionable. On the contrary, a system of horizontal school accountability between the school leadership team and government structures, and internally between the leadership team and teachers, characterized by non-hierarchical relationships and supportive rather than controlling mechanisms, could constitute a valuable way of aligning and coordinating policy intentions.
Leaders’ positional power, in terms of task distribution, and gender emerged as crucial when distinguishing between a distributed and collaborative leadership style or a managerial and an authoritative one. “What it means to be a leader” in a demanding accountability context is usually associated with the attitudinal drivers of men in leadership positions, characterized by structure, instruction-giving and transactional aspects (Appelbaum et al., 2003; Court, 1998). Without essentializing gender attitudes and nature, since there is no universal ‘woman's way of leading’ (Court, 1998), balancing autocratic, managerial and human/social leadership traits may be essential. This equilibrium could overcome internal school conflicts and facilitate a favourable climate for implementing policy reforms, challenging the “traditional and managerial notions of `the principal” in a hierarchical environment (Court, 1998). In addition, in a context where there is a clear gap between males and females in positions of responsibility, it may be advisable to support the attractiveness of the principal’s work and distribute (“stretch” (Spillane et al., 2004)) leadership responsibilities to female participants in schools, as a means of overcoming traditional gender roles in the profession. Acknowledging the socially distributed role of leadership also emerged as crucial, since other leaders in the school, who worked closer to teachers and students, more often prioritized a professional and moral accountability over an external accountability, and teachers experienced more favourable relationships, opinions and work climate with them than with the principal.
Schools’ intake in terms of students’ socio-economic backgrounds was found to play a significant role in shaping principals’ practices in the four schools. In poor socio-economic contexts, leaders focused on aligning direct practices relating to the school’s curriculum and instruction and adopting a traditional rather than a progressive pedagogical structure (Bernstein, 2003). They also had greater difficulty supporting teachers’ engagement and motivation, due to teachers’ low levels of trust and expectations regarding students’ futures. Positive relationships between staff and school leadership mitigated some of these difficulties in such contexts.
Finally, regarding the school’s administrative dependencies, teachers in state schools (with flexible working contracts) expressed greater dissatisfaction, involvement in policy decisions and less cohesion than in municipal schools, which undermined the creation of a shared professional learning community in schools. Moreover, a closer dialogue and support from educational authorities were more in evidence in municipal schools, mitigating principals’ perceived pressures and increasing their understanding and use of testing data. Hence, investing in and sustaining teachers’ working conditions and schools’ capacities to understand, use and integrate data in school practices effectively appears to be fundamental.
To conclude, school principals play a crucial role in enacting accountability reforms in Brazil through different behaviours and practices. However, several contextual factors question the way in which such practices support teachers’ positive understanding and implementation of the reform. Since diverse variables shape and mediate principals’ practices and policy translation in schools (Braun et al., 2011), it is worthwhile building upon policy research that considers the contexts and contingencies of principals’ leadership.
Future research lines could integrate a gendered and distributed perspective into studies on leadership and accountability. Secondly, in the context of other federal/decentralized countries, analysing the difference in policy enactment within different administrative units (i.e., municipal/state schools) is crucial; in the case of Brazil, this emerged as being a relevant variable in explaining adopted practices. Finally, ethnographic visual techniques, such as photo-elicitation methods (see Werts & Brewer, 2015) may facilitate the way in which school leaders and teachers experience accountability pressures in their daily lives and integrate such experiences into their discourses and negotiated practices.
