teaching critical thinking and metacognitive skills through philosophical enquiry. a practitioner's report on experiments in the classroom.

In 2017 The Philosophy Foundation and King's College London carried out a small-scale study into teaching critical thinking skills in primary school philosophy classes with children aged 8-10. The study compared The Philosophy Foundation's community of philosophical enquiry (PhiE) approach against a new addition of the explicit teaching of critical thinking skills within the enquiry itself. Although there were limitations² to the first year, it was used to inform developments in the follow-up study of Autumn 2018, which this report describes. This study has transformed TPF's practice by resulting in a method which supports both metacognitive and the explicit teaching of critical thinking skills in a way that is mutually integral (where the CT skills are the tools of metacognitive strategy and where the metacognitive structures have the concrete CT skills to appeal to).

critical thinking, metacognition and philosophical enquiry

In 1990 a statement of expert consensus was created to define critical thinking (CT) using a qualitative research methodology known as the Delphi Method (^{Facione, 1989}). 46 experts were consulted and the consensus statement begins:

We understand critical thinking to be purposeful, self-regulatory judgement which interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgement is based. CT is essential as a tool of inquiry³.

CT is outlined here as both metacognitive - the awareness, evaluation and strategising about one's own thinking in order to improve it - and necessary for enquiry.

In his paper, Plato, Metacognition and Philosophy in Schools (2018a), Peter Worley makes a case for how philosophy and philosophical enquiry are linked to critical thinking and metacognition, with some recommendations for how this might be done. According to Worley, though critical thinking skills are not necessarily metacognitive per se, they are helpful tools when answering the metacognitive questions linked to making progress in philosophy: e.g. what is wrong with an argument? Or, how do we make progress and improve what we are claiming? And although critical thinking is not sufficient for doing philosophy (one can be using CT skills to assess an argument in a newspaper, for instance), it is a necessary element: to be able to do philosophy properly one will need at some point to use CT skills. Without the critical element it is easy to fall into merely agreeing or disagreeing with one another based on opinion-sharing (^{Worley, 2018b}), and this is not sufficient for a philosophical enquiry session with children to become critical nor metacognitive, and arguably, therefore philosophical. For metacognition children should not only be able to make logical moves but also be able to reflect on the use of these tools, their effectiveness and appropriateness. For instance, not only to use a counterexample but also to be able to understand what a counterexample is, when it is effective and what role it plays in the conversation as a whole.

This means that for metacognition to occur in a philosophy session it is not enough that children intuitively use what might be described as a CT move (such as making a counterexample: 'I disagree that for something to be alive it has to have a heartbeat because a tree is alive but it doesn't have a heartbeat'); it is not enough that CT skills have been taught to the children and that the children have grasped and understood how to use them in tests; it is not enough that they undergo some kind of cognitive shift such as changing their mind because of what someone else has said, and it is not enough that the children are asked to reflect on how the session went ('Did everyone listen well?', 'Was there anything we could do to improve how people listen?'). ^{Worley's (2018a}) definition of metacognition is a strong one and results in quite a tall order for the students in that it is intellectually exacting: they must, in addition to having things to say in response to a first-order question, have some awareness that certain kinds of cognition have occurred, they must in some way monitor their progress and finally they must strategise how to use the tools and skills at their disposal as well as evaluate how well their strategies have gone.

p4c and critical thinking skills

Although critical thinking⁴ is linked and promoted as skills that philosophy and philosophical enquiry can develop, Alvarez Ortiz, C. M. (2007) concludes in her paper Does Philosophy Improve Critical Thinking that there is 'insufficient evidence to be confident that studying philosophy improves critical thinking (CT) skills any more than studying other academic disciplines.' Ortiz indicates that studying philosophy is actually less effective than studying CT skills themselves, and that there are techniques that could be introduced into philosophy to improve CT skill development.

‘Philosophy for children’ (P4C) began with the work of Professor Matthew Lipman, who founded the Institute for the Advancement of Philosophy for Children at Montclair State University, USA in 1974. Inspired and informed by Dewey’s developments of Pierce’sCommunity of Inquiryideas (Pierce’s was a ‘community ofscientific inquiry’ while Dewey’s was a ‘community ofphilosophicalinquiry’), Lipman believed that young children possess the tools needed to think rationally, and through his method of reading philosophically stimulating narratives, a community of philosophical enquiry, thinking activities and logic exercises he believed that children could develop the critical thinking skills he saw lacking in his pupils at Columbia University.

This dialogue-driven community of enquiry method would go on to influence other approaches to doing philosophy in the classroom all over the world. However, Lipman's P4C curriculum, with its thinking skills and logic exercises, has been used less over the last 20 years, particularly in the UK where this study took place. The curriculum and manuals are out of print in English and although there were experiments with Lipman's P4C curriculum in the UK in 1993 (^{Williams 2016}), some drawbacks to using the novels and teaching material became apparent. This from Steve Williams, A brief history of p4c, especially in the UK (2016):

P4C teachers and trainers in the UK started developing their own materials (^{Fisher 1996}), and moved into using a 'found curriculum' - such as using picture books (^{Murris 1992}, ^{Murris & Haynes 2002}, 2011), films, pictures, stories, poems, objects etc to stimulate philosophical enquiry. This found curriculum does not have the explicit teaching of reasoning skills through activities as Lipman's curriculum had, but still uses the community of enquiry pedagogy, and has the advantage of being accessible to a wide range of young people / levels and can be integrated within curriculum subjects easily. Although there was a move away from Lipman's curriculum the community of enquiry pedagogy is still the predominant way of doing philosophy with children in the UK (see ^{Williams 2016} and Sutcliffe in ^{Anderson 2017}):

There was, it must be admitted, a glossing over of the fact that not all of the research results involved Lipman's materials… What can be said, with some security, is that a common strand to all of the projects has been the intended and actual practice of a community of enquiry approach, and this has increasingly been the banner under which P4C has been promoted, at least by SAPERE. (Sutcliffe, 2017, p. 7)

The community of enquiry model which can be said to activate the dispositional aspects of CT highlighted in the Facione report (^{Facione, 1989}, Table 1, p.3),

The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgements, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeing relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit. Thus, educating good critical thinkers means working towards this ideal.

It is important to distinguish between dispositions and skills, as one may have a disposition towards making certain thinking moves, such as making or objecting to inferences without knowing that this is part of the discipline of critical thinking (i.e. modus ponens and the other related fallacies). The ideal critical thinker has dispositions that are developed through enquiry: judiciously inquisitive, reasonable, and open-minded⁵; and CT skills that can be used towards making good judgements: evaluation, re-evaluation and systematic structures of thinking. Pupils may be disposed to make use of what might be described as 'thinking skills' but they may not use or recognise them as skills. This has implications for the skills' evaluative power (see page 4) within an enquiry.

Prior to this study The Philosophy Foundation used a community of enquiry model - Philosophical Enquiry or PhiE (^{Worley 2011}) - with no explicit teaching of CT skills. However, TPF use dialectical questioning strategies that have been designed to activate CT dispositions from the group. For example, the techniques of 'anchoring' and 'opening-up' (^{Worley 2011}) elicit the pupils' argument structures (premise and conclusions), and the technique of 'iffing' encourages more careful consideration of inferences ("if…then…" and "just because…doesn't mean that…")⁶.

There are many advantages to using communities of enquiry to do philosophy with children and the latest meta-analysis by Moriyón, Botella, Centena-Gutiérrex and González-Lamas (2018) shows support for the existence of a positive effect of a variety of P4C interventions (all communities of enquiry) on cognitive skills.

Although the meta-analysis shows a positive impact on cognitive skills, it doesn't say anything particularly about critical thinking skills. Reasoning tests used ranged from cognitive abilities tests (CAT scores) to the New Jersey Reasoning Tests, which were developed specifically to measure the impact of Lipman's original curriculum. Although they are not synonymous, reasoning skills are linked to critical thinking: one needs to use reasoning in order to benefit from using CT skills - but reasoning tests say nothing about the techniques or skills used within an enquiry, nor do they say anything about pupil's abilities to successfully use specific skills associated with critical thinking.

As a way of considering if the community of philosophical enquiry alone is enough to improve critical thinking, and to preserve the advantages of running philosophy as a community of enquiry, this study tested to see whether the addition of the explicit teaching of critical thinking skills within the enquiry would enhance children's critical thinking abilities above what happens in a PhiE alone. The teaching methods used were developed within TPF's practice, although they are similar to other methods used by the international P4C community, such as Philip Cam's 20 Thinking Tools (Cam 2007). Cam's thinking tools capture a broad picture of thinking in a community (which includes CT skills, but is not limited to them) and this study focused on CT skills specifically. There has not been, as far as the authors can determine, a study into the development of CT skills in communities of enquiry compared to teaching CT skills within an enquiry.

communities of enquiry and metacognition

Research⁷ and meta-analysis has shown that the development of metacognition helps improve learning across a range of subjects and disciplines. According to the meta-analysis completed by UK based research centre The Education Endowment Foundation ( ^EEF ):

Metacognition and self-regulation approaches have consistently high levels of impact, with pupils making an average of seven months’ additional progress. These strategies are usually more effective when taught in collaborative groups so that learners can support each other and make their thinking explicit through discussion. The potential impact of these approaches is high, but can be difficult to achieve in practice as they require pupils to take greater responsibility for their learning and develop their understanding of what is required to succeed.⁹

Metacognitive thinking has also been linked with philosophy and P4C: ‘The cultivation of metacognition, or meta-thinking, is probably the best gift P4C can give’ (Gasparatau 2016). The EEF recommends P4C for developing metacognition (MC) skills¹⁰ despite providing no explanation of any link between the two¹¹. So, although there are claims to P4C’s effectiveness at developing MC (e.g. ^{Fisher 2007}, ^{Oyler 2015}, Gasparatau 2016, ^{Murris 2008}), there is little research to show that P4C bears this out beyond some case studies¹².

Attempts to bring metacognition into TPF's standard PhiE sessions over the past ten years have proved difficult because students, on the whole, don't find it easy to distinguish between metacognitive questions and their first-order considerations. In other words, asking a more metacognitive question such as, 'how can we best answer the question?' in most cases, either results in responses such as 'it's just a matter of opinion', 'we need to listen to each other's opinions', and, 'there's no right and wrong' (either confirming subjectivist assumptions or are merely procedural), or it results in students simply returning to the first-order question in a decidedly non-metacognitive way ('I think we should answer the question by saying that it's not his property because…' where the first-order question is 'Does Og own the land s/he has fenced off?'). This general observation is confirmed with examples of both of these kinds of responses occurring during the first sessions with both the control and intervention groups in this study.

TPF also noticed that during a typical PhiE session, though there were plenty of examples of children making important thinking moves (e.g. identifying assumptions, challenging inferences, identifying an infinite regress and many more), there was little evidence of this being metacognitive. That is to say, there was usually no awareness of what they had done or knowledge of the move they had made beyond their performing it or, perhaps most notably, recognition of the move's evaluative power. For example, counterexamples are very common from around age five and upwards, but they are usually seen as just one of many opinions rather than - if it is a good counterexample - a definitive determiner of the truth or falsity of a claim. Following Worley's conceptual link between metacognition and philosophy (2018a) this study was developed with the aim of seeing whether a PhiE session yields any critical thinking and metacognition, and whether the 'Menonic' method developed in Worley's paper and the teaching of CT skills within an enquiry, would further develop CT and MC.

pedagogical approach: teaching ct skills and developing metacognition

In order to develop metacognition skills explicitly during a philosophical enquiry ^{Worley (2018a}) created a 'Menonic' method of doing philosophy, inspired by a model he identified in Plato's Meno and Theaetetus (^{Worley 2018a}). This method extends the usual model for conducting a PhiE session in schools (^{Worley 2011}, 2015a, 2015b) to include a systematic metacognitive dimension within the sessions themselves (^{Worley 2020}). This includes not only answering philosophical questions (e.g. 'Can you choose what you believe?') but also a metacognitive question about how to answer the main question (e.g. 'How can the main question best be answered?'), the explicit teaching and subsequent use of critical thinking skills (e.g. 'inference-making' and 'objections to inferences'), and a device for having one or two pupils each session take a synoptic view of the discussion as a whole with structured questions (PIES[S]) for helping them evaluate and strategise the progress of the discussion.

The pedagogical approach to skill teaching within philosophical enquiry was also informed by the work of Dr. Ellen Fridland on skill-learning and refinement in conceptual development (^{Fridland 2013a}, ^{Fridland 2013b}, ^{Fridland and Moore 2014} and ^{Fridland 2018}). In the pursuit of refining and improving one’s own cognitive skills, Fridland's emphasis on technique, allows pupils to begin thinking about their own thinking. Fridland states that explicitly considering the appropriateness and effectiveness of CT skills, immediately opens up the development of metacognition.

This study aimed to measure the development of CT skills and metacognition during PhiE and compare this with classes which have the addition of teaching CT skills descriptively (see below) and a focus on metacognitive questions, integrated into usual PhiE (not teaching them in isolation) using the Menonic method. By comparing the results of normal PhiE sessions to PhiE plus the teaching of CT and MC skills, it could be ascertained whether this addition would be worthwhile including in future classes and training for facilitators, as well as making a clear link between philosophical enquiry and the development of critical thinking and metacognition, and thus to the wider research on the impact of metacognition.

study

For the follow-up the study in 2018 there were four control and four intervention classes of Year 6 children (age 10/11) all of whom had had philosophy sessions with The Philosophy Foundation for a number of years. The three primary schools in South East London all have above average number of pupils who are supported by pupil premium¹³ - meaning they have barriers to learning. All four of the philosophers who facilitated sessions in the first year were involved in the second.

Control and intervention classes were matched within the same year group and same school to mitigate against difference. There were 220 children involved in this study. Control classes had normal Philosophy Foundation PhiE sessions and the intervention classes had normal sessions plus the explicit teaching of CT and MC interventions as described below based on Worley's Menonic method. There were twelve weeks in total, all sessions were filmed. The first and last sessions were used as data points to compare the classes CT skills and MC progress.

teaching critical thinking skills

As part of the teaching of CT skills to the children TPF facilitators labelled a skill and gave a description of the skill after they had been used naturally during an enquiry, teaching the CT skills descriptively rather than prescriptively: that is, teaching them according to what they in fact do rather than what they should do. For example, teaching counterexamples after someone has used a counterexample naturally during discussion (descriptive), rather than first teaching counterexamples and then setting exercises to test understanding of counterexamples (prescriptive). This also shows the power of the skills used in helping the group to re-evaluate their thinking. Once a good counterexample has been used the original statement must be revised. For example,

Although labelling moves during an enquiry is done by some P4C facilitators in the UK and internationally (e.g. ^{Oyler's 2015} paper on expert teacher contributions highlights 'naming' as one of the seven facilitator moves), it is not something TPF has practiced systematically in the past, nor is it widespread in UK practice. The teaching of the CT skill is not labelling alone, it requires a description of the skill itself and encouraging the pupils to think about how they can use them during the enquiry. So rather than facilitator moves such as “So you are making a distinction.” “You agree with John then.” “I can add another example.” (^{Oyler 2015}) the facilitators ask, "are there any tools we could use to help us answer this question?" "Has anyone made use of a CT skill today?" It is an awareness of the CT skills used, and the ability to purposely and strategically use them to progress the discussion that is metacognitive.

Once a skill had been labelled and explained they were put on a 'thinking wall' for all to see. Sentence templates were given where appropriate for some of the CT skills taught (see below) and put on the board next to the named and described skill.

As the skills were only being taught after the children in the intervention group had used them it was not possible to prescribe which skills would be taught beforehand in the second year of the study, though the facilitators, as experienced philosophy teachers, had a good idea which ones were likely to come up. This differed from the first year of the study where it was decided beforehand which skills would be taught: counterexamples, distinctions and inferences. These skills were chosen based on extensive classroom experience and knowing which philosophy sessions would engender particular CT skill use. For example, distinctions need to be drawn around 'freedom' when discussing the story of The Sirens from The If Odyssey (^{Worley 2012}), and children frequently do this when discussing the main question: who is the most free?

Some of the teaching elements of the previous year were used - for example teaching what a counterexample is through the use of a definition ("an example that goes against a general claim") and examples on slides (general statement: "all birds fly" counterexample: "penguins are birds but they do not fly"), but examples were taken from the class rather than using the pre-designed slides in the second year of the study. Games and activities were used as another way of teaching and reminding the children about the CT skills they had learned in previous weeks. For inferences, for example, the class were given a statement and they have to say what follows from it: "John has a big house, what do you think that means?" Often this game would lead to inferences and objections/challenges to inferences:

It was decided to teach no more than 3 CT skills over the course of the term, although the classes labelled new skills themselves, so each class learned between 4 and 6 skills in total. All the skills taught went up on the wall as a prompt. After the programme, a poster was given to each class to aid them in continuing to use the skills outside of the philosophy classes. The taught CT skills are listed below, and there were certain skills that the children came up with, labelled themselves and used as part of this programme. They have been marked below with 'class'.

Making an inference: template 'If… then that means…' (e.g. ‘If it’s got a mental brain then that means it has to have a physical brain because you need to have a physical brain to have a mental brain.’)

Challenging an inference: template ‘Just because… that doesn't mean…' (e.g. 'Just because it has arms and legs does not mean that it is human.')

Qualifications: template 'Just because… that ________ ', where the children insert one of the following qualifications:

Which ones were used, again, depended on the language used by the pupils.

Conceptual Meaning: template ‘It depends what you mean by X.’ This was sometimes reduced to the acronym 'IDWYMB' said as a word, 'idwib'. Some children started using #conceptualmeaning.

Distinction: e.g. ‘Trees don’t have a physical brain, they have a mental brain.’ Working definition: When you say what is different about two or more ideas/concepts. There were two kinds of distinction that were focused on:

Sentence templates introduced to classes for distinctions were as follows:

Challenging Distinctions: e.g. ‘There isn’t more than one kind of X.’

Exclusion (class): e.g. ‘A friend is…’ / ‘A friend is not…’ Saying what X is not as well as saying what X is.

Relating (class): e.g. ‘It’s like when I was at home and…’ Seeing how we can relate ourselves to the question/characters/situation. Similar to the use of analogy.

Categorisation (class): e.g. with the question ‘Is CB (a computer robot) a friend?’ and the assumption that a friend is a person, seeing if CB fits the category of person in order to qualify as a friend. Taking a relevant category (e.g. ‘human’, ‘person’) and then seeing if X fits the categories.

Context (class): e.g. May is real in the context of the story, not in the context of the world. Noting that how we evaluate something can depend on the context in which we view it or the perspective we take up.

developing metacognition

The intervention class had one or two ‘scribes' for each session, and this role changed every week so that all children had the opportunity to scribe over the course of a term. This role involved concept-mapping the discussion on a whiteboard or in a book. Then, halfway through the session, the scribe was asked the following questions (which fall into the acronym PIES[S]):

These questions start with an easy way in - summary and interest, and then move to the more evaluative, strategic and synoptic (view of the conversation as a whole) questions about the progress of the discussion. The whole group was then invited to respond to one or more of these questions or answers, before swapping over the scribe and continuing the philosophy session.

measuring metacognitive progress

In order to ascertain whether there was a difference between the control classes' and intervention classes' metacognition, 'starter' and 'plenary questions' were used in the first and final sessions with all classes. These two questions were written in different colours from the main philosophy question in order to aid distinguishing them and any tasks surrounding them.

The starter question, which was asked after the stimulus and main philosophy question (Task Question) had been presented, was 'How do you think we should answer this question?' This was put on the board next to the Task Question and pupils could choose which question to answer, although facilitators did prompt for the starter question if no-one was answering it. After 5 minutes the session would focus on the Task Question as usual.

The last ten minutes of first and last sessions also had a plenary where the pupils were asked to reflect on metacognitive question/s such as:

It was believed that the classes would be fairly equally matched to begin with, and hypothesised that by the end of the programme the intervention class would be able to answer the starter and plenary questions more easily and clearly, giving examples of the CT skills they could use to answer the main Task Question well.

data collection and analysis

All classes were filmed and a philosophy research assistant from KCL watched through the films of all classes first and final sessions, marking uses and successful uses of the critical thinking skills and instances of metacognitive moves. Successful use was marked for valid reasoning and relevant application of the CT skill. Inferences were quantified as successful when valid. Soundness was not considered in quantification.

Metacognitive use was marked as successful or unsuccessful as follows:

1) Accuracy (covers Awareness, some Monitoring, and Awareness of CT Skill). If a pupil says they're going to make a distinction and then offers a counterexample, the 'Awareness of CT Skill' fails. If a pupil accurately reports what was said by another pupil earlier, the demonstration of 'Awareness' succeeds.

2) Clarity & Consistency (covers Strategy, Evaluation, some Monitoring). Here the success or failure assessment is similar to that of conceptual analysis/definition. If the pupil clearly suggested taking a vote or having a discussion as a way of answering the question, the contribution would count a success because it demonstrates strategic thinking. Similarly, if a pupil clearly highlights what someone else said and then says what they think of that idea, this would be a successful instance of Monitoring or Evaluation. This would hold even if the researcher didn't think the strategy or evaluation was particularly strong, so long as it was clear and internally consistent. If on further questioning, a pupil's strategic or evaluative reasoning became inconsistent or perhaps too unclear, it would count as a failure. Perhaps more common than this kind of failure was a failure to explain a strategic or evaluative idea that wouldn't count as a success on its own. For example (from intervention class week 12):

Facilitator: Do you think [John]’s idea might help us answer the question or not? Child L: Yeah. [Introduction of strategic idea] Facilitator: You do, why’s that? Child L: Because the way of having the core belief, the big belief, and then the smaller beliefs, might, yeah, make us answer the question. [Failure to clearly explain how the idea is useful or how it can help us answer the question]

As with all the general CT skills, and especially in terms of the second way of assessing success (see 2 above), there were borderline cases that were harder to judge. A qualitative report from the researcher was submitted alongside the quantitative data. Qualitative feedback from philosophers, teachers and pupils added to the overall picture.

The questions used to evaluate the success or otherwise of the intervention were:

data results

1. was there a notable increase over time in the frequency of use of ct and mc skills in the intervention group compared to the control group?

Over the 12 weeks the control groups increased their use of CT and MC skills by 6% - going from an average of 29 in week 1 to 30.75 in week 12. The intervention groups increased their use of CT and MC skills by 62%, going from an average of 33 in week 1 to 53.5 in week 12. This shows a 56% increase in frequency of use of CT and MC skills for the intervention group compared to the control group.

Fig. 1. Critical Thinking and Metacognitive skill use, average over 8 classes at week 1 and week 12 

2. was there a notable increase over time in the rate of successful use of ct and mc skills in the intervention group compared to the control group?

Over the 12 weeks the control group increased their successful use of CT skills and MC skills from 20.75 in week 1 to 26.25 in week 12. This is a 26% increase. The intervention group raised their successful use of CT and MC skills from 24 in week 1 to 39.25 in week 12, a 63% increase.

Fig. 2. Critical Thinking and Metacognitive skill successful use average over 8 classes at week 1 and week 12 

3. was there a difference between use and successful use of just ct skills between intervention and control groups?

The control group decreased both their use and their successful use of CT skills between weeks 1 and week 12. Their use went down from 21 in week 1 to 15 in week 12 (29% decrease) and successful use went down from 13.75 in week 1 to 12 in week 12 - a 13% decrease. The intervention group on the other hand increased their use by 4.5% - in week 1 they used 22.5 CT skills and in week 12, 23.5 CT skills. Their success rate increased by 14% - with 14.25 CT skills being successful in week 1 and 16.25 being successful in week 12.

It is worth noting here that although the control group went down, and the intervention group improved slightly, the final session itself was recognized as difficult by all the philosophers involved and the research assistant:

The enquiry into 'Can we choose what to believe?' seems to push and puzzle the pupils more than the enquiry into 'Does Og own the land?' Although pupils generally seem to have an intuitive response they find it harder to give or develop clear supporting reasons for that response. Superficially it does seem a harder question to connect to and is certainly more abstract. This sometimes seems to result in less notable responses in terms of the core critical thinking skills that are being quantified. The metacognitive responses don't seem to be influenced by this increase in difficulty.

4. was there a difference between use and successful use of just mc skills between intervention and control groups?

Metacognitive skills improved in both groups, but more substantially in the intervention group. The control group raised their use of metacognitive skills by 97%, from 8 in week 1 to 15.75 in week 12. They increased successful use of metacognitive skills by 104%, going from 7 in week 1 to 14.25 in week 12.

The intervention group raised their use of metacognitive skills from 10.5 in week 1 to 30 in week 12, a 186% percentage increase. They raised their successful use of metacognitive skills from 9.75 in week 1 to 23 in week 12 - a percentage increase of 136%.

Fig. 6 Metacognitive skills successful use week 1 and week 12 

This does seem to show that philosophical enquiry - both with and without the explicit teaching of MC skills - improves pupils abilities to think metacognitively. It also shows that the explicit teaching of MC skills improves the use and successful use of MC skills - by 30% in this case.

It is also worth noting, though to be expected, that CT skill awareness was zero for all classes in week 1 and in the final session all intervention classes had CT skill awareness marked, whereas the control groups were still not able to use this metacognitive skill. Metacognitive skills were used more by the intervention group in every section of the philosophy session in the final session (starter, enquiry and plenary) by almost 50% in all cases.

5. did the intervention enhance or hinder the philosophical enquiry sessions?

qualitative reports: philosophers

The feedback from the philosophers throughout was very positive. All philosophers commentated that they could see a difference between the control and intervention group in the final session, particularly in reference to the MC skills they were using.

Facilitator 1

The final classes were very interesting. I was really impressed with the intervention group. That will speak for itself in the video. But one other observation. When we have the two questions present (1 the Task Question (TQ) and 2 the metacognitive question about how we should answer the TQ) the intervention group were really good at sticking to the metacognitive question when instructed. The control group would flit between the two. For the control group there was also a strange osmosis of the questions: they seemed to be trying to answer them both at the same time, e.g. "We can answer the question by believing what we want to believe." This suggested to me that the control group weren't as good, in this instance, at moving to a second-order position.

Facilitator 2

I thought something similar. It just seemed that the intervention group was more fluent in abstracting from the question than the control group as a result of the metacognition work we’ve done. I also found that at the start of the session with the question ‘are there any thinking tools that can help us answer the Q?’ the intervention group - like the control group - were more keen to get on to the main TQ but once we were in the thick of the discussion they came up with better suggestions as to how to answer the questions and what tools to use. Perhaps the urgency of the tools’ relevance became more apparent at that point.

Facilitator 3 said, after transcribing sections of dialogue half-way through the programme:

I don’t think I’ve ever, in all my 16 years of doing this, seen such high-level examples of strategizing how they should answer the question. Not only making use of what I’d shown them, but also creating new strategies that had not been taught. It is very difficult not to conclude from this that these unusual contributions are as a result of the intervention we are implementing with this class. What is of particular note is that they are explicitly stating that they are strategizing (e.g. ‘What we need to do to help answer the question…’); in normal sessions a) it is unusual to strategise in general (rather than merely respond to the question with ‘I think…’ responses) and b) it is even more unusual to state that that is what they are doing. I would say that this is clear evidence of metacognitive awareness - the sort of thing we are hoping to develop.

teachers

We asked teachers in the intervention classes to follow a couple of the children over the course of the term (they choose these children and did not tell the philosophers who they were). The responses were positive with teachers talking about children who don't normally contribute in class growing in confidence. They also noted the classes’ ability to use the CT and MC skills they had been taught: "children of all abilities were able to access and use the thinking tools that were on display."

In philosophy, I have noticed a gradual improvement in the quality of the children’s responses, from ‘every opinion is valid’ to a more comparative approach, relating responses to others’ and grouping responses. They are also using the sentence-stems they have explicitly been taught.

This teacher’s comment is especially interesting because the facilitators all noted that there seemed to be a shift towards a more evaluative mode within the intervention groups and away from the usual subjective mode that philosophy ‘has no right and wrong answers’ or ‘is just a matter of opinion’. A positive consequence of this is that it affords more rational force to critical comments, allowing for more progress to be made; but another consequence is that it seemed to result in some pupils becoming more reticent, possibly because they perceived a greater risk in sharing their opinions when they also perceived that opinions could be more robustly critiqued.

And by another teacher:

In the final session, child 1 commented that someone's use of a tool ('it depends what you mean by believe') was ineffective and got in the way because 'everyone already agrees what 'believe' means'. [They] even explicitly complained that the thinking tools can get in the way of philosophising. [They] and other children had thus started to consider when not to use the thinking tools. Child 2 interestingly commented that the class had internalised the thinking tools in any case.

In another class a teacher noted:

Child 1 was able to compare using thinking tools in philosophy to toolkits in literacy and ground rules in PSHE; [they] remembered 'distinction' as one of the thinking tools at the start of a session; said thinking tools can be helpful sometimes - if it's an easy question, you don't need them; if it's a harder question, they can be useful.

This teacher's feedback shows the children's abilities to metacognise about the CT skills they are using, their effectiveness and appropriateness.

feedback from children

The children were asked the following questions in the intervention group (100 pupils):

Q1: Did you enjoy philosophy this term? Q2: Did you use the thinking tools in philosophy? Q3: Have you used the tools in any other situations? Q4: Do you feel confident using the thinking tools? Q5: Do you think thinking tools should be taught to other children?

The chart below shows the binary answers of Yes and No, a minority of children responded with 'yes and no', 'maybe' or 'don't know', to some of these questions.

A large proportion of children reported that they enjoyed the classes (82%) and have gained from being taught CT skills. With 93% of respondents thinking that CT skills (thinking tools) should be taught to other children (Question 5). Three did not fill in this question, one said 'maybe', one said 'depends' and two said 'I don't know'.

Feedback from the control groups (106 children) also shows a high percentage of children enjoying the philosophy sessions (81%), but with a smaller percentage thinking it should be done with others (52%) and the same for those who found philosophy useful in other classes (52%) - compared to the intervention group, where nearly 60% thought they had used the thinking tools in other classes.

The percentage of children who said that they enjoy the philosophy classes mirror previous research carried out into The Philosophy Foundation's work by the Institute of Education, where 86% of boys and 81% of girls said they enjoyed the PhiE sessions.

Qualitative feedback from the facilitators suggested that the intervention classes seemed less engaged, particularly with some of the CT teaching components, despite this the data above shows a high value for the children of learning CT skills - even if they didn’t enjoy the sessions or the CT teaching component, they still thought, unilaterally, that CT skills should be taught to others. This is worth highlighting. As is the similarity of results from children across both control and intervention saying they enjoyed the sessions: over 80% in both cases. This suggests that the intervention can be implemented without hindering the enjoyment of the philosophy sessions overall from the children's perspective, and that the intervention itself was seen as valuable by the children.

6. did the intervention impact on individual learning, e.g. through reading results?

It was not possible to measure individual progress in philosophical aptitude through the data gathered in this study therefore we choose to consider individual progress through a usually quantified school subject. This would also show a transfer effect, a highly desirable feature of any intervention. Previous results have shown that a term of philosophical enquiry with The Philosophy Foundation improves the reading scores of children on free school meals (Institute of Education 2012, unpublished). So, it would be interesting to see if there was any difference: positive or negative, from this intervention. Any results seen here would be suggestive rather than strict evidence.

A baseline score was used from all the classes at the end of the Summer Term when they were in Y5, and had not yet had the intervention, and then compared to results after the intervention, at the end of the Autumn Term or beginning of the Spring Term. Statistical analysis of the reading tests was performed by Dr Georgina Donati (Centre for Brain and Cognitive Development, Birkbeck).

There were no statistically significant effects of the intervention in terms of the whole sample although there was a trend for a main effect of group (control vs intervention). There was a significant effect of school which was explored further. In comparing time 1 with time 2, School A increased their results most over this time - but with little difference between the intervention and control group, they both improved at a similar rate. However, for School B, there was no significant increase over time for the control group but there was for the intervention and the same but slightly smaller effect for School C, although it should be noted that there was still no significant difference between control and intervention in either school.

This could suggest that certain schools are likely to benefit more than others, for example, those with a higher proportion of children on free school meals (School B and School C have a larger percentage of children on free school meals than School A) - and perhaps varying the classes or running the intervention for a longer period of time might show significant results in other areas of learning.

conclusion and recommendations

The overall aim of this project was to consider whether, by focusing on the explicit teaching of CT and MC skills within an enquiry, it would improve the children's abilities to think metacognitively and successfully use CT skills, compared to a standard PhiE session. The quantitative and qualitative data suggest that this development does enhance children's use and successful use of CT and MC skills, above normal PhiE sessions. These findings also support Fridland's theoretical position that a key component of skill-learning involves attention and effort towards improving not only the ultimate ends of an activity but the technique by which that activity is performed. As metacognition increased in both the control and intervention group it is possible to say that PhiE helps children develop metacognitive skills, although a larger scale study would need to confirm this finding.

There were limitations to this study: it was a small sample size and the data is descriptive rather than coming from a recognised CT test with pre-test and post-test measurements. However, the data has been measured during an enquiry, which is valuable if considering the dispositional development of CT and the use of CT skills within an enquiry itself. Further studies could be carried out to compare individual development, using recognised CT skill tests in the future, as well as having a control group which receives no philosophy intervention at all. A longer scale study (e.g. 1 year rather than 1 term) would also be of interest, as other research into this area ranges from 1 to 2 years minimum.

Averages and group-measured data were used rather than individual pupil development, and it would be interesting to go back over the film footage to look at individual pupil's development over time. It could be that the intervention is suiting some pupils more than others. Firstly, mobilizing those with exceptional skills within the group should be regarded as a good, as different pupils with different skills can enable the group as a whole to progress; this, after all, is the aim of a community of enquiry. And secondly, pupils do learn from one another, so even if it is the same pupil or a minority of pupils furthering the CT/MC skills each week in the early stages, they will be modelling their use and successful use for their peers in a way that can emphasize and elucidate techniques for the future incorporation and skill development of their peers. This kind of peer-to-peer modelling is consistent with the TPF approach.

Facilitator 3 noted that though the intervention seemed to be impressive,

there may have been a slight downturn with regard to enthusiasm in the sessions for my intervention group. This may have something to do with the fact that we rarely have time for games, but it may also be due to the interventions slowing down the flow. However, it could also be because the sessions, being more evaluative, were more difficult, more demanding and required more from them intellectually. If the sessions are more evaluative and less a ‘mere conversation’ (^{Gardner 1995}), then this is what we might expect and, possibly, even hope for.

Teaching the skills over a longer period of time might make this 'slowing down' effect less prominent and maintain the right balance between evaluative and exploratory sessions.

Although the study had limitations, the results were strong enough for TPF to use the study to inform its pedagogy, incorporating the Menonic aspect into teaching across all age groups. TPF Facilitators are also finding ways of teaching CT and MC skills with younger and older classes within PhiE sessions.

For many years TPF had been developing the teaching of CT skills, but the missing link was the relationship between CT skills and metacognition. This study has transformed our own practice in that we now have a systematic procedure for implementing metacognitive questions that don't simply collapse back into the first-order question or consolidate subjectivist assumptions within the group (see page 4). This is done by combining the metacognitive structure with the explicit teaching of CT skills, so that when the pupils are asked the metacognitive task question (meta-TQ¹⁵) at the beginning, middle or end of the enquiry they have a repertoire of substantive, evaluative tools to apply to their metacognitive considerations (reflections, analyses and strategy) such as, among others, Counterexamples, Distinction-Drawing, Inference-making/Inference-challenging and Conceptual meaning.

Qualitatively, a clear difference can be seen in the responses from the intervention group in the last session when compared to their own performance in the first control session and that of the control group also at the end. So, rather than simply saying the usual, expected responses (see page 4) they said, in response to the meta-TQ, things like, '[In order to answer the question¹⁶] we need to say what we mean by choice and believe', drawing upon the repertoire of thinking skills (in this case, 'IDWYMB' or 'It depends what we mean by…' labelled as 'conceptual meaning'). But, further than this, they also explicitly challenged the limitation of the tools on the 'thinking wall' and started to suggest new tools not already there (e.g. 'using examples'). This showed a grasp of the remit set by the facilitator¹⁷, but also a creative, untutored will towards breaking out of that remit to expand it.

The Facione report concludes in its description of critical thinking:

Thus, educating good critical thinkers means working towards this ideal. It combines developing CT skills with nurturing those dispositions which consistently yield useful insights and which are the basis of a rational and democratic society.

This study suggests that a combination of enquiry, which nurtures critical thinking dispositions, and the explicit teaching of CT skills could help to develop critical and metacognitive thinking in young people and fulfil one of the stated aims of doing philosophy in schools, namely the development of critical thinkers.