Smithycroft Learning Community:Critical Skills Project - Evaluation Report
Introduction
This report is an evaluation of aspects of the Critical Skills Project (CSP) in the Smithycroft Learning Community, City of Glasgow Council. The evaluation was undertaken by members of the Faculty of Education in the University of Glasgow as part of the national evaluation of projects supported through the Scottish Executive Education Department's Future Learning and Teaching programme (FLaT). The evaluation was undertaken between April 2003 and October 2004.
The Critical Skills Programme (CSP) has been acclaimed as a form of teaching and learning that will better equip young people for the challenges of the future. Before this can be validly accepted, however, it needs to be established whether such pedagogic methods can effectively be introduced into existing procedures and, perhaps more importantly, whether the critical skills pedagogy impacts on young learners in a meaningful way that better equips them with the tools they will need to think and act both independently and with others when faced with a new learning challenge.
CSP has its origins in the US and is now located and extensively used in Antioch University in New Hampshire. It is a particular model of co-operative learning – an approach to teaching used widely in USA and Canada. It involves learners in an active, collaborative and student- centred learning process.
In 2001 CSP was introduced into a cluster of schools in the Smithycroft Learning Community in the City of Glasgow. Formal training sessions were organised for teachers in a number of schools and pre-five establishments within the Community.
Aims of the evaluation
The evaluation was undertaken in two phases:
Phase 1 focused on the understanding, adoption and implementation of CSP teaching and learning methodologies in the schools and pre-five establishments in the Smithycroft Learning Community in Glasgow. More specifically the research questions were:
- how are staff new to CSP inducted into the CSP model and supported in its application?
- can staff identify changes to their pedagogic practice?
- what are staff's perceptions of the model's value, including its limitations?
- how do staff implementing CSP collaborate both within and between schools and across sectors?
- what support is required for changes in pedagogic practice to be sustained?
- how does the LC leadership promote and support CSP?
- are parents aware of any changes in their child's approach to learning?
Phase 2 used a quasi-experimental design to explore the impact of CSP on two age groups of learners in one primary school in the Smithycroft Learning Community. More specifically, the research questions were:
- does involvement in the CSP learning experience impact on children's strategies when faced with a new learning situation?
- can differences be identified in two learning skills (problem solving and communication) and two dispositions (collaboration and community) between those children who have experienced the CSP model for a period of time when compared to similar groups of children who have had no or very little CSP experience?
- does the deployment of CSP skills and dispositions change over time? If so, are the changes sustained?
Evaluation Methods
For Phase 1, a qualitative methodology was used based on interviews with key stakeholders involved in delivering CSP at each stage of the schooling process – secondary, primary and pre-five establishments. The evaluation focused on participants' responses to the training sessions and their adoption of CSP good practice. Of the ten schools and six pre-five establishments in the Smithycroft Learning Community, nine schools and all pre-five establishments had sent representatives to the training sessions. 34 staff from a total staff complement of 192 in Smithycroft Learning Community had participated by the summer of 2003. It was decided to interview all pre-five and primary school staff and a cross-section of secondary school staff. In addition, seven heads of establishment who had not been trained were interviewed.
In Phase 2 a quasi-experimental design was used to explore the impact of CSP on children's learning strategies. In order to make the experiment manageable, two CSP skills (Problem Solving and Communication) and two CSP dispositions (Collaboration and Community)based on two curriculum areas in the 5-14 National Curriculum guidelines were selected as the basis for collecting the data.
Two primary schools were selected in the Smithycroft Learning Community to participate in phase 2. One school (School A, designated the 'experimental' school) had considerable experience in the implementation of CSP whilst the other (School B, designated the 'control' school) was still to engage with CSP. Within each school two age groups were selected in consultation with the headteacher of School A as having most exposure to CSP, thus allowing for the maximum opportunity to display CSP characteristics when faced with a new learning challenge. These were P3 and P7. The curriculum areas selected from the National 5-14 Guidelines were Environmental Studies and Personal and Social Development.
For the experiment, three new challenges were devised for both P3 and P7 classes in both schools.
For each of the three sessions each of the P3 and P7 classes in Schools A and B were allocated to one of four small groups for the purpose of tackling the set challenges. As far as possible, these small groups were identical for each of the three sessions though, due to absences, some variation occurred. Allocation to the groups was made by the respective class teachers, the only guidance being given was that the children should not have displayed any anti-social behaviour to each other in their respective classes. Each groups' engagement with the challenge was video- and audio-recorded.
The Learning Challenges
The challenges were based on the idea of a blind person having to be guided round a village by a guide dog. This involved map reading and direction skills of both P3 and P7, as well as their interpersonal skills.
The knowledge criteria selected for Challenge 1 were related to the 5-14 Environmental Studies topic, Understanding People and Place, in particular the ability to use left and right and reference to landmarks to give directions. For Challenge 2 the knowledge criteria were related to Health Education, Healthy and Safe Living, in particular, potential risks in the environment. Challenge 3 focused on 5-14 Personal and Social Development, in particular demonstrating empathy, respect and tolerance for others.
For all challenges, the skills related to problem solving and communication, in particular, an ability to evaluate and test ideas, observe the results and respond accordingly and to speak with clarity, using an appropriate tone.
For all challenges, the dispositions related to collaboration and community, in particular, to value and engage fully in collaboration, working to optimise the outcomes of a common effort, and to enter into productive groupwork – helping others to achieve a common goal and taking responsibility for a share of the work.
Using a 3D model village, the children had to devise a set of directions to guide the blind person round the village as she took charge of her new guide dog ( Challenge 1) and identify the dangers she would encounter on her trip ( Challenge 2) . Each group then devised a role play (Challenge 3) where the blind person would explore her feelings about her initial journeys with the guide dog. A simulated village obstacle course and a real guide dog were used as a stimulus for this final challenge.
The Sample of Pupils
The pupil sample comprised two 'experimental' groups and two 'control' groups. The experimental groups consisted of 35 children in total in two age-groups, drawn from one primary school . The first age-group consisted of 20 7/8 year old children drawn from P3; the second group, aged 11, consisted of 15 P7 children.
Similarly, the two 'control' groups comprised 36 children in total, drawn from the same two age-groups from another primary school in the same Learning Community as the experimental school. One age-group consisted of 20 P3 children; the second of 16 P7 children.
Findings and Conclusions
In terms of the first perspective, that is, the training and implementation of CSP pedagogy, this evaluation is reasonably optimistic. Professionals from a range of backgrounds from prefive to secondary school teachers responded positively to the principles and practices of CSP. They enthusiastically endorsed the CSP philosophy and were receptive to its practical implications. They willingly engaged in the appropriate training and reported applying the principles of CSP to their practice.
In terms of the second perspective, that is, the impact of CSP on children's learning, the evaluation provides tentative evidence in support of CSP. Using an experimental methodology, video-recordings of children engaging with a new learning challenge were made on three successive occasions over a period of four months. The extent to which children engaged with the new learning challenges and their deployment of selected skills and dispositions in the challenge were subsequently rated by the evaluators on a minute by minute basis. Although no overwhelming evidence emerged to endorse CSP in terms of its impact on children, tentative indications emerged showing that some children taught by teachers trained in CSP might have benefited in terms of their willingness to engage in a new learning challenge and deploy skills and learning dispositions to that challenge. Such benefits were in evidence in terms of younger, less academically able and more socially competent children:
- Improvements in the deployment of both skills and dispositions were most notable in the early stages of the primary school. However, this change cannot be entirely attributed to the CSP as the improvement was evident in both groups of children. With pupils in the upper stage of primary school there was no observable improvement over time and some evidence of regression in the control group.
- Younger children taught by teachers trained in CSP showed a tendency to deploy appropriate problem-solving skills in selected areas of the curriculum to a greater extent than children taught using more traditional methods.
- Improvements in skills are most notable amongst children at the early stages of development in terms of the 5-14 levels of achievement. More advanced children do not appear to improve to the same extent. Improvements in dispositions are observable in children at all stages of development.
- Improvements in skills are observable at a significant level for both children rated as possessing a high level of social skills, and for those from less prosperous socio-economic backgrounds as defined in terms of free school meals entitlement. However, there is no significant evidence of improvement in dispositions in either group over time.
- Girls are better equipped than boys to deal with new learning challenges though both boys and girls improve over time.
The evaluation concludes that it would be premature to provide a blanket endorsement of CSP. However, it seems that CSP has the potential to engage younger learners in new learning challenges to a greater extent than more traditional methods. Until there are more sustained and intensive methods of training teachers in the practice of CSP, such that their teaching is fundamentally different, the future of CSP in Scottish schools must remain uncertain.
In terms of taking forward this training, the report underlines the importance of teachers' engagement with and the need for on-going support to ensure sustainability of the CSP philosophy. Initial engagement of practising teachers with CSP needs to be intense in order for CSP pedagogy to become firmly embedded in day-to-day classroom practice. In addition, continuing support is necessary both at the school level in which CSP is being introduced and from the respective local authority. Successful implementation of CSP ideals will require a whole school approach in order to embed this fundamentally different culture into the learning and teaching process.
In the light of these findings, the report raises implications for the consolidation and expansion of CSP and argues that a more intensive model of 'change' to existing pedagogic practices be considered. Such a model involves the 'immersion' of a single institution in the innovation at any one time.
Acknowledgements
The evaluation team wishes to express its appreciation to all those - teachers, headteachers,pre-five staff and children – who were involved with the Critical Skills Programme (CSP)in Smithycroft Learning Community, Glasgow and who willingly provided to the evaluators their perceptions of, and reactions to CSP. Without such information the evaluation would not have been possible. We also wish to express our gratitude to Mr. David Cumming,Principal of Smithycroft Learning Community and Ms. Lesley Dunlop, former headteacher of Carntyne Primary School, for their support and co-operation. Appreciation is specially due to the parents who unreservedly allowed their children to be video-recorded in the experimental phase of the evaluation.
Thanks are also due to the headteachers and staff of the two schools involved in obtaining the video-recordings of children in their schools. Such facilitation and co-operation is greatly appreciated and has been a hallmark of our liaison with all the schools and pre-five centres in the Smithycroft Learning Community.
The team also wishes to thank the staff of the Future Learning and Teaching (FLaT) Unit in the Scottish Executive Education Department who approved the evaluation and provided the finance. Similarly, thanks are due to members of the evaluation Advisory Group and its convenor, Dr. Colin Holroyd. Meetings of the Group were always lively and constructive.
The team has a special word of thanks to the staff of the Media Services Department in the University of Glasgow who undertook the arduous task of making the video material and editing it in a form suitable for analysis. We also wish to acknowledge financial support from the Faculty of Education in the University of Glasgow which greatly facilitated the data collection process. Our thanks also go to Dr. Jennifer Gourlay who undertook the tedious task of setting up the SPSS datafile.
Last but not least, our gratitude goes to the two secretaries – Anne Currie, who painstakingly prepared this report, and Fay Barclay, who provided support in the early stages of the evaluation.
The report can be downloaded in PDF format by clicking on the link below:
Smithycroft Learning Community:Critical Skills Project - Evaluation Report - Full Report