Dynamic Futures - Evaluation Report - Executive Summary

Background

In February 2004, the Scottish Executive Education Department commissioned an external evaluation of the Dynamic Futures Project (DFP) through the Future Learning and Teaching Programme (FLaT) under the leadership of Professor J. E. Wilkinson in the Faculty of Education, University of Glasgow with Dr G. Head as co-investigator.

The Project is located in the cluster of three primary schools and one secondary school in the Vale of Leven in West Dunbartonshire. The evaluation was undertaken between March 2004 and December 2006. During this time, qualitative data was collected through interviews with DFP staff, teachers and headteachers in the four schools involved. Data was also collected from a fourth primary school not involved in the Project in order to provide a comparison. In addition, focus groups were carried out with small groups of pupils in the schools in which DFP operated during this period. Finally, for the children involved in the Project and those in the comparison group, quantitative data relating to attainment, attendance and conduct was also collected. This is the Final Report of the evaluation.

Aims of the Dynamic Futures Project

The key aims of the Dynamic Futures Project are as follows:

To break down traditional barriers between pupils, teachers and parents
For pupils to demonstrate:

· calmer approaches to the school situation

· better relationships with other children or adults

· better concentration levels

· more mature reactions in class to areas of challenge or difficulty

· ability to understand own self and reactions to situations

· higher levels of motivation

· more developed sense of humour

Teachers and parents to also demonstrate some, if not all, of the above attributes

The project was carried out over a three year period as set out below.
Year 1: March 2003 – February 2004
Ten pupils were selected from P7 in Haldane Primary School for participation in the project. Subsequently, two pupils withdrew.

Year 2: March 2004 – February 2005

Ten pupils were also selected from P7 in Highdykes Primary School whilst offering ongoing advice and support to the first group who were in S1 progressing to S2. One pupil subsequently withdrew.

Year 3: March 2005 – February 2006

Ten pupils were selected from P7 in Renton Primary School whilst offering ongoing advice and support to the first two groups who had moved to the secondary school. Two pupils subsequently withdrew.

In each of the three years, the pupils selected were those who were perceived to have problems with some aspect of emotional literacy, for example, issues with impulse control, anger management, relationship difficulties, motivation or self-awareness. Up to three staff in each of the schools, as well as the parents of the children, were invited to participate in the project to explore issues in terms of the emotional development of the pupils and of their own emotional literacy. The project offered activities and personal development opportunities to the young people, staff and parents based on the four principles of the Dynamic Futures Programme: responsibility, commitment, collaboration and self-evaluation.

Aims of the evaluation

The aims of the evaluation were as follows:

To assess the overall impact of the Dynamic Futures Project on the teachers, pupils and parents in the three primary schools involved and the associated secondary;
To identify what, if any, improvements it has made to pupil attitudes, participation, attainment, attendance;
To identify what, if any, impact it has made on teacher/pupil relationships, school ethos and teaching and learning environment;
To explore the impact of the involvement of the pupils on their relationship with their parents, the attitudes of parents towards the school, their child and their child’s education.

Research questions

Overall impact

What assumptions underpin the design of Dynamic Futures Project? Do staff in schools share such assumptions?
Have primary school teachers (principally headteachers, P6 and P7 teachers) detected an improvement in attitudes and behaviour in those pupils who participated in the project?
How did teachers respond to the input from Dynamic Futures staff?
How did participating pupils respond to the input from Dynamic Futures staff?
Were parents of participating pupils aware of their child’s involvement in the project and the reasons for such involvement?
Have parents of participating pupils detected any improvement in their child’s attitudes and behaviour?
Have secondary school teachers with responsibility for S1 and S2 detected any particular behavioural problems in the pupils who participated in DFP? If so, what response has been forthcoming?

Pupil behaviour and attitude

To what extent have participating pupils improved their commitment in response to participation in the DFP?
To what extent have participating pupils improved their collaboration in response to participation in the DFP?
To what extent have participating pupils improved their self-esteem in response to participation in the DFP?
To what extent have participating pupils improved their self-evaluation in response to participation in the DFP?
To what extent have any such changed attitudes and behaviour impacted on attendance and attainment in national tests?

Schools

To what extent has the DFP been perceived by teachers to have impacted on relationships, ethos and pedagogy?
To what extent are such matters rated by external experts (HMIE, Advisers, ITE/CPD staff)?
How is the DFP work managed by the headteacher in each primary school?
What communication mechanisms are in place as a consequence of DFP participation?
Has teachers’ own self-esteem improved?
To what extent have school procedures/systems/arrangements been modified as a consequence of engaging with DFP?

Pupils and Parents

Do participating pupils display greater co-operation, communication and better behaviour (e.g. less aggression) with parents?
Are parents of participating pupils more inclined to ‘listen’ to their children?
Are parents able to manage their children more effectively?
Has there been an impact on the interaction between schools and parents?

Methodology

The research used both a longitudinal and cross-sectional design involving both qualitative and quantitative data. The longitudinal aspect focused on three successive cohorts of P7 pupils and their teachers engaged with the Dynamic Futures Project and a group of pupils not involved as a comparison group. The cross-sectional aspect was focused on the perceptions of secondary school teachers based in the Vale of Leven Academy, on the support provided by local authority staff, the input from DFP staff and the response of parents. In addition, the third cohort of primary school pupils involved in the DFP was monitored to assess the developmental aspect of the DFP input.

Findings

The findings can be summarised as follows:

DFP staff are a major asset of the project;
there is a clear commitment to the ideas that underpin DFP among all respondents;
there is support for the project among school staff;
there is strong support for the project from the Local Authority
the participating pupils are generally enthusiastic about DFP;
teachers and headteachers in primary schools indicated that they would find it difficult to develop and sustain the project without considerable staff development;
teachers and headteachers in primary schools highlighted both advantages and disadvantages of carrying out DFP with selected small groups;
teachers and headteachers in primary schools felt that if DFP were to operate at its optimum effectiveness within the primary, it would need to begin earlier.
the Vale of Leven Academy is providing a high level of support for a significant number of pupils participating in DFP;
a majority of the pupils participating in DFP are maintaining a level of attendance at the Vale of Leven Academy commensurate with average levels for the Authority;
levels of academic attainment among pupils participating in DFP are no different from the attainment of those pupils not experiencing the project;
the data on punishment exercises indicates that a number of DFP pupils appear to be experiencing difficulties at the Academy;
in terms of the project’s impact on the pupils involved, whilst the project’s activities were clearly welcomed by the pupils, there is little evidence to indicate that the project has had any deep or sustained impact.

Key issues

Three key issues emerged from the findings:

Sustainability and development

The project is highly dependent upon the personal and professional qualities of DFP staff. Whilst this is a major asset, it is also a significant factor when considering how to develop and sustain the project. If teachers were to run this project in the future, they would require a considerable degree of staff development. Additionally, teachers involved in delivering DFP would have to be familiar with the philosophical and psychological theories that form the basis of the project.

Furthermore, any decision to use DFP beyond the period of the research would have to take cognisance of the degree to which the Local Authority has supported the project.

Effectiveness of DFP within primary schools
Whilst there was clear support for the project among pupils, teachers and headteachers there was also a feeling of ‘too little, too late’. Teachers’ and headteachers’ comments concerning the earlier introduction of DFP indicate that they perceived the project as being worthwhile but there had not been the opportunity for them to engage with it in a more meaningful way in order to support the pupils involved and the other pupils in their current and future classes.

The operation of DFP in secondary school

Contributions from teachers in the secondary school reflected the concerns of their colleagues from primary schools. Like them, they assumed the project to be of some value but their lack of involvement rendered them unable to comment authoritatively on how DFP operated within the school.

Primary teachers felt that, having begun supporting children, the DFP involvement in the pupils’ experience of secondary school should reflect the level of commitment and enthusiasm they had been given at primary school. The evidence gathered from the quantitative data suggests that high levels of support were maintained at Vale of Leven Academy but it is not clear to what extent the DFP was influential in providing this support.

There appeared to be a tension between the qualitative responses of the primary teachers and pupils involved, and the experiences of the pupils at the Academy, as far as it can be assessed from the quantitative data. Whilst figures for attendance and attainment suggested that pupils’ and primary teachers’ expectations of difficulty and failure were not met, the pattern of punishment exercises indicated that life at school was still difficult for a significant number of the DFP pupils. In particular, the fact that DFP pupils appeared to do no better than a similar group who had not undertaken the project, has implications for how the initiative might operate in the future.

PDF File: Dynamic Futures - Final Evaluation Report (304 KB)