Entrepreneurial Spirit Programme - Evaluation Report - Executive Summary

Aims and objectives of the evaluation

The main aim of the evaluation was to provide an independent external evaluation that focused primarily on key outcomes of the pilot ESP. The objectives of the evaluation included:

• To assess whether the Programme achieved all its objectives.

• To assess the extent of any impact on pupils participating in the Spirit and Experience courses.

• To assess the extent of any wider impact from the ESP across the six schools.

• To identify the level of awareness across participants’ parents and explore their views on its impact and their level of engagement with the Programme.

• To assess whether the ESP engaged with the needs of all students regardless of gender and ethnicity.

• To determine whether the Programme was sustainable and transferable.

Evaluation methods

Given the aim and objectives of the evaluation, a range of quantitative and qualitative methods were seen as appropriate. The numbers of pupils involved in the Micro Programme were relatively small and qualitative methods were seen as particularly useful to gather detailed information concerning their experiences and their views on the impact of the Programme. Focus groups with Micro group pupils were conducted midway through the Programme and six months after its completion. The Mindscreen Experience course was taught to a greater number of pupils in three of the six schools. Therefore, a survey supplemented by focus groups were used to obtain relevant data.

Given the need to gather detailed information and insights from teachers and other adult stakeholders, primarily the advisory/management group for the ESP, face-to-face, small group and telephone interviews were used. Key teachers involved with the ESP were interviewed midway through the Programme and six months after its completion. Parents of those involved in the initiative were likely to have varied levels of recollection of the Project, and the most appropriate method was, therefore, to conduct semi-structured telephone interviews with a sample of parents of Micro and Macro Course participants.

The structure and content of all research instruments were informed by the research questions pertinent to each stakeholder group and strand of the evaluation. Where appropriate, the evaluation also drew on relevant documentary information to provide contextual information and further assist the interpretation of data.

Main findings

The outcomes of the Entrepreneurial Spirit (Micro) Programme

Any attempt to assess whether the Micro component of the ESP achieved its main objectives depends on the interpretation of these objectives. If the main aim of the Micro Programme was, as senior school management had originally believed, to ‘recruit’ disaffected pupils and promote their engagement with school, improve their school attendance and school performance, then there is very little evidence of this. In general, teachers reported that there had been no notable impact on participants’ attendance or performance, but stressed that there had been few if any issues concerning attainment or attendance for most of the Micro participants. Pupils’ views concurred with teachers’ perspectives on this topic.

However, if, as Mindscreen stated, the main aims of the Micro Programme were to develop pupils’ entrepreneurial skills as well as to promote their selfconfidence, develop their broader life skills and raise aspirations, then the evaluation findings indicate a more positive outcome. There was consensus across pupils, teachers and parents concerning the positive impact of the Micro Programme in terms of affective and shorter-term outcomes for participants.

These included:

• Improvements in self-confidence and self-esteem

• Improvement in social skills

• Improvements in presentation and communication skills

• A greater willingness to be proactive

• Evidence of goal setting, planning and taking forward ideas in a constructive way

• Increased status among peers

• In one school a number of pupils identified as ‘difficult’ were making progress, including increased application and motivation in lessons.

Such positive outcomes must be placed in the context of a relatively high level of dropout from the Micro Programme. In addition, teachers and pupils added the caveat that many of the positive outcomes concerning personal development could also be due in part to pupils’ developing maturity. Nevertheless, for many Micro group participants the Programme had been inspirational and this was clear in their praise for the Programme and its facilitators.

Entrepreneurial outcomes of the Micro Programme

By the end of the ESP the actual business set up across the Micro groups was limited. This appeared to be due to a range of factors but the fact that the Micro Programme finished at a time when students had to prepare for their prelims was particularly important. In the main, it was small ventures involving one or two pupils that were successfully launched. These included car washing and gardening services, required few resources or outlay and could start and stop again when other factors impinged. More complex ventures required further support and/or investment finance to develop and were, therefore, more difficult to launch and/or sustain.

At the end of the ESP there was a sense of frustration and disappointment among many of the Micro groups because they had developed plans for their ventures but there was not sufficient time to seek investment because of their exam commitments. There was consensus across the groups that further support and advice for those who wished to pursue their venture after their exams would have been welcome.

One of the more ambitious and complex ventures, an indoor skateboard and youth shopping centre, was at the time of writing this report, continuing to develop its business plan and seek investment. This venture had attracted the support of a number of entrepreneurial advisors who were actively supporting the young people.

Despite the varied success of launching their ventures, all of the Micro group pupils believed that their entrepreneurial skills had been enhanced and they were generally confident that they would be able to deploy such skills in the future.

The outcomes of the Mindscreen Experience (Macro) course

In contrast to the different understandings about the aims of the Micro Programme, there was consensus across the various stakeholders concerning the main aims of the Mindscreen Experience (Macro) course. The course was seen as using personal profile information to gear PSD content to individual’s needs and thus promote self-confidence, awareness of personal strengths and areas for development, aspirations and other generic life skills. Given these aims, teachers generally believed that the Mindscreen Experience course had had a positive impact on pupils, in particular it was seen as having:

• Promoted pupils’ awareness of their strengths and helped them consider possible options for the future

• Promoted pupils’ self-confidence

• Promoted their ability to reflect and make choices based on their original Mindscreen profiles

• Promoted quieter participation in lessons and the life of the school.

During focus groups with Macro group pupils, their accounts of outcomes generally agreed with those of teachers. However, when surveyed at a later date, pupils’ views concerning the delivery and outcomes were equivocal. It is possible that this could be a feature of limited recall given the time elapsed since their course.

Did the ESP address the needs of all students regardless of gender and ethnicity?

In all but one of the six schools, there were relatively few minority ethnic groups but in that school the Micro group participants included two male pupils and one female pupil with ethnic minority origins. Their comments and reported experiences were no different to the others in the Micro group.

Across the Micro groups there were proportionately fewer female pupils. However, males were more likely than females to drop out of the Programme. The relatively small proportion of female pupils raises issues concerning the image of the Programme among candidates, its attractiveness for young women and the selection process. The findings from the survey of Mindscreen Experience course participants indicated that females were more likely than males to attribute positive outcomes to the course, including improved selfconfidence.

Appropriateness of the ESP learning and teaching approaches

There was consensus across the Micro group pupils that the teaching approaches used by the Mindscreen facilitators were appropriate, interactive and stimulating. Some pupils believed that early sessions had been more enjoyable because they were varied and involved excursions, whereas this was lacking from the second year of the Programme. The reduced frequency of session in the second year was also criticised. By the end of their programme, the Micro group pupils believed that the course could have established ventures earlier in the first year. Micro group pupils would have appreciated the opportunity for more experiential learning.

In contrast to Micro group participants, the Macro group pupils gave varying accounts of the range and quality of the teaching approaches used to deliver their course. Importantly, those Macro group pupils who were more positive about the delivery of their course were those who had been taught by teachers trained by Mindscreen to deliver the course. In the other two schools not all those teachers who had been trained were able to go on and deliver the course.

Overall, the quality of the profile documents and Mindscreen Experience course materials were praised, but teachers and pupils generally believed that the volume of the resources could be reduced and terminology simplified to better suit all pupils’ abilities.

What wider impact has the ESP had in the pilot schools?

Teachers believed that there was little evidence of any notable impact on the wider learning and teaching environment in their schools. The absence of integration with the Micro Programme and the wider curriculum of the schools meant there was no opportunity for this programme to influence teachers’ practice or views on approaches. This also limited teachers’ ability to articulate the Micro Programme with the wider curriculum and other measures that could reinforce participants’ positive outcomes.

In one school, however, the Depute Headteacher reported that the positive impact of the Micro Programme had helped to convince some initially sceptical staff about the value of the Programmes and innovative approaches generally.

The Mindscreen Experience course was a part of the PSE curriculum for about nine weeks in three of the schools. Some of the teachers involved reported that the training by Mindscreen had provided insights on using profile information to help ‘fine-tune’ PSE content, but that the course itself did not have any major impact on their teaching styles or classroom organisation.

Parents' awareness of the ESP and involvement

The information collected from a sample of parents of Micro and Macro group pupils revealed that parents in both groups varied in their views concerning the value of the Micro and Macro programmes. Most of the Micro group parents who were interviewed could highlight positive outcomes such as increased confidence, communication skills and aspirations. However, in a small number of cases such outcomes could also have less positive implications. For example, where pupils became so motivated in their Micro Programme that their commitment to their wider education was temporarily reduced.

The nature and volume of Micro group parents’ views reflect a greater involvement and interest with the course. In contrast, views of parents of Macro group pupils revealed a general lack of involvement and awareness. This could reflect the more intensive and longer duration of the Micro Programme.

Is the ESP sustainable and transferable?

The majority of professional stakeholders believed that in the form deployed in the pilot programme, the ESP, particularly the Micro Programme, was severely limited in its sustainability. The ability of teachers to buy into the ESP in the longer-term was seen as being influenced by a range of other priorities facing schools’ curricula and resources.

All of the senior teachers who were responsible for the ESP in their school believed that, in principle, the entrepreneurial component (Micro Programme) could provide a valuable way to engage with pupils for whom school had limited appeal or relevance. A unique feature and strength of the Programme was seen to be its delivery by experienced entrepreneurs. However, most teachers realised that, because of the limited availability of such personnel it was unlikely that the Micro Programme could be expanded.

Most professional stakeholders believed that if the Mindscreen Experience component of the Programme were to be delivered within the PSE curriculum then it would have to vie for space with other content prioritised by national and local policies. This could mean that future Mindscreen Experience courses would be either episodic or, ideally become successfully integrated across the curriculum.

Towards the end of the ESP pilot, the Mindscreen creators of the ESP appear to have reflected on the emerging findings of the internal and external evaluations and informal feedback from teachers and their advisors. This resulted in a revised programme which has already been adopted by a number of Scottish schools across different local authorities. Across the original six pilot schools, two schools in the east of Scotland have shown interest in adopting this new programme. Senior teachers in two of the remaining four schools reported that they might be interested at some future date in adopting the revised programme.

Conclusion and issues for consideration

The evaluation findings, therefore, suggest that aspects of the ESP have had, at least, some initial impact on the participating young people concerning key life skills (goal-setting, raised aspirations, increased communication and team working skills) and preparing them to deal with challenges in life. This fits well with the vision of government policy in recent years concerning enterprising schools and the Determined to Succeed agenda. However, some issues for consideration remain:

• It must be reiterated that the goal of promoting the incidence of successful entrepreneurship cannot be adequately evaluated in the short-term. It might be that those leaving the Micro Programme without starting a business will go on to be entrepreneurially active in the future. The design and funding of evaluations of entrepreneurial programmes, therefore, needs to address the issue that entrepreneurial activity outcomes from such programmes are likely to be more evident in the longer-term rather than immediately following a programme.

• If one views the ESP as the pilot it was, then it has been productive. It has: illuminated many issues about the feasibility of programme components; prompted fine-tuning of the profiling instruments to better suit pupils’ abilities; refined the nature and volume of the PSD component (Mindscreen Experience course). The programme has allowed Mindscreen to develop a revised and possibly more appropriate and sustainable programme. However, the suitability and impact of the latter programme will only be determined with suitable monitoring and evaluation. There is also scope for Mindscreen to consult with other organisations such as Learning and Teaching Scotland given the aims of the revised Mindscreen Experience and its place within the curriculum.

• The notable level of dropout from the Micro Programme suggests that schools and Mindscreen should systematically assess reasons for drop-out to ascertain whether this is indicative of problems with programme delivery, the original selection procedures or pupils’ personal issues that require support.

• Many of the factors limiting the ability of Micro group pupils to launch their ventures are outwith the control of Mindscreen. However, given the raised expectations and personal investment of the Micro group participants, greater thought should have been given as to how their ventures and plans could have been articulated with other enterprise initiatives or organisations and programmes or supported via liaison with local enterprise officers, local entrepreneurs and enterprise organisations such as Young Enterprise Scotland, the Prince’s Scottish Youth Business Trust, or Careers Scotland.

• The difference of opinion between Mindscreen and senior school management concerning the original aims of the Programme strongly suggest that Mindscreen and other key stakeholders should develop explicit statements concerning programme aims, objectives and agreed criteria to assess the desired outcomes, at the outset, to avoid later misunderstandings about the purposes of programmes.

• The implementation of the ESP has illuminated the attitudinal and cultural differences that can exist, at least initially, between some teachers and entrepreneurs concerning the relevance of courses that focus on developing explicit entrepreneurial skills. Such findings stress the need for entrepreneurial and enterprise programmes to be carefully introduced with appropriate consultation between partners and awareness of the prevailing school ethos. The pilot has also highlighted that private organisations such as Mindscreen when working with schools often take time to become aware of the needs of schools and the factors affecting teachers’ abilities to implement innovative programmes.

• Some teachers argued that the observed outcomes could have been achieved with less input, more cost effectively and with greater articulation with the wider curriculum.

• Pupils and teachers strongly believe that entrepreneurs are the most appropriate people to provide programmes that aim to promote explicitly entrepreneurial outcomes. Further research is required to assess whether teachers can be trained and supported to effectively deliver entrepreneurial programmes, or whether teachers with certain characteristics or entrepreneurs are indeed better able to provide these lessons.

• Programmes that focus on specific groups and provide sessions outside of the school premises need to ensure that communication takes place with teachers about the progress of the group, and the approaches used, and to consider possible strategies teachers can implement that might reinforce any positive outcomes from the Programme. When such programmes also involve excursions and residential events they require stringent procedures to ensure pupils’ health and safety and suitable standards of behaviour.

PDF File of Entrepreneurial Spirit Programme Final Evaluation Report (2.8 MB)

• Email this page
• Print friendly format
• Digg this
• Save to del.icio.us