ICT in Science Evaluation Report

Executive Summary

Section 1: The Context of the School and Evaluation Projects

This evaluation project is concerned with the effects of the introduction of sets of laptop computers and associated data-projectors and interactive whiteboards into the three science departments of a large secondary school in Scotland, and focuses on three aspects of the school initiative: the impact on the teachers and teaching; the impact on learners and learning; and the implications for the future development of the school as a learning community. The school initiative is set within the national context of the Future Learning and Teaching Programme, the principal aim of which is to encourage schools, education authorities and other bodies with a stake in school education to create a learning and teaching environment for the future which is sensitive to individual needs, will promote attainment and which tackles the barriers to inclusive learning and teaching in the community.

The School Context

The school is a large non-denominational secondary school, (Roll 1100, teaching staff N= 85). Its catchment area includes a small town area with additional elements of rural and city areas; these do not include significant areas of social deprivation. The attainment statistics indicate that the school performs well with respect to the nationally available statistics. The science departments have together a total of 17 staff, some of whom are part time. (1.3)

The School's ICT in Science Project Proposals

The school initiative was introduced partly as a result of an HMIE report, in which it was suggested that more use should be made of ICT to enhance teaching and learning, and partly in response to the SEED report A Science Strategy for Scotland. The main aims of the school project were to enhance the learning, motivation and independence of pupils in both science and in ICT; and to develop and pilot the use of technology to enhance learning and teaching strategies which maximised the effective use of ICT. (1.4)

The aims of the Evaluation Project

The evaluation project which began in March 2003 and ended in July 2004, had four specific aims:

To evaluate the impact of the project on integrating ICT into teaching practice, in selected areas of the science curriculum, including any impact of more flexible deployment of ICT resources.
To evaluate the impact of the project on student competence and confidence in science and developing key related transferable skills and dispositions.
To evaluate the impact of the project on students' attitudes to science, and investigate any resulting impact on take up of science subjects at key stages in the curriculum.
To evaluate the relationship between the project and the school's CPD programme on teacher competence and confidence in using ICT in science and on developing the school learning community. (1.5)

The methodology

The methodology included: interviews with teachers, and with the initiators and developers of the school project, with senior management staff in the school and with pupils (1.6.1 and 1.6.2); classroom observations (1.6.3) and a staff questionnaire (1.6.4.).

Section 2: Introducing and Implementing the Changes

Although the project was initially conceived and planned by senior staff (the Project Team Leaders), they put considerable thought into the means by which they would ensure that all the science staff were included in the active implementation of the technology, and the degrees and kind of support needed so that all the science staff (N= 17) could be included. The planners focussed their attention on S1/2 and developed set exercises within the S1/2 curriculum (2.3.1 a).

The hardware initially comprised two class sets of laptops, 4 digital projectors, two interactive whiteboards and a number of accessories such as data loggers and sensors. Over the period of the project additional whiteboards, laptops and peripherals were added (2.3.1 b). Additional funding allocated by SEED in March 2003 was used to purchase 3 fixed interactive whiteboards and the school subsequently sourced funds for an additional 4 mounted boards, with projectors which were taken into use after Easter.

The Project Team Leaders comprising the three Principal Teachers and an AHT initiated departmentally-based staff development to introduce the other science staff to the units they had developed. The initial introduction comprised awareness raising of what the technology (e.g. the interactive whiteboards) could do; tuition on the working of the boards and the laptops; tuition to all staff involved on how to manage the set ICT-based materials with a class (2.3.2). The planning decisions carried through into implementation appear to have resulted in the regular and sustained use of the equipment by the whole staff group in science. This is a significant achievement, and in further sections of this report we examine in greater detail how this was brought about.

The staff development took the form of demonstrations and active modelling of teaching by more experienced members of staff; staff acting as mentors for colleagues; informal visits to each others' classrooms; the frequent and open discussion in the shared staff room of how things could be done and how problems might be resolved (2.4). A few staff had regular and long-standing contacts with colleagues in other schools and shared subject materials and information e.g. on useful websites with them.

The cross-curricular interdepartmental exchanges in the school had not been frequent, mainly comprising demonstrations of a department's activities during inservice days. Additional opportunities were provided at twilight CPD sessions on Active Learning. This almost certainly reflects the subject focussed interests and discourse of most secondary teachers when talking about ICT. What we would judge to be eminently transferable to other departments, however, is the quality of the planning and induction processes which the development team had set in place which had resulted in the full use of the technology in classrooms by all teachers, even if at this early stage the benefits were difficult to quantify in terms of learning (2.4.3).

The factors which were identified as supporting the development were:

The additional ICT provision of extra interactive whiteboards. This hugely increased the ease of use of the technology for many teachers. This was further enhanced by the purchase of additional interactive whiteboards, allowing easy access in every science room (2.5.1 a).
The provision of the set material for the S1/2 curriculum as a focus for common action and the associated programme of support from the promoted staff in the early stages. Although ICT was observed in use at all age stages within the school, the commitment of the promoted staff to anticipating and /or being responsive to the needs of their colleagues in introducing the S1/2 activities had clearly contributed significantly to the high level of use of the equipment observed (2.5.1 b and c).
The enthusiasm and skills shown by the pupils was a highly motivating experience for the staff, making them feel their efforts were worthwhile (2.5.1 d).
The staff room shared by the three science departments provided the forum for formal and informal meetings and exchanges which allowed information to be exchanged rapidly across quite a large staff group. The provision of set S1/2 activities gave a common curriculum context and focus for professional discussions (2.5.1 e).

The factors which were identified as barriers to the development were:

The scarcity and lack of immediacy of technological support (2.5.2 a).
The lack of time to review software and the unavailability of guidance on its uses (2.5.2 b).
The nature of the existing curriculum with which they had to comply, particularly the extensive content coverage dictated by external exams 2.5.2 c).

The basis of the decision taking by staff (2.6)

The reliability of the equipment and the convenience of using it.
Their confidence in and level of their ICT skills.
What they had seen working in another teacher's classroom, or what had been recommended as 'good' by a current or former colleague.
The degree of enthusiasm which was shown by the pupils.
Whether the programme or activity matched well to the topics in the given curriculum.

What was clearly discernible was the value of informal staff communication and mutual support in helping the staff of the science departments to undertake a significant step in the journey towards using ICT regularly in their classrooms and to begin thinking about changes in teaching and learning. What was not discernible was an articulated or shared model of learning which could give coherence to the developments and to the experiences of the pupils and which could form a framework for the ongoing monitoring and evaluation of their plans and their outcomes. We return to this in a later round of interviews discussed in Section 5.

Section 3: The Impact of the Technology in the Classrooms

How have the ICT resources been integrated into selected areas of the science curriculum and what has been the impact on the teaching practices? What changes have the technologies brought about in the pupils' learning activities?

The methods of data gathering included classroom observation and pupil and teacher interviews (3.2.2). A wide range of technology was observed in use, for example for presentational purposes, (e.g. PowerPoint); for data management (e.g. Excel); for simulations (e.g. ACTIVESTUDIO); and for tutorial purposes (e.g. SCHOLAR). The extent to which potential learning gains will actually be made using the technology in these ways will, of course, depend on other aspects of the learning context known for some time to be potential limiting factors in understanding science information e.g. the level of difficulty of the information, the compatibility of the information with what pupils already know, the account taken of pupil misconceptions etc. (3.3.1 a).

The teachers had a typical range of different teaching styles which were clearly well established before the installation of the technology, and which they continued to follow regardless of whether the technology was being used in the class. These styles were derived from personal characteristics and preferences and from past experiences and determined that the technology was used in different ways and to different degrees by individual teachers, even within the set pieces which had been prepared by the PTs (3.3.1 b).

How were the pupils and teachers acquiring the necessary ICT skills?

The model of learning which was applied to the pupil acquisition of ICT related skills was that of 'just in time' learning, integrated purposefully into the science activities. This model of learning was also that being applied by the teachers to their own learning as they themselves started to use the technology for their teaching. The pupils, like the teachers, rapidly learned from each other. Teachers expressed frustration and feelings of being professionally undermined when they could not get systems to work. This suggested that having individuals with adequate technical expertise to hand to provide support was an important and valued source of learning for those teachers with limited experience, particularly at the start of the project (3.3.2).

What specific knowledge and skills within science do teachers believe are promoted by their technology use? Are these closely determined by the curriculum e.g. 5-14 or Standard Grade? Are they supplemented by the teachers' additional aims?

The aims of the lessons remained fairly tightly linked to the aims of the existing courses. The most common learning activities of the pupils were directed towards finding, acquiring and presenting science knowledge. Generic skills: The generic skills identified by the teachers were research skills, independent learning and collaborative learning. Research skills include the use of a search engine, the ability to refine searches, recognise reliable web sources and the ability to scroll and select from information on a computer screen. These activities belong to the field of information literacy which is the area best known to and understood by librarians. The staff were becoming aware that more structure and guidance should be built into 'researching' activities to assist pupils (3.4.1 a).

Because of their growing confidence and levels of expertise with the technology, the pupils were seen to take ownership of the management of it in the classroom, and this was seen by teachers as a contribution to independent learning and the enhancement of enthusiasm for science. Independent learning is generally used in schools to mean that pupils are able to find specific information without the teacher having to present it. However, being a competent life-long learner also encapsulates the skill of knowing what kinds of questions are appropriate and important to raise in order to engage meaningfully with information on a subject. This tends not to be part of the model of independent learning applied within a school science curriculum based mainly on content acquisition (3.4.1 b).

Group work in which pupils were working in pairs on given tasks – usually searching for information and occasionally putting together a joint presentation were frequently used and were considered to be promoting collaborative skills. .Our observations suggest that the form and purpose of group tasks to support learning in science could profitably be discussed and developed across the science team with the aim of changing practice in such a way that ensures that an effective, coherent, articulate and informed approach to collaborative learning is developed (3.4.1 c).

Specific science knowledge and skills: The basic structure and content of the curriculum in the certificate years S1 to S6 had not been changed as a consequence of the introduction of ICT so the main content of the curriculum was therefore being presented largely as before, with the structure and progression unchanged. The staff anticipated that the ICT resources would allow the pupils faster access to information, a wider field of information and increased motivation through the quality of information accessed which would enhance conceptual understanding, which, in turn, would form the basis of more effective learning in later school courses (3.4.1 d).

What was the impact of ICT use on the pupils' development of skills and knowledge in science?

There is an aspiration and an expectancy on the part of all involved in the Science Project that the pupils' understanding of science will be improved. However, although there was evidence of increased enthusiasm for science, there is at this early stage in the uses of ICT little objective evidence of the extent to which the pupils have actually acquired more robust knowledge as a consequence of the use of ICT. Software is available to support virtual experiments. Some evidence from the pupil interviews suggested that the pupil responses to virtual experiments were very positive and pupils generally indicated enthusiasm for learning science through using the technology for information finding, data logging and simulations (3.4.2 a). The interview data from the pupils confirmed the teachers' view that the pupils' attention and concentration span was improved through both seeing the teachers' comments coming up on the whiteboard, and seeing information presented in a more accessible form than board diagrams or teachers' verbal explanations could provide (3.4.2 b). Some reservations were expressed by Learning Support about the extent to which the different ICT based forms of either transmission of information, or the forms of it's re-presentation back to the teacher did actually assist the slow learners significantly in their understanding of intrinsically difficult topics. However, teachers did suggest in interviews that with the use of ICT in some lessons, the burden on teacher dominated transmission and directions were reduced and more time could be available for talking and discussing difficulties with pupils (3.4.2 c).

What has been the impact on the teaching and learning activities? (3.5)

Teaching was judged to be enhanced through specific ICT use: e.g. the data projector and interactive whiteboard were observed being used to make illustrative points through good quality photographic images and an appropriate software programme. It was possible too for pupils to access software which was also on the teacher's board, and they could be talked through the software. The use of the technology also allowed a greater variety of teaching activities which actively involved the pupils (3.5.1).

Both teaching and learning were judged to be enhanced through e.g. the use of revision software; through enhanced visual presentations of complex concepts (e.g. chemical bonding); for enhancing data analysis activities through data-logging (3.5.2).

Neither teaching or learning were judged to be enhanced when there was a breakdown in the technology, or when learning activities were unfocussed on science issues, being more dominated by the ICT procedures (3.5.3).

Overall we judged that the main areas of enhancement were in the traditional activities of the teachers to enhance knowledge coverage and uptake by the pupils. Several of the Project Team Leaders indicated their growing awareness of the need to start thinking about different strategies for learning and hence for teaching. They had come to realise that the use of technology did not in itself deliver change in practices; once change in practices were identified and planned through, the technologies were merely the tools to make the changes possible (3.6).

Section 4: The Perspectives of the Pupils

The pupil interviews

Overall, twenty nine pupils (20 male, 11 female) were interviewed: 20 in May 2003, 16 in May 2004 – seven were interviewed on both occasions. They covered all year groups and a range of abilities. Pupils in S1/2 were interviewed in pairs, the older pupils as individuals (4.2.2).

What have been their experiences with the technology at home?

Although all pupils said that they had access to a computer at home, some were reportedly 'broken', and had been for sometime, while some pupils had 'privileged access' – i.e. access to family computers in addition to their own personal laptops. Factors such as informed support from or competition with siblings and parents, personal preferences for writing by hand, in addition to beliefs regarding the benefits of word processing were all reported as determining the range and frequency of PC use at home (4.3.1 a). Home computers were regularly used for homework, particularly using the Internet and word processing, including 'cut and paste'. Most pupils indicated a clear understanding of plagiarism (4.3.1 b). The school web page provides a very useful resource which could enhance partnerships between the home and the school but its use was still at an early stage and few pupils reported using it (4.3.1 c).

Where did pupils acquire their ICT skills?

Pupils had mainly acquired their ICT skills in the home or primary school, and in IT classes, computing or business studies in the secondary school (4.3.2). When asked about experiences of using ICT as a learning tool in the school subjects, Modern Languages, Geography and English were mentioned. The interactivity of the programmes was the key factor in generating enthusiasm and approval (4.4).

What have been the benefits and disadvantages to their learning in science lessons?

Pupils reported more control and independence, the aiding of comprehension and recall of information, and increased understanding prompted by animated or three dimensional representations. Clearly the technology had contributed to assisting teachers and pupils to bring variation to what had been hitherto a predominantly talk and text dominated situation. However, as one sixth year pupil observed, having a whiteboard did not necessarily mean a teacher would teach better. The disadvantages identified were associated with the early difficulties with the technology (4.5 a).

Was the technology used to support pupils with learning difficulties?

The classroom observations and pupil interviews indicated that all the pupils were reporting benefiting in some degree from the way in which information – either in the form of knowledge or instructions – became more engaging and accessible through the use of the computers. However, the subject materials and the programmes and sites used were not in any way differentiated and all pupils read or interacted with the same level of material. No activities or programmes were seen in the observations or reported by pupils which were specifically designed to be helpful to slow learners or those with particular learning difficulties. The allocation of additional time to the Learning Support staff for consultations could support the improvement of this situation in the future, and ultimately lead to a more coherent approach to learning within and across the different subject areas (4.5 c).

Did the laptops have a beneficial effect on pupils' attitudes and motivation?

Overall, pupils from across year groups indicated that they benefited from the use of lap tops in science subjects. With a few exceptions, most pupils were positive and enthusiastic (4.5 d).

Did the technology detract from practical engagement in science?

Information in the first round of interviews had suggested that there might be a trend away from interest in real hands-on science in favour of the clean and certain world of the simulations. However, specific questioning of pupils in the second round indicated there was still an enthusiasm for hand-on experiments. Most pupils appeared to favour the sometimes 'smelly' and 'messy' practical experiments undertaken in class. Older pupils (i.e. those in S6) were found to appreciate the way in which use of lap top programmes in conjunction with hands on experiments could facilitate and ease their understanding of science (4.5 e).

Section 5: Teachers as Learners and the Learning School

Achievement of the Aims of the School Project

Had the aims of the school project been achieved?

It seemed to us there was much evidence that the technology had been very successfully introduced into the science classrooms of the school, from S1 through to S6, and was in regular use (5.3.1).

What factors had been the key to the successful initial adoption of ICT?

Two general strengths had contributed to this success: the initial thought and planning given to the introduction of all staff to the technology; and the initial processes of regular demonstrations, modelling sharing and practising. A Project Team Leader gave the following pointers to other school staff considering a similar initiative:

plan in detail for the integration of ICT into the lessons, not used as an add on;
involve all staff not just the enthusiastic;
ensure continuity of coverage across the curriculum and the year;
embed and integrate into pupil activities for active learning;
start with staff development and invest a lot of time in it;
after school staff sessions are important;
share experience and expertise in different areas across the team;
develop sharing and trust. Trust is easier with respect to the use of technologies – less so in areas related to teaching and learning. People are wary of saving 'I can't do this style of teaching or support learning like this'. (5.3.2)

Had the project promoted the development of new ways of learning and teaching?

There was rather less evidence that the project had succeeded in the aspiration set out in the original project proposals that the use of the technology would usher in 'new ways of learning and teaching in science subjects' as indicated in the original Project proposals (5.3.3).

Had the Project had an effect on attainment in science?

We could not find nor was it possible for us to generate data within this project which could link any changes in attainment or in the uptake of science to the use of the ICT. The project was in far too early a stage and other factors within the school were more likely to be influencing any variation from previous years (5.3.4).

What factors were likely to influence the future developments?

Staff identified the following as influencing their further developments: the fading of the initial novelty factor; the need for materials to be more fully integrated into the curriculum rather than slotting into an already existing curriculum; the extent to which staff were feeling overall 'innovation overload'; their identification of a need for a more structured approach particularly in the tasks given to pupils (5.4).

What models of learning would be appropriate for future consideration?

It is increasingly recognised that the improved learning and the transformation of teaching are not guaranteed merely by the presence of the technology. In our view there had been no opportunity or context for science staff to overview the aims and objectives of the range of different initiatives which had already been introduced into the school (CASE, VAK, Critical Skills etc.) to see the common threads and to capitalise on the commonalities in order to optimise their understanding, save duplication and - most importantly of all - to give a coherent experience of education within the school to the pupils (5.5).

At the end of the first phase of the project (Dec. 2003) we generated some models of enhanced teaching and learning based on the activities with we had seen in our classroom observations and structured in the form of Critical Skills principles (Appendix 1) which we hoped might offer a context for the staff to discuss aspects of ICT and learning as well as teaching (5.5.1). At this stage we were advised that although individual teachers would continue to develop their skills and different ways of using the ICT in their own style and for their own specific purposes, there was unlikely to be further significant developments in the thinking and discussions of the whole staff group in relation to the overall aims of the project during the management reorganisation period (5.5.2).

The Impact of the School's CPD Programme

What has been the impact of the school's CPD programme on teacher competence and confidence in using ICT in science and in developing the school learning community?

A questionnaire was devised in consultation with the ICT co-ordinator, and was circulated to all the school staff in the Spring of 2004. 24 teachers (28% of the teaching complement) responded, representing 16 departments including Guidance and Support for Learning (5.6.1). The responses from the staff sample indicated that their basic computer-using skills were well consolidated. The access to ICT in their classrooms varied widely but the figures appeared to indicate that the level of regular use in classrooms was low. The obstacle most frequently identified to further use was access, either to the equipment per se or to the classroom beforehand to set up in preparation for the next lesson (5.6.2 a).

What were the staff experiences of Staff development?

No staff member had undertaken Masterclass training but most had completed their NOF training. Only a minority indicated that they found it a positive and useful experience (5.6.2 b)

What forms of CPD did they value?

The form of staff development most frequently engaged with (96%) - and in this questionnaire the most highly rated in terms of usefulness - was informal discussions with immediate subject colleagues, and for a minority, (25%), meetings with subject colleagues from other schools. For 88% of the staff, 'exploring through trial and error in my own classroom' received the next highest rating of usefulness. While experienced by a majority, (75%), whole school in-service with outside input was rated as the least useful of all the forms of staff development listed, with external award bearing courses and subject-oriented professional journals also rated on the 'not useful' side of the scale (5.6.2 c).

What are the key characteristics of staff learning?

The model of teacher learning which emerges from a variety of studies indicates that the development of teachers' knowledge and use of ICT is an individual and subject centred model. The planning and leadership associated with the early phase of the project (Section 2) allied with the challenge presented by the novelty of the ICT use allowed a small but powerful learning community to develop within the science department. For developments to move forward in a unified way however, strong curriculum or management leadership would be required (5.7.2).

The Developing School

How can the school develop further as a learning community?

The model of planning, inclusion, support, practice and modeling adopted by the Science Project Team worked well in creating a productive 'learning community' of teachers who successfully engaged with ICT and began its regular use in classrooms. However, it is not certain that all staff would wish to change in any radical way, and subsequently individualistic developments were being adopted by most of the science teachers now they were familiar with the technology. The literature on school development suggests that at this stage the school needs to work together on developing a unifying statement about the features of its philosophy and practices in learning and teaching. The Critical Skills Coaching Kit offers a model of key ideas for developing coherent classroom approaches (5.7.3). The school was identified as being at level 3 on the Mooij and Sweets (2002) model of school development with respect to ICT implementation (5.7.4)

Why should secondary schools seek to develop innovative pedagogy ?

Arguments are advanced for the adoption of the innovative pedagogies which the powers of ICT enable. The development of appropriate innovative pedagogies must involve teachers in the two way process of bridging theory and practice. This project demonstrated just how challenging and professionally stimulating such pedagogy-led technological innovation can be when the ownership of development is put in the hands of a team of teachers committed to curriculum development as a vehicle for their own professional development.

PDF Icon ICT in Science Project Evaluation Report